Method, electronic device and computer program product for updating a dataset

Abstract

Embodiments of the present disclosure relate to a method, an electronic device and a computer program product for updating a dataset. The method comprises obtaining a reference dataset comprising a plurality of reference images, wherein each reference image has a soft label and the soft label represents a probability of each reference image belonging to each of a plurality of target classes. The method can further comprise determining an image soft label of an input image according to a similarity between the input image and each reference image. The method further comprises adding the input image and the image soft label of the input image to the reference dataset to update the reference dataset.

Description

Technical Field

[0001] Embodiments of this disclosure relate to the field of data processing, and more specifically, to methods, electronic devices, and computer program products for updating datasets. Background Technology

[0002] In recent years, small sample datasets have been increasingly used, such as for training machine learning models. For example, in edge computing, end users typically have access to only a limited amount of data, and even then, edge devices still need to update their models with limited sample data. However, machine learning models are prone to overfitting because they are based on bias distributions formed from only a small number of samples. Therefore, learning from a limited number of samples is challenging and remains a significant problem in the field of machine learning. Summary of the Invention

[0003] Embodiments of this disclosure provide a method, electronic device, and computer program product for updating a dataset.

[0004] According to a first aspect of this disclosure, a method for updating a dataset is provided. The method includes obtaining a reference dataset comprising a plurality of reference images, each reference image having a soft label, and the soft label representing the probability that each reference image belongs to each of a plurality of target categories. The method may further include determining an image soft label for an input image based on the similarity between a received input image and each reference image. The method further includes adding the input image and its image soft label to the reference dataset to update the reference dataset.

[0005] According to a second aspect of this disclosure, an electronic device is provided. The electronic device includes at least one processor; and a memory coupled to the at least one processor and having instructions stored thereon, the instructions causing the device to perform actions when executed by the at least one processor, the actions including: acquiring a reference dataset comprising a plurality of reference images, wherein each reference image has a soft label, and the soft label represents the probability that each reference image belongs to each of a plurality of target categories; determining an image soft label for an input image based on the similarity between a received input image and each reference image; and adding the input image and the image soft label of the input image to the reference dataset to update the reference dataset.

[0006] According to a third aspect of this disclosure, a computer program product is provided, which is tangibly stored on a non-volatile computer-readable medium and includes machine-executable instructions that, when executed, cause a machine to perform the steps of the method in the first aspect of this disclosure. Attached Figure Description

[0007] The above and other objects, features and advantages of this disclosure will become more apparent from the accompanying drawings, in which like reference numerals generally denote like parts.

[0008] Figure 1 The illustration shows a schematic diagram of an example environment 100 in which the devices and / or methods according to embodiments of the present disclosure may be implemented;

[0009] Figure 2 A flowchart illustrating a method 200 for updating a dataset according to an embodiment of the present disclosure is shown;

[0010] Figures 3A-3D Taking the distillation dataset as an example, this paper illustrates a schematic diagram of multiple distillation reference images with soft labels and their soft labels.

[0011] Figure 4 The illustration shows a schematic diagram of a method 400 for determining soft tags for an input image according to an embodiment of the present disclosure;

[0012] Figure 5 A schematic flowchart illustrating the determination of a soft tag according to an embodiment of the present disclosure is shown.

[0013] Figure 6 A schematic block diagram of an example device 600 suitable for implementing embodiments of the present disclosure is shown.

[0014] In the various figures, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0015] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0016] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0017] In recent years, few-shot datasets have been increasingly used. For example, researchers are exploring various model training methods based on small datasets. One approach is to optimize the gradient descent process to improve the model's learning efficiency; however, this method requires improvements to traditional supervised learning training methods and introduces many learning parameters. Another approach is to use methods such as Generative Adversarial Networks (GANs) to generate more usable sample images; however, this method requires designing complex models and loss functions to generate more usable sample images.

[0018] To address at least the aforementioned and other potential problems, embodiments of this disclosure propose a method for updating a dataset. The method includes acquiring a reference dataset comprising multiple reference images, each reference image having a soft label representing the probability that each reference image belongs to each of a plurality of target categories. The method may further include determining an image soft label for an input image based on the similarity between a received input image and each reference image. The method also includes adding the input image and its image soft label to the reference dataset to update the reference dataset. The method for updating a dataset based on embodiments of this disclosure does not require improvements to traditional supervised learning training methods, nor does it introduce learning parameters, and it does not require the design of complex models and loss functions. It can significantly reduce computational costs, improve model learning efficiency, and has broad application prospects.

[0019] The embodiments of this disclosure will now be described in further detail with reference to the accompanying drawings, wherein... Figure 1 A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown.

[0020] exist Figure 1 In Example 100, a reference dataset 110 is shown. Reference dataset 110 may, for example, include a pre-generated dataset. This dataset can be generated by a trained neural network model, or it can be generated by pre-annotating reference images in the dataset; this disclosure does not limit the method of generating the reference dataset. Reference dataset 110 may include multiple reference images, such as reference image 1, reference image 2, ..., reference image m, etc. This disclosure does not limit the number of reference images in reference dataset 110.

[0021] In one embodiment, each reference image in the reference dataset 110 has a soft label, and the soft label represents the probability that each corresponding reference image belongs to each of a plurality of target categories. In one embodiment, the reference dataset 110 may include a distilled dataset, and accordingly, each reference image in the reference dataset 110 may include a distilled reference image, and each distilled reference image includes a soft label, which represents the probability that each corresponding distilled reference image belongs to each of a plurality of target categories. In another embodiment, the reference dataset 110 may include an undistilled dataset, and accordingly, each reference image in the reference dataset 110 may include an undistilled reference image, and each reference image includes a soft label, which represents the probability that each corresponding reference image belongs to each of a plurality of target categories.

[0022] like Figure 1 Example 100 shown can be executed by a computing device (not shown). The computing device includes, but is not limited to, personal computers, server computers, handheld or laptop devices, mobile devices (such as mobile phones, personal digital assistants (PDAs), media players, etc.), multiprocessor systems, consumer electronics, wearable electronic devices, smart home devices, minicomputers, mainframe computers, edge computing devices, and distributed computing environments including any of the above systems or devices. The computing device can store the aforementioned reference dataset 110 and can receive input image 120. The computing device can determine image soft labels for input image 120 based on the similarity between the received input image 120 and each reference image i in the reference dataset 110. Further, the computing device can add the input image 120 and the determined image soft labels together to the reference dataset 110 to obtain an updated dataset 130. Therefore, based on the soft labels of the reference images in the reference dataset 110, the computing device can assign new image soft labels to the input image 120, and add the input image 120 and its assigned image soft labels to the reference dataset 110, thereby obtaining an updated dataset 130. This allows full utilization of the inherent characteristics of the data in the prior reference dataset to obtain an enhanced dataset. This method can be applied in various scenarios, such as small sample dataset augmentation, to improve the accuracy and efficiency of model learning.

[0023] although Figure 1 Only one input image 120 is shown in the diagram. Those skilled in the art should understand that the number of input images required can be received according to the actual application scenario of the dataset, thereby obtaining the desired updated dataset. This disclosure does not limit the number of input images 120.

[0024] In one embodiment, the reference image dataset 110 includes a distillation dataset, which can be generated by a neural network model 180. The neural network model 180 can include any known and / or future-developed machine learning model, and this disclosure does not limit the specific structure and architecture of the neural network model 180. Preferably, in one embodiment, the neural network model 180 can include a capsule neural network model. Furthermore, in one embodiment, a computing device or another computing device (not shown) can fine-tune the neural network model 180 based on an updated dataset 130, thereby making the reference images in the generated reference dataset 110 more accurate and robust.

[0025] The method for updating a dataset based on embodiments of this disclosure does not require improvements to traditional supervised learning training methods, nor does it introduce learning parameters. Furthermore, it eliminates the need to design complex models and loss functions, significantly reducing computational costs, improving model learning efficiency, and possessing broad application prospects. Therefore, the dataset updating method according to embodiments of this disclosure can also be deployed in edge devices, enabling dataset augmentation with higher security, lower latency, and higher reliability. This allows for model updates at the edge device, thereby improving the model's application capabilities and processing efficiency, and significantly enhancing the computing power of the edge device.

[0026] The above combination Figure 1 A block diagram of an example system 100 in which embodiments of the present disclosure can be implemented is described. The following is in conjunction with… Figure 2 A flowchart describing a method 200 for updating a dataset according to embodiments of the present disclosure is provided. Method 200 can be... Figure 1 It is executed on the computing device (not shown) described herein and any suitable computing device.

[0027] At box 202, the computing device can acquire a reference dataset 110, which may include multiple reference images, such as reference image 1, reference image 2, ..., reference image m, etc. Each reference image i has a soft label, where the soft label represents the probability that the corresponding reference image belongs to each of a plurality of target categories. In one embodiment, the reference dataset 110 may include a distilled dataset; correspondingly, each reference image in the reference dataset 110 may include a distilled reference image, and each distilled reference image includes a soft label representing the probability that the corresponding distilled reference image belongs to each of a plurality of target categories. In another embodiment, the reference dataset 110 may include an undistilled dataset; correspondingly, each reference image in the reference dataset 110 may include an undistilled reference image, and each reference image includes a soft label representing the probability that the corresponding reference image belongs to each of a plurality of target categories.

[0028] Figures 3A-3D Using reference dataset 110 as an example, this paper illustrates a schematic diagram of multiple distillation reference images with soft labels and their soft labels. Figure 3A Using the distillation reference image in the image as an example, Figure 3A The distillation reference image shown has a soft label L represented in probabilistic form. soft [0,0.82,0,0,0,0,0.03,0.11,0,0.04], where the values ​​in the soft label, from left to right, correspond to the target categories of numbers 0-9, and the soft label L soft express Figure 3A The distillation reference image in the image has the following probabilities: 0 for target category 0, 0.82 for target category 1, 0 for target category 2, 0 for target category 3, 0 for target category 4, 0 for target category 5, 0 for target category 6, 0.03 for target category 7, 0.11 for target category 8, and 0.04 for target category 9. Furthermore, the sum of the values ​​of all elements in the soft label is 1. For example, soft label L... soft In the equation, 0.82 + 0.03 + 0.11 + 0.04 = 1.

[0029] Similarly, Figures 3B to 3D Distillation reference image and corresponding soft label L soft Similar Figure 3A For the sake of brevity, the descriptions in the text will not be repeated here. Furthermore, those skilled in the art should understand that, although... Figures 3A-3DThe example uses numbers, and the target category includes the numbers 0-9. However, depending on the actual classification task, any target category and corresponding distillation reference image can be used. This disclosure does not limit the target category.

[0030] Continue to refer to Figure 2 At box 204, the computing device can determine the image soft label L of the input image 120 based on the similarity between the received input image 120 and each reference image. soft,i The specific steps for determining the soft label of input image 120 will be described in detail below. It should be understood that by calculating the similarity between input image 120 and each reference image in the reference image set 120, a soft label can be determined for the input image based on the inherent characteristics of the reference images, thereby updating the dataset and further enhancing the dataset.

[0031] At box 206, the computing device can input image 120 and the image soft label L of the determined input image. soft,i Add to reference dataset 110 to update the reference dataset and obtain the updated dataset 130.

[0032] The method 200 for updating a dataset based on embodiments of this disclosure does not require improvements to traditional supervised learning training methods, nor does it introduce learning parameters. Furthermore, it eliminates the need to design complex models and loss functions, significantly reducing computational costs, improving model learning efficiency, and possessing broad application prospects. Therefore, the method for updating a dataset according to embodiments of this disclosure can also be deployed in edge devices, enabling dataset augmentation with higher security, lower latency, and higher reliability. This allows for model updates at the edge device, thereby improving the model's application capabilities and processing efficiency, and significantly enhancing the computing power of the edge device.

[0033] The following will combine Figure 4 A flowchart describes a method 400 for determining image soft tags for an input image according to an embodiment of the present disclosure. Method 400 can be... Figure 1 It is executed on the computing device (not shown) described herein and any suitable computing device.

[0034] like Figure 4 As shown, at box 402, the computing device determines the weight w for each reference image based on the similarity between the input image 120 and each reference image in the reference dataset 110.

[0035] For example, taking input image 120 as image i as an example, the weight w for each reference image j is... i,j It can be as shown in Formula 1 below:

[0036]

[0037] Where P represents the reference dataset, and sim<…,…> represents the similarity between two images. In this example, cosine similarity is used for calculation. Those skilled in the art will understand that other similarity calculation methods can be used, and this disclosure does not limit such methods.

[0038] After determining the weight w for each reference image j i,j After that, as Figure 4 As shown in box 404, the computing device can determine the soft label L for each reference image j. soft,j and the calculated weight w i,j Determine the soft label L of the input image i. soft,i .

[0039] Specifically, the computing device can utilize this weight w i,j For each corresponding reference image j, the soft label L soft,j Weighting is performed to obtain a weighted soft label L for each reference image j. soft,wj Then, the computing device can process multiple (e.g., all) weighted soft labels L soft,wj The probabilities of the corresponding target categories in the input image i are summed to determine the soft label L of the input image i. soft,i .

[0040] The above steps can be represented by Formula 2, that is:

[0041]

[0042] For example, let's take a reference dataset that includes three reference images, each with a corresponding soft label for one of the three target categories. For instance, the reference dataset includes: a first reference image A1, and the first reference image A1 has a soft label L... soft,1 = [0.3, 0.5, 0.2]; second reference image A2, and the first reference image A2 has a soft label L. soft,2 = [0.6, 0.1, 0.3]; and a third reference image A3, wherein the first reference image A3 has a soft label L. soft,3 = [0.4, 0.2, 0.4]. Assume that according to Formula 1 above, the values ​​for each reference image A have been obtained. k The weights are as follows: for the first reference image A1, the reference weight w1 = 0.3; for the second reference image A2, the reference weight w2 = 0.4; and for the third reference image A3, the reference weight w3 = 0.3. Therefore, for each corresponding reference image A... k Soft label Lsoft,k Perform weighting and then combine the three weighted soft labels L soft,wk The probabilities of the corresponding target categories in the input image i are summed to determine the soft label L of the input image i. soft,i .

[0043] That is, in the example above, the weighted soft label L corresponding to the first image A1 soft,w1 =0.3×[0.3,0.5,0.2]=[0.09,0.15,0.06]; corresponding to the weighted soft label L of the second image A2. soft,w2 =0.4×[0.6,0.1,0.3]=[0.24,0.04,0.12]; corresponding to the weighted soft label L of the third image A3. soft,w3 =0.3×[0.4,0.2,0.4]=[0.12,0.06,0.12]. Correspondingly, the soft label L of the input image. soft,i =[0.09+0.24+0.12,0.15+004+0.06,0.06+0.12+0.12]=[0.45,0.25,0.3].

[0044] Based on the above description, we have introduced the soft label L of the input image i. soft,i The process of determining the soft label L of the input image i. soft,i Then, the input image i and its corresponding soft label L can be... soft,i They are added together to the reference dataset, thereby updating the dataset and, more specifically, enhancing it.

[0045] In one embodiment, the reference dataset 110 includes a distillation dataset, which includes multiple distillation reference images obtained by distilling the original dataset. The soft label of each distillation reference image represents the probability that the corresponding reference image belongs to each of the multiple target categories. Furthermore, the soft label of each distillation reference image is obtained based on the probability that each original image in the original dataset belongs to each of the multiple target categories.

[0046] In one embodiment, where the reference dataset 110 includes a distillation dataset, the distillation dataset may be based on a neural network model (e.g., Figure 1 The neural network model 180 is generated. In one embodiment, the neural network model 180 may include a capsule neural network model. Specifically, in one embodiment, when generating the distillation dataset, the computing device may acquire a raw dataset, wherein the raw dataset includes multiple raw images. The computing device may further utilize the capsule neural network model (e.g., ... Figure 1The neural network model 180 in the dataset performs distillation on the original dataset to obtain a distilled dataset. Each distilled reference image has a corresponding soft label, and the corresponding soft label represents the probability that the corresponding distilled reference image belongs to each of the multiple target categories.

[0047] For example, still combined Figures 3A-3D Let's take an example to illustrate. Figures 3A-3D Taking reference dataset 110 as an example, this paper illustrates a schematic diagram of multiple distillation reference images with soft labels and their soft labels. Figure 3A The distillation reference image shown has a soft label L represented in probabilistic form. soft [0,0.82,0,0,0,0,0.03,0.11,0,0.04], where the values ​​in the soft label, from left to right, correspond to the number categories 0-9, and the soft label L soft express Figure 3A In the distillation reference image, the probability of belonging to target category 0 is 0, the probability of belonging to target category 1 is 0.82, the probability of belonging to target category 2 is 0, the probability of belonging to target category 3 is 0, the probability of belonging to target category 4 is 0, the probability of belonging to target category 5 is 0, the probability of belonging to target category 6 is 0.03, the probability of belonging to target category 7 is 0.11, the probability of belonging to target category 8 is 0, and the probability of belonging to target category 9 is 0.04. Furthermore, the sum of the values ​​of all elements in the soft label is 1. For example, soft label L... soft In the equation, 0.82 + 0.03 + 0.11 + 0.04 = 1.

[0048] Similarly, Figures 3B to 3D Distillation reference image and corresponding soft label L soft,i Similar Figure 3A For the sake of brevity, the descriptions in the text will not be repeated here. Furthermore, those skilled in the art should understand that, although... Figures 3A-3D The example uses numbers, and the target category includes the numbers 0-9. However, depending on the actual classification task, any target category and corresponding distillation reference image can be used. This disclosure does not limit the target category.

[0049] In one embodiment, after receiving the desired number of input images, the computing device can update the reference dataset. Furthermore, in this updated dataset, each image can have a corresponding soft label and a hard label. Further, this updated dataset can be used by the computing device or another computing device to fine-tune the neural network model 180 that generated the aforementioned distillation dataset. During fine-tuning, the label type used in the training data is not limited. For example, when the computing device performing the fine-tuning detects that the current input sample image has a hard label, the computing device can set the loss function for the neural network model to a first type of loss function, such as the argmax function; when the computing device performing the fine-tuning detects that the current input sample image has a soft label, the computing device can set the loss function for the neural network model to a second type of loss function, such as the difference between the mutual information of the original dataset X and the distillation dataset Z and the similarity between the original dataset X and the distillation dataset Z, i.e., ... Here, Lor(Z) represents the similarity between the original dataset and the distillation dataset, and I(X,Z) represents the mutual information between the original dataset and the distillation dataset. This loss function will be described below.

[0050] The following will combine Figure 5 This paper describes the generation of distillation images in a distillation dataset and the method for determining the soft label for each distillation image. Figure 5 A schematic block diagram is shown showing the generation of corresponding soft labels for each distillation reference image.

[0051] According to embodiments of this disclosure, distillation reference images can be generated using a capsule neural network model. Capsule neural networks are a high-performance network architecture in the field of machine learning. The entropy of weights in the dynamic routing of capsule neural networks is very low. Embodiments of this disclosure fully utilize this characteristic of capsule neural networks, directly using the output features of the dynamic routing of the capsule neural network model to calculate the soft label for each distillation image in the distillation dataset. This significantly reduces computational costs and improves computing power and efficiency. The following will combine... Figure 5 The process of determining the soft label for the distillation image is described in detail.

[0052] When acquiring the distillation dataset, the computing device can input each raw image from the original dataset into the capsule neural network model (e.g., Figure 1 In the capsule neural network model 180, the output feature set (such as...) is obtained. Figure 5 The matrix M1 shown is used to illustrate this. The computing device can acquire the capsule feature set of the capsule neural network (e.g., ...). Figure 5The computing device can obtain a distillation dataset based on the output feature set and the capsule feature set, as shown in matrix M2. For example, the computing device can obtain an intermediate feature set based on the output feature set and the capsule feature set, and further process the intermediate feature set by a fully connected layer in the capsule neural network model to obtain the final output distillation image.

[0053] In one embodiment, when a computing device inputs each raw image from the raw dataset into a capsule neural network model, it can obtain the output of a higher-level capsule (e.g., the first capsule layer) in the capsule neural network model and use that output as a set of output features. Figure 5 As shown, in one embodiment, the output feature set may include an output feature matrix M1, where the number of rows in the output feature matrix M1 corresponds to the number of original images n in the original dataset, and the number of columns in the output feature matrix M1 corresponds to the number of capsules k in the capsule neural network model. Furthermore, each row in the output feature matrix M1 represents the image feature of the corresponding original image Nj, and each element in that row can represent the output probability of the corresponding original image Nj for each capsule (e.g., obtained by normalizing the image features of each row). For example, as... Figure 5 As shown, for the original image N1 corresponding to the first row, the elements (a11, a12, a13, ..., a1k) in the first row represent the image features of the original image N1, and each element a1j (1≤j≤k, and j is a positive integer) represents the output probability of the original image N1 for the capsule dj in the corresponding capsule neural network model.

[0054] Figure 5 The document further illustrates a capsule feature set, for example, in one embodiment, the capsule feature set can be represented as a matrix M2, where the number of rows corresponds to the number of capsules in the capsule neural network model, and the number of columns corresponds to i, indicating that each capsule has an i-dimensional vector. For example, as... Figure 5 Specifically, for the first row, the vector of the first capsule in the capsule neural network model is (v11, v12, ..., v1i). In one embodiment, this matrix M2 can be obtained through learning.

[0055] The computing device can output a feature matrix M1 and multiply it with the capsule feature matrix M to obtain a feature matrix M. This feature matrix represents the intermediate features of the original image. Furthermore, the feature matrix M is further processed by the fully connected layer in the capsule neural network model, and the capsule neural network model can output a distilled image.

[0056] like Figure 5As shown, for the output feature matrix M1, each column vector pj of the feature matrix M1 can represent the output probability of each original image in the original dataset at its corresponding capsule di (e.g., obtained by normalizing the image features of each row). Since the capsules correspond to the target categories, it can be understood that the elements in each column vector of the output feature matrix M1 represent the probability that each original image in the original dataset belongs to that corresponding category. For example, taking the first column of the output feature matrix M1 as an example, element a11 can represent the output probability that the first original image belongs to the first target category (corresponding to capsule d1), element a21 can represent the output probability that the second original image belongs to the first target category (corresponding to capsule d1), and so on, element an1 can represent the output probability that the nth original image belongs to the first target category (corresponding to capsule d1). For simplicity, the elements in the other columns Pi of the output feature matrix M1 will not be described one by one.

[0057] Based on the column vectors in the output feature matrix M1, a soft label for each distillation reference image can be obtained. Specifically, in one embodiment, the computing device can obtain the column vector corresponding to the target category in the output feature matrix M1. For example, the computing device can obtain the column vector pj corresponding to the target category Cj (corresponding to capsule dj, and 1≤j≤k, and j is a positive integer) in the output feature matrix M1, where each element in the column vector pj represents the output probability that each original image belongs to that target category. For example, the output probability can be represented as (a1j, a2j, a3j, ..., anj). The computing device can determine the probability value (a1j, a2j, a3j, ..., anj) corresponding to each distillation image based on the elements in the column vector pj. For example, the computing device can first determine which distillation image each original image belongs to, and then determine the probability value corresponding to each distillation image. The computing device can further determine the probability of each distillation image under the corresponding target category dj based on the probability value corresponding to each distillation image, as the soft label of the distillation image.

[0058] Specifically, in one embodiment, the computing device can determine the distillation image corresponding to each probability value in the column vector pj. In an exemplary example, taking the generation of L distillation reference images from n original images as an example, for the j-th target category, the column vector pj is obtained, and based on this column vector pj, the probability value corresponding to each distillation reference image can be determined. For example, assuming the probability corresponding to the first distillation reference image I1 is a1j, the probability corresponding to the second distillation reference image I2 is a2j, the probability corresponding to the third distillation reference image I3 is a3j, and so on, the probability corresponding to the L-th distillation reference image IL is aLj. It should be understood that the above assumptions are merely illustrative for purposes of explanation.

[0059] The computing device can determine the number of original images corresponding to each probability value in the column vector based on the number of original images corresponding to a given distillation reference image. Continuing with the example above, assume that the first distillation reference image I1 corresponds to n1 original images, the second distillation reference image I2 corresponds to n2 original images, the third distillation reference image I3 corresponds to n3 original images, and so on, with the Lth distillation reference image IL corresponding to nL original images, and n1 + n2 + n3 + ... + nL = n. Therefore, based on each probability value and the number of original images corresponding to each probability value, the probability of each distillation reference image in the corresponding target category can be determined.

[0060] For example, taking the above example, for each distilled image z, its probability in the j-th target category can be calculated using Formula 3:

[0061] Pro zj =nz×azj / n1×a1j+n2×a2j+n3×a3j……+nL×aLj (Formula 3)

[0062] Among them, Pro zj Let z be the probability of each distillation reference image z in the j-th target category, and azj be the probability value of the corresponding distillation reference image z in that column vector. Furthermore, 1 ≤ z ≤ L, and z is a positive integer.

[0063] The computing device can calculate and determine a corresponding probability value Pro for each distillation reference image for each target category. zj The determined probability values ​​are combined to form a soft label corresponding to each distillation reference image. z1 Pro z2 ,……Pro zk ]. Where k is the number of target categories.

[0064] Furthermore, to improve computational efficiency, the computing device can select probability values ​​greater than a probability threshold from the selected column vectors, determine the number of original images corresponding to each distillation reference image for each probability value from the selected probability values, and determine the probability of each distillation reference image under the corresponding target category based on each probability value and the number of original images corresponding to each probability value. The specific determination process can be referred to the description above, and will not be repeated here for the sake of brevity.

[0065] In one embodiment, the distillation dataset described above can be obtained using a capsule neural network model. During the training of this capsule neural network, its loss function is correlated with the difference between the following parameters: the mutual information between the original dataset and the distillation dataset; and the similarity between the original dataset and the distillation dataset.

[0066] In one embodiment, the mutual information between the original dataset X and the distilled dataset Z can be expressed as the following formula 4:

[0067]

[0068] in,

[0069] Furthermore, the similarity between the original dataset and the distilled dataset can be expressed as follows: Formula 5:

[0070]

[0071] in,<i,j> This represents the similarity between each corresponding image in two datasets.

[0072] Therefore, the loss function used to train the capsule neural network model can be expressed as:

[0073]

[0074] Furthermore, as described above, when fine-tuning the capsule network using the updated dataset, the loss function defined in Equation 6 can also be used for fine-tuning.

[0075] The method for updating a dataset based on embodiments of this disclosure does not require improvements to traditional supervised learning training methods, nor does it introduce learning parameters. Furthermore, it eliminates the need to design complex models and loss functions, significantly reducing computational costs, improving model learning efficiency, and possessing broad application prospects. Therefore, the dataset updating method according to embodiments of this disclosure can also be deployed in edge devices, enabling dataset augmentation with higher security, lower latency, and higher reliability. This allows for model updates at the edge device, thereby improving the model's application capabilities and processing efficiency, and significantly enhancing the computing power of the edge device.

[0076] Figure 6 A schematic block diagram of an example device 600 that can be used to implement embodiments of the present disclosure is shown. A computing device for performing methods according to embodiments of the present disclosure can be implemented using device 600. As shown, device 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) 602 or loaded from storage unit 608 into random access memory (RAM) 603. Various programs and data required for the operation of device 600 may also be stored in RAM 603. CPU 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0077] Multiple components in device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of monitors, speakers, etc.; storage page 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0078] The various processes and procedures described above, such as method 200 for updating a dataset and related processes (e.g., process 400), can be executed by processing unit 601. For example, in some embodiments, method 200 for updating a dataset and related processes (e.g., process 400) can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed on device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by CPU 601, one or more actions of method 200 for updating a dataset and related processes (e.g., process 400) described above can be performed.

[0079] This disclosure can be a method, apparatus, system, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of this disclosure.

[0080] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0081] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0082] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0083] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0084] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0085] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0086] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0087] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical applications, or technical improvements to the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for updating a dataset, comprising: Obtain a reference dataset comprising multiple reference images, each reference image having a soft label, and the soft label representing the probability that each reference image belongs to each of multiple target categories; Based on the similarity between the received input image and each reference image, an image soft label for the input image is determined, the number of original images corresponding to each probability value is determined, and the probability of each distillation reference image under the corresponding target category is determined based on each probability value and the number of original images corresponding to each probability value. as well as The input image and its soft label are added to the reference dataset to update the reference dataset.

2. The method according to claim 1, wherein, Based on the similarity between the received input image and each reference image, determine the image soft label of the input image, including: Based on the similarity between the input image and each reference image, a weight is determined for each reference image; and The soft label of the input image is determined based on the soft label of each reference image and the weight.

3. The method of claim 2, wherein determining the image soft label of the input image based on the soft label of each reference image and the weight comprises: The soft labels of each reference image are weighted using the weights to obtain weighted soft labels for each reference image. as well as The probabilities of the corresponding target categories among the multiple weighted soft labels are summed to determine the image soft label of the input image.

4. The method according to claim 1, wherein, The reference dataset includes a distillation dataset, which comprises multiple distillation reference images obtained by distilling the original dataset. The soft label of each distillation reference image is obtained based on the probability that each original image in the original dataset belongs to each of the multiple target categories.

5. The method according to claim 4, further comprising: Obtain the original dataset, wherein the original dataset includes multiple original images; The distillation operation is performed on the original dataset using a capsule neural network model to obtain the distilled dataset. The soft label represents the probability that the corresponding distillation reference image belongs to each of the plurality of target categories.

6. The method according to claim 5, wherein, The input image also includes a corresponding hard image label, and the method further includes: The capsule neural network model is fine-tuned based on the soft and / or hard labels of the input image.

7. The method according to claim 5, wherein, Distilling the original dataset to obtain the distilled dataset includes: Each original image in the original dataset is input into the capsule neural network model, and the output feature set is obtained; Obtain the capsule feature set of the capsule neural network model; and The distillation dataset is obtained based on the output feature set and the capsule feature set.

8. The method according to claim 7, wherein, The output feature set includes an output feature matrix, wherein the elements in the column vectors of the output feature matrix represent the probability that each original image in the original dataset belongs to a corresponding target category among the plurality of target categories.

9. The method according to claim 8, wherein the original dataset is distilled to obtain the distilled dataset, comprising: Obtain the column vector corresponding to the target category in the output feature matrix; Based on the elements in the column vector, determine the probability value corresponding to each distillation reference image; Based on the probability value corresponding to each distillation reference image, the probability of each distillation reference image under the corresponding target category is determined, and used as a soft label for the distillation reference image.

10. The method of claim 5, further comprising: The capsule neural network model is trained based on the original dataset. The loss function used to train the capsule neural network model is correlated with the difference of the following parameters: The mutual information between the original dataset and the distilled dataset; and The similarity between the original dataset and the distillation dataset.

11. An electronic device for updating a dataset, the electronic device comprising: At least one processor; as well as A memory, coupled to the at least one processor and having instructions stored thereon, which, when executed by the at least one processor, cause the electronic device to perform actions, including: Obtain a reference dataset comprising multiple reference images, each reference image having a soft label, and the soft label representing the probability that each reference image belongs to each of multiple target categories; Based on the similarity between the received input image and each reference image, an image soft label for the input image is determined; the number of original images corresponding to each distillation reference image for each probability value is determined; and the probability of each distillation reference image in the corresponding target category is determined based on each probability value and the number of original images corresponding to each probability value. The input image and its soft label are added to the reference dataset to update the reference dataset.

12. The electronic device according to claim 11, wherein, Based on the similarity between the received input image and each reference image, determine the image soft label of the input image, including: Based on the similarity between the input image and each reference image, a weight is determined for each reference image; and The soft label of the input image is determined based on the soft label of each reference image and the weight.

13. The electronic device of claim 12, wherein determining the image soft label of the input image based on the soft label of each reference image and the weight comprises: The soft labels of each reference image are weighted using the weights to obtain weighted soft labels for each reference image. as well as The probabilities of the corresponding target categories among the multiple weighted soft labels are summed to determine the image soft label of the input image.

14. The electronic device according to claim 11, wherein, The reference dataset includes a distillation dataset, which comprises multiple distillation reference images obtained by distilling the original dataset. The soft label of each distillation reference image is obtained based on the probability that each original image in the original dataset belongs to each of the multiple target categories.

15. The electronic device according to claim 14, wherein, The instructions, when executed by the at least one processor, cause the electronic device to perform: Obtain the original dataset, wherein the original dataset includes multiple original images; The distillation operation is performed on the original dataset using a capsule neural network model to obtain the distilled dataset. The soft label represents the probability that the corresponding distillation reference image belongs to each of the plurality of target categories.

16. The electronic device according to claim 15, wherein, The input image also includes a corresponding image hard label, and the instructions, when executed by the at least one processor, cause the electronic device to perform: The capsule neural network model is fine-tuned based on the soft and / or hard labels of the input image.

17. The electronic device according to claim 15, wherein, Distilling the original dataset to obtain the distilled dataset includes: Each original image in the original dataset is input into the capsule neural network model, and the output feature set is obtained; Obtain the capsule feature set of the capsule neural network model; and The distillation dataset is obtained based on the output feature set and the capsule feature set.

18. The electronic device according to claim 17, wherein, The output feature set includes an output feature matrix, wherein the elements in the column vectors of the output feature matrix represent the probability that each original image in the original dataset belongs to a corresponding target category among the plurality of target categories.

19. The electronic device of claim 18, wherein the original dataset is distilled to obtain the distilled dataset, comprising: Obtain the column vector corresponding to the target category in the output feature matrix; Based on the elements in the column vector, determine the probability value corresponding to each distillation reference image; Based on the probability value corresponding to each distillation reference image, the probability of each distillation reference image under the corresponding target category is determined, and used as a soft label for the distillation reference image.

20. The electronic device according to claim 15, wherein, The instructions, when executed by the at least one processor, cause the electronic device to perform: The capsule neural network model is trained based on the original dataset. The loss function used to train the capsule neural network model is correlated with the difference of the following parameters: The mutual information between the original dataset and the distilled dataset; and The similarity between the original dataset and the distillation dataset.

21. A computer program product tangibly stored on a non-volatile computer-readable medium and comprising machine-executable instructions that, when executed, cause a machine to perform the steps of the method according to any one of claims 1 to 10.

Images