Data-free noisy integrated knowledge distillation method

Abstract

The invention discloses a data-free noisy integrated knowledge distillation method. The method comprises the following steps: pre-training a teacher model by using real data; in the data synthesis stage, a generator is selected according to the type of sample data; the first generator takes the noise as an internal parameter and is used for constructing low-dimension structured sample data; the second generator takes the noise as external input and is used for constructing high-dimension sample data with nonlinear distribution; in the knowledge distillation stage, sample data are respectively predicted through the student model and the teacher model; the data synthesis stage and the knowledge distillation stage are alternately performed; distributing weights for the teacher model by using a gating network with a random shielding layer; the random shielding layer randomly shields prediction results of part of the teacher models so as to avoid adverse effects caused by noise teachers; in a data synthesis stage, freezing network layer parameters of the student model, network layer parameters of the teacher model and a gating network; in the knowledge distillation stage, network layer parameters of the teacher model and the generator are frozen.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence and deep learning, and in particular to a data-free integrated knowledge distillation method for achieving model compression and migration without real data. Background Art

[0002] Currently, large-scale deep neural networks with massive parameters have achieved success in a variety of tasks, but their high complexity and high resource consumption limit their deployment on edge devices or in resource-constrained scenarios. Knowledge distillation, as an effective model compression technology, can reduce the computational and storage overhead of the model while ensuring performance by transferring the knowledge from the complex teacher model to the lightweight student model. Traditional knowledge distillation methods still rely on a large amount of real data (such as corpora, image and audio resources, user preference information, etc.). These data serve as the basis for knowledge transfer between the teacher model and the student model, and are often unavailable in distillation tasks involving data privacy protection or data scarcity. This forces us to focus the knowledge distillation operation on zero-sample scenarios, that is, to perform data-free knowledge distillation.

[0003] Data-free knowledge distillation relies primarily on two key steps: generating synthetic input samples that can be used for training, and extracting prior knowledge from a pre-trained teacher model to guide the training of the student model. In recent years, some studies have used model inverse optimization methods to attempt to reconstruct input data that can activate specific behavioral patterns of the teacher model; others have focused on optimizing the loss function design during the distillation process to more effectively utilize the soft labels or intermediate features output by the teacher model, thereby enhancing the distillation effect in the absence of real data. Although these methods have made some progress, there are still many challenges in the two core links of data generation and knowledge transfer:

[0004] A drawback of existing technologies is that during the data generation phase, the generated user behavior data is often overly simple, which can lead to overfitting of the student model after distillation. For example, in the case of class prior strategies for multi-classification tasks and binary sample synthesis strategies for click-through rate prediction tasks, the data generation phase typically constructs high-confidence samples—samples that are close to a specific category in the teacher model's prediction results. This ensures that the training data provided to the student model has strong consistency and clear label orientation. In contrast, difficult samples near category boundaries with low confidence are often treated as noise and eliminated. However, in reality, in most tasks, the difficulty and diversity of generated data are strongly or weakly coupled. Samples near the decision boundary represent ambiguous areas where the model has difficulty making judgments. This data contains prior information that is particularly critical for task discrimination. Ignoring these "high-difficulty samples" can weaken data diversity, thereby affecting the overall performance and robustness of the knowledge distillation process.

[0005] In addition, in practical applications, a single teacher model often finds it difficult to fully cover all discriminative modes in the task space, nor to provide sufficient stability and generalization capabilities. Introducing multiple teacher models to alleviate the bias of a single model is a common and effective strategy. However, most multi-teacher distillation research focuses only on fusion methods, while ignoring the interference of unreliable predictions of some teacher models on specific samples (i.e., "noisy teachers"). Failure to distinguish the discriminative capabilities of different teacher models and directly averaging and aggregating them may introduce knowledge bias or drift, which in turn has a chain effect on distillation performance and the quality of data generation. Summary of the Invention

[0006] In view of the above-mentioned defects of the prior art, the present invention provides a data-noise-free integrated knowledge distillation method, which aims to extract comprehensive prior knowledge from a mixed cluster of good and bad teacher models under zero-sample conditions to ensure robust data synthesis and knowledge distillation.

[0007] In order to achieve the above object, the technical solution adopted by the present invention is:

[0008] A data-free noise-integrated knowledge distillation method includes the following steps:

[0009] S1. Pre-train the teacher model using real data;

[0010] S2. Generate sample data:

[0011] In the data synthesis stage, a generator is selected according to the type of the sample data; the generator includes a first generator and a second generator;

[0012] The first generator uses noise as an internal parameter to construct low-dimensional, structured sample data;

[0013] The second generator uses noise as external input to construct the sample data with high dimension and nonlinear distribution;

[0014] S3. Predict the sample data:

[0015] In the knowledge distillation stage, the sample data is predicted by the student model and the teacher model respectively;

[0016] The data synthesis stage and the knowledge distillation stage are performed alternately;

[0017] S4. Aggregate the prediction results of the teacher model:

[0018] A gating network with a random shielding layer is used to assign weights to the teacher model; the random shielding layer randomly shields some of the prediction results of the teacher model to avoid the adverse effects of the noisy teacher model.

[0019] As an advantage, it also includes step S5, optimizing the student model and the generator: taking the absolute value of the difference between the prediction result of the student model and the prediction result after aggregation of the teacher model as the prediction similarity; in the knowledge distillation stage, taking the prediction similarity as the final distillation loss L dfkd , train the student model and the gating network together; in the data synthesis stage, the opposite number of the predicted similarity is used as the main loss L adv , train the generator.

[0020] Preferably, in the data synthesis stage, when the second generator is selected to generate the sample data, diversity loss and l2-norm loss are added for supervision; the total loss of selecting the second generator to generate the sample data is the linear sum of the three losses:

[0021] L NasI =αL adv +βL diversity +γL l2-norm

[0022] Among them L diversity It is a diversity loss, which is used to ensure the multi-level and multi-dimensionality of the sample data when it is generated.

[0023] Preferably, in step S2, if the first generator is selected to generate the sample data, random noise z is sampled for each feature category of the sample data, and the dimension of each random noise z matches the shape of the feature value; the sample data generated by the first generator is characterized as follows:

[0024]

[0025] Where φ represents the model parameters of the generator, Softmax(·) is a function used to normalize the probability, and λ represents the difficulty control coefficient, which is used to reduce the learning rate when synthesizing ultra-high-dimensional features.

[0026] Preferably, in step S2, if the second generator is selected to generate the sample data, the low-dimensional noise z' of the input sample is converted into high-dimensional data after initializing the network layer parameters of the second generator; the sample data generated by the second generator is characterized as:

[0027]

[0028] Compared with the prior art, the beneficial effects of the present invention are embodied in:

[0029] 1. Different from the generators of traditional technologies, the present invention introduces two types of generators. One type directly synthesizes data of the same size as the expected data (i.e., the generator uses noise as the internal parameter), and the other type expands the data into samples of the expected size layer by layer through the network (i.e., the generator uses noise as the external input). By selecting the generator according to the type of expected data, the generated data is made more realistic and diversified, better simulating the distribution of real data, thereby improving the effect of knowledge distillation.

[0030] 2. Unlike traditional techniques that are affected by noisy teacher models, this invention uses a gated network with a random shielding layer to randomly shield the teacher's output. This assigns differentiated "expert groups" to the student model at each teaching stage, thereby reducing the impact of the noisy teacher and ensuring that the student model is not overly dependent on a single teacher model. It also forces the student model to synchronize with a more superior and comprehensive judgment perspective, ensuring that the student model can learn accurate and comprehensive knowledge. By reducing the impact of the noisy teacher and avoiding the student model's over-reliance on a single teacher model, the robustness of knowledge distillation can be improved.

[0031] 3. The present invention enables effective model training and knowledge transfer even when there is no real sample data for reference. Through this method, efficient machine learning models can be deployed in environments where data is scarce or data privacy requirements are strict. At the same time, the teacher model cluster may contain models with varying performance. The present invention aims to extract comprehensive and useful knowledge from these models to ensure that the student model can learn extensive knowledge from multiple teacher models, thereby improving its performance and generalization ability. Secondly, the synthetic data generated by the present invention is sufficiently real and diverse so that the student model can learn useful features and patterns from it. Through a robust data synthesis process, high-quality synthetic data can be generated that can simulate the distribution of real data, thereby improving the training effect of the student model. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 Schematic diagram of the data-free integrated knowledge distillation framework of Example 1;

[0033] Figure 2 is a schematic diagram of a noise-boosted sample generation method according to Example 1;

[0034] Figure 3 Schematic diagram of the teacher model cluster knowledge aggregation method of Example 1. DETAILED DESCRIPTION

[0035] In order to make the technical means, creative features, objectives and effects of the invention easier to understand, the present invention is further described with reference to specific figures. However, the present invention is not limited to the following implementation cases.

[0036] It should be noted that the structures, proportions, sizes, etc. illustrated in the drawings in this specification are only used to match the contents disclosed in the specification so that people familiar with this technology can understand and read them. They are not used to limit the conditions under which the present invention can be implemented. Therefore, they have no substantive technical significance. Any structural modification, change in proportional relationship or adjustment of size should still fall within the scope of the technical content disclosed in the present invention without affecting the efficacy and purpose that can be achieved by the present invention.

[0037] Example 1:

[0038] like Figure 1 A data-free noisy integrated knowledge distillation method is shown. In this embodiment, the data-free noisy integrated knowledge distillation method is applied to a complex image classification task, including the following steps:

[0039] S1. Pre-train the teacher model using real data:

[0040] First, it is necessary to pre-train multiple teacher models with similar structures using real data.

[0041] S2. Generate sample data:

[0042] During the data synthesis phase, a generator is selected based on the type of sample data. The generator consists of a first generator and a second generator. The first generator uses noise as an internal parameter to construct low-dimensional, structured sample data, such as preferred features, simple graphs, and sparse data. This requires direct optimization of internal parameters for fine-grained control. The second generator uses noise as an external input. Its internal structure is composed of a stack of dimensionality-increasing network layers. It is used to construct complex, high-dimensional sample data, such as high-dimensional images, audio, and time series data. This generator strongly relies on the representational capabilities of the internal network layers.

[0043] If the first generator is selected to generate the sample data, a batch of random noise z is sampled for each feature category of the sample data, and the dimension of each random noise z matches the shape of the eigenvalue. The dimension of the noise refers to the number of elements in the noise vector. For example, if a noise vector has 10 numbers, then its dimension is 10. The shape of the eigenvalue refers to the range or distribution of values for each feature in the data. When we say that the dimension of the noise matches the shape of the eigenvalue, it means that the noise vector used by the generator should have the same structure as the target data features. For example, if image data is being generated and each pixel is represented by three color values, then the noise vector used by the generator should also have three elements, each corresponding to a color channel. These noise matrices are encapsulated and become internal parameters of the generator. By optimizing these internal parameters, the generator can be finely controlled to generate the desired data type.

[0044] Image data is a complex, high-dimensional data type, so a generator (the second generator) is chosen here that uses noise as an external input. Specifically, it stacks inverted convolutional layers one after another as network layers. Before training the second generator, a parameter initialization method is used to set initial values for the weights and biases in the network layers. Appropriate parameter initialization methods can effectively speed up training, avoid vanishing or exploding gradients, and improve overall convergence stability.

[0045] The stage of using the generator to generate the required samples is called the data synthesis stage. The stage of using the synthesized sample data as the dataset for joint prediction between teachers and students, forcing the student predictions and the teacher cluster predictions to be close to each other, is called the knowledge distillation stage.

[0046] During the data synthesis phase, the second generator is required to obtain the synthetic image samples needed for the distillation phase. The purpose of the second generator is to reconstruct image information. The second generator needs to take low-dimensional noise z' as input and expand the noise into a tensor of the same size as the target image through the network layer. The low-dimensional noise z' is sampled from a normal random distribution, and a batch of noise is sampled each time as external input information. The network layer within the second generator plays a role in flexibly shaping the sampled noise. With each layer of the network, the shape of the reconstructed data will become closer to the expected data form. This process is called mapping from low-dimensional noise to data space.

[0047] S3. Prediction sample data:

[0048] In the knowledge distillation phase, the student model and the teacher model predict sample data separately. The data synthesis phase and the knowledge distillation phase are performed alternately. In the knowledge distillation phase, the synthesized image samples are input into the student model and each teacher model (teacher cluster), allowing both to participate in the prediction.

[0049] S4. Prediction results of aggregated teacher model:

[0050] A gating network with a random masking layer is used to assign weights to the teacher model; the random masking layer randomly masks some of the teacher model's predictions. To allow the student model to obtain valid information, the predictions of all teacher models are fed into the gating network, which then aggregates the outputs of each teacher model. The gating network consists of a dimensionality-reduced multilayer perceptron network. An additional random masking layer is introduced at the end of the gating network. This random masking layer randomly masks some of the teacher model's outputs with a fixed probability, providing a differentiated teacher group for the student model and preventing the student model from relying on a single teacher model. Because the overall teacher model's judgment perspective is superior, the student model gradually synchronizes with the superior teacher's perspective, achieving even better results.

[0051] Freeze the network layer parameters of some models according to the training stage. All pre-trained teacher models freeze their network layer parameters at all stages. The data synthesis stage and knowledge distillation stage are performed alternately. If the stage is the data synthesis stage, freeze the student model and the gating network that aggregates the output knowledge of the teacher model. If the stage is the knowledge distillation stage, freeze the generator.

[0052] The entire process, from sampling noise to constructing sample data, then passing the samples to the student and teacher clusters, and having them collectively predict, is called forward propagation. Forward propagation demonstrates the process by which the teacher and student perform predictive reasoning on the same batch of data. It characterizes the generator's ability to construct samples in its current state, as well as the reasoning capabilities of the teacher and student.

[0053] The entire process of starting with the target loss function, updating the parameters of the gated network layer, then updating the parameters of the student model, and finally updating the parameters of the generator model is called backpropagation. Backpropagation illustrates the process by which the teacher and student summarize the batch of data in stages. During this stage, the gated network layer parameters are updated to reduce the influence of the noisy teacher model. The student model updates its own network layer parameters to bring its judgment closer to that of the teacher cluster. The generator updates its own network layer parameters to generate more diverse and robust data suitable for the next stage of student model learning. Backpropagation is a dynamic change made by the entire distillation framework to adapt to emerging changes.

[0054] The final results show that the student model can achieve performance similar to that of supervised training through the method of the present invention without any real image data, verifying the feasibility of the method and its transferability to the visual field.

[0055] The generator is used to obtain continuous synthetic data. The data generated by the generator with noise as internal parameters can be characterized as follows:

[0056]

[0057] Where λ represents the difficulty control coefficient, which acts to reduce the learning rate when synthesizing ultra-high-dimensional features.

[0058] The data generated by the generator that takes noise as external input can be characterized as:

[0059]

[0060] For simple linear samples like recommended data, a linear layer can be used to complete the dimensionality expansion of noise to continuous data:

[0061]

[0062] Among them, BN(·) is a technology used in deep learning to accelerate training and stabilize models. It standardizes the middle activation layers to make them more stable and easier to train. Linear(·) is a linear transformation layer used for linear expansion of tensors. The outermost Linear outter The outermost linear layer denoted by (·) is used for the final adjustment in dimensionality.

[0063] For structured feature maps similar to images, inverted convolution can be used for synthesis:

[0064]

[0065] DeConv2D(·) represents the operation used to "upsampling" the feature map, which expands the size of the feature map and achieves the reverse effect in structure. ReLU(·) represents the thread rectification function, which retains the input if it is integer, otherwise the output is 0.

[0066] For non-image or non-pure vector data, consider using a multi-layer perceptron mixture or self-attention module:

[0067]

[0068] MixerBlock(x) is a hybrid neural network module that combines the MLP-Mixer and Transformer structures. This module fuses the two paths through the weighting factor α; LayerNorm(·) normalizes each sample of input x in the feature dimension to improve training stability; MLP token (·) Allows information exchange between different locations; MLP channel (·) MLP is used in the channel dimension to improve feature representation capabilities; MultiHeadAttention(·) is a multi-head attention mechanism, the core idea of which is to enable the model to understand the relationship between input information from different attention angles; FFN(·) is a feedforward network used to improve nonlinear expression capabilities; finally, the overall aggregation is performed through the fusion parameter α.

[0069] The present invention will be further described below with reference to the accompanying drawings:

[0070] (1) The overall process of the present invention is as follows Figure 1 shown.

[0071] Data synthesis and knowledge distillation are performed alternately. The generator generates continuous data, which are then fed into the student model and the teacher cluster for prediction. The final prediction results of the teacher cluster are aggregated using the corresponding integration strategy.

[0072] In the knowledge distillation stage, the student model needs to learn from the integrated teacher information to synchronize its decision-making perspective with the teacher cluster perspective. Therefore, in this stage, the student model needs to be trained with the goal of narrowing the prediction gap between teachers and students. The knowledge distillation loss function L in this stage is dfkd It can be expressed as:

[0073]

[0074] Where f represents the sample or feature constructed by the generator, S(·) represents the predicted output of the student model, and T i (·) represents the predicted output of the teacher model, w i Indicates the weight value assigned to each teacher.

[0075] In the data synthesis stage, it is necessary to iteratively generate more diverse samples. The new synthetic samples should ensure a certain degree of challenge to provide a large information gap and encourage students to learn further. Therefore, in this stage, the generator is trained with the goal of increasing the prediction between teachers and students. Specifically, the adversarial loss can be represented as:

[0076] L adv =-L dfkd ,

[0077] Under balanced adversarial strategies, a Nash equilibrium is achieved, meaning each participant makes the optimal choice given the other participant's fixed strategy. Consequently, neither participant has the incentive to unilaterally change their strategy, and each participant in the system can be considered to have achieved a stable optimal state. This means that under the current framework, the constructed samples generated by the generator will become increasingly realistic throughout the entire process, and the overall performance of the student model will increasingly approach that of the best teachers in the teacher cluster.

[0078] (2) Noise-boosted sample generation methods, such as Figure 2 shown.

[0079] The generation methods of constructed samples are divided into the generation method using the difficulty control coefficient and taking noise as the internal parameter and the generation method using the network layer to gradually increase the dimension and taking noise as the external input, as follows:

[0080] Generators that use noise as internal parameters can meet the needs of fast distillation, such as Figure 2As shown in the left figure, the generator encapsulates multiple randomly sampled noise matrices. When the weight parameters within the generator are updated, the process is essentially to optimize and update the encapsulated noise matrix. In some tasks, certain feature dimensions are abnormally high (such as ID classes). The generator often requires an extremely high learning rate (the learning rate value may even exceed 1) to update the continuous data form of these high-dimensional features. To achieve this, the difficulty control coefficient λ is introduced. When the feature dimension is high, the difficulty control coefficient can be set to a low value close to 0, thereby reducing the generator's learning rate and achieving more stable sample updates.

[0081] The purpose of the generator that uses noise as external input is to explore the performance of continuous form data when multiple synthetic quantities are used. The generator consists of a stack of network layers for specific tasks, which maps the sampled low-dimensional noise to the data space through a linear layer. The data synthesis stage randomly samples from the entire noise space, and the mapped data space is also relatively wide, which provides a basic guarantee for diversity. In order to ensure the quality and diversity of the generated samples, additional l2-norm constraints and diversity constraints are imposed when using the generator that uses noise as external input. The l2-norm constraint ensures that the constructed sample as a whole does not diverge excessively, and its loss function is in the form of:

[0082]

[0083] When the sample generation and student training are out of balance, mode collapse is more likely to occur, that is, the generator will only construct samples within a single soft label interval. To alleviate this phenomenon, diversity constraints are introduced. Intuitively, the difference {f1, f2} between samples generated from different noise clusters {z'1, z'2} should be positively correlated with the distance between the noise clusters. Otherwise, it means that mode collapse has occurred. The constructed samples from different noise clusters are represented as follows:

[0084]

[0085] in Indicates the connection between constructed features. At this time, the diversity ratio and diversity loss can be set as:

[0086]

[0087] Where ∈ is a small value greater than 0. In summary, through the noise-promoted sample generation method, the continuous form data required in the knowledge distillation stage is obtained.

[0088] (3) Teacher cluster knowledge aggregation method, such as Figure 3 shown.

[0089] The core task of the teacher cluster knowledge aggregation method is to reduce the weight of the noisy teacher model predictions in the overall prediction. The traditional gating network is a multi-layer perception with layer-by-layer dimensionality reduction. The input is the output value of each teacher on a batch of generated data. The output weight value dimension is consistent with the number of teacher models. It can be formally expressed as:

[0090]

[0091] Where {f'} is the constructed sample synthesized by the generator, t represents the normalized temperature, Denote N as the teacher model and ω as the gating network. In the absence of true label supervision, the gating network and the student model share the same optimization objective, which is to reduce the distance between the student's predicted output and the soft labels of the teacher cluster. This means that if the student model is sufficiently similar to a certain type of teacher, the gating network, driven by the optimization objective, will increase the weight of this type of teacher model, causing the student model to strongly rely on the representation output of this type of teacher for learning. In the early stages of training, the student model does not have sufficient discriminative power. At this time, the student model's feedback on constructed samples is more likely to be similar to the noisy teacher model whose output is ambiguous. If the gated output is not controlled, the student model may be harmed by the noisy teacher.

[0092] Therefore, we randomly mask the gated output to ensure that a different combination of teachers is provided to the student model each time. Since the noise teachers are in the minority and the masking mechanism prevents the student from learning only from the teachers with the highest gated output, over time the student model's judgment perspective will synchronize with the excellent teachers, and the generated samples will gradually contain correct view information. The masking mechanism is implemented by introducing a random dropout layer. The final output weight value after normalization can be expressed as:

[0093]

[0094] The numerator and denominator are the same Dropout operation, that is, they represent the same masking operation. ∈ is a small value greater than 0 to prevent division by 0. In addition, to further ensure multi-view supervision, we set an additional penalty term to limit the maximum value of the model output to avoid polarization. The penalty term setting can be expressed as:

[0095] L penalty =max(0,max(W)-upper limmit )stmin(W)≠0.

[0096] Where W represents the weight value assigned to each teacher, upper limit represents the minimum weight value assigned by the gate to the teacher model. The overall loss function of the gated network can be expressed as:

[0097] L gating =L dfkd +μ Lpenalty .

[0098] Through a gating network with random dropout layers, we achieve aggregation of teacher models and ensure that the student model is steadily aligned towards excellent multi-view information.

Claims

1. A data-free, noisy integrated knowledge distillation method, characterized in that: The following steps are involved: S1. Pre-train the teacher model using real data; S2. Generate sample data: In the data synthesis stage, a generator is selected according to the type of the sample data; the generator includes a first generator and a second generator; The first generator uses noise as an internal parameter to construct low-dimensional, structured sample data; The second generator uses noise as external input to construct the sample data with high dimension and nonlinear distribution; S3. Predict the sample data: In the knowledge distillation stage, the sample data is predicted by the student model and the teacher model respectively; The data synthesis stage and the knowledge distillation stage are performed alternately; S4. Aggregate the prediction results of the teacher model: A gating network with a random masking layer is used to assign weights to the teacher model; the random masking layer randomly masks some of the prediction results of the teacher model.

2. The data-free noisy integrated knowledge distillation method according to claim 1, characterized in that: The method further comprises step S5, optimizing the student model and the generator: The absolute value of the difference between the prediction result of the student model and the prediction result after aggregation of the teacher model is used as the prediction similarity; in the knowledge distillation stage, the prediction similarity is used as the final distillation loss L dfkd , train the student model and the gating network together; in the data synthesis stage, the opposite number of the predicted similarity is used as the main loss L adv , train the generator.

3. The data-free noisy integrated knowledge distillation method according to claim 2, characterized in that: In the data synthesis stage, when the second generator is selected to generate the sample data, diversity loss and l2-norm loss are added for supervision; the total loss of selecting the second generator to generate the sample data is the linear sum of the three losses: L NasI =αL adv +βL diversity +γL l2-norm Among them, L diversity It is a diversity loss, which is used to ensure the multi-level and multi-dimensionality of the sample data when it is generated.

4. The data-free noisy integrated knowledge distillation method according to claim 1, characterized in that: In step S2, if the first generator is selected to generate the sample data, random noise z is sampled for each feature category of the sample data, and the dimension of each random noise z matches the shape of the feature value; the sample data generated by the first generator is characterized as follows: Where φ represents the model parameters of the generator, Softmax(·) is a function used to normalize the probability, and λ represents the difficulty control coefficient, which is used to reduce the learning rate when synthesizing ultra-high-dimensional features.

5. The data-free noisy integrated knowledge distillation method according to claim 1, characterized in that: In step S2, if the second generator is selected to generate the sample data, the network layer parameters of the second generator are initialized and the low-dimensional noise z' of the input sample is converted into high-dimensional data; the sample data generated by the second generator is characterized as:

Images