Data processing method and device

Abstract

The embodiment of the present application relates to a data processing method, which includes: obtaining an original data set; using the average port Jason Shannon divergence to obtain a first indicator; inputting the data in the original data set into a generative adversarial network and generating first expanded data; combining the first expanded data and the data in the original data set, and using the average port Jason Shannon divergence to obtain a second indicator; when the second indicator is greater than the first indicator, adding the first expanded data to the original data set to obtain a first data set; replacing the original data set with the first data set. Through the above method, diverse data can be continuously obtained, and a complete and diverse data set can be obtained. At the same time, the complete training data set obtained can effectively improve the generalization ability of the network model.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a data processing method and device for continuously acquiring diverse data based on a generative adversarial network. Background Art

[0002] Currently, machine learning-based image recognition technology trains network models on a large number of sample images to achieve broad recognition capabilities. However, during actual training, the number of images collected for certain categories may be relatively small, failing to fully capture the information from these categories. Furthermore, using these relatively sparse sample images to train network models results in low model accuracy and weak transferability.

[0003] The accuracy and robustness of a model are closely related to the training data, but continuously acquiring data through manual annotation is costly and inefficient. Traditional data augmentation methods include simple image scaling, cropping to a specified size centered on the image, and random horizontal or vertical flipping of the image. While traditional data augmentation methods can increase the number of samples, the augmented data is likely to contain a high level of redundancy. Current image augmentation methods are primarily used in specific task scenarios. Image augmentation enhances useful information in an image. Its purpose is to improve the visual quality of a given image, enhance image interpretation and recognition, and meet the needs of specific analyses. However, current image augmentation methods still have drawbacks in terms of data redundancy for continuously generated data. Summary of the Invention

[0004] The present invention provides a method and apparatus for obtaining diverse data. It can continuously obtain diverse data and continuously expand the original data set, thereby obtaining a complete and diverse data set.

[0005] In a first aspect, a data processing method is provided, comprising: obtaining an original dataset; evaluating the original dataset using the mean inception Jensen-Shannon divergence to obtain a first metric, the first metric being used to represent the diversity of the original dataset; inputting data from the original dataset into a generative adversarial network (GAN) to generate first augmented data; combining the first augmented data with data from the original dataset and evaluating the data using the mean inception Jensen-Shannon divergence to obtain a second metric, the second metric being used to represent the diversity of the original dataset after the first augmented data is combined; when the second metric is greater than the first metric, adding the first augmented data to the original dataset to obtain a first dataset; and replacing the original dataset with the first dataset. The mean inception Jensen-Shannon divergence (JS) represents the average inception Jensen-Shannon divergence of each data point in the dataset, i.e., the average distance between each data point in the dataset, to reflect whether the dataset is diverse. JS divergence, also known as JS distance, is a symmetry measure used to measure the difference between two probability distributions. GAN is a deep learning model that is usually used to generate data that is very similar to real data and is used for data enhancement.

[0006] In one possible embodiment, after obtaining the first indicator, the method further includes: performing feature extraction on the data in the original dataset; detecting the data in the feature-extracted original dataset using a local outlier factor (LOF) algorithm to obtain an outlier set; and inputting the data in the outlier set into a GAN to generate first augmented data. In the feature space, the proximity between an outlier and its nearest neighbor significantly deviates from the proximity between other data in the dataset and their own nearest neighbors; the outlier set is the set of all outliers.

[0007] In one possible embodiment, the loss function used by the generative adversarial network GAN has multiple parameters, wherein the parameters include port Jensen-Shannon divergence and kernel maximum mean difference. Among them, port Jensen-Shannon divergence (Inceptionjensen-Shannondivergence, Inception_JS) is to use the port (Inception) network to map the data to the category space, and use the Jensen-Shannon divergence (divergence) to measure the distance between the data in the category space. Kernel maximum mean discrepancy (kernel_MMD) is to use the kernel function to calculate the maximum mean difference, which is used to judge the similarity between two distributions; and the dimension of the data set in the feature space is high, and its solution only uses the inner product. In the low-dimensional space, there is a function that is equal to the inner product in the high-dimensional space. In order to simplify the calculation, this function can be used directly for calculation, and this function is the kernel function.

[0008] In one possible embodiment, the method further includes: for the labels in the original dataset, introducing data with the same labels from the public dataset; performing feature extraction on the data in the public dataset with the same labels; combining the data in the public dataset with the same labels and the data in the original dataset, and evaluating them using the average port Jason Shannon divergence to obtain a third indicator, which is used to represent the diversity of the original dataset after combining the data in the public dataset with the same labels; when the third indicator is greater than the first indicator, using the data in the public dataset with the same labels as the second expanded data.

[0009] In one possible implementation, having the same label includes: the label in the public dataset is the same as the label in the original dataset; or the label in the public dataset is a subset of the label in the original dataset.

[0010] In one possible implementation, the method further includes: adding the second expanded data to the original data set to obtain a second data set; and replacing the original data set with the second data set.

[0011] In one possible embodiment, the method further includes: using GAN to generate third augmented data for the second augmented data; combining the third augmented data and data in the original dataset, and evaluating using the average port Jason Shannon divergence to obtain a fourth indicator, where the fourth indicator is used to represent the diversity of the original dataset after combining the third augmented data; when the fourth indicator is greater than the first indicator, adding the third augmented data to the original dataset to obtain a third dataset; and replacing the original dataset with the third dataset.

[0012] In a possible implementation, the data type of the data in the original data set, the first data set, the second data set, and / or the third data set is a picture type.

[0013] In a second aspect, a data processing device is provided, which includes: an acquisition module for acquiring an original data set; an evaluation module for evaluating the original data set using the average port Jason Shannon divergence to obtain a first indicator, which is used to represent the diversity of the original data set; a first generation module for inputting data in the original data set into a generative adversarial network (GAN) and generating first expanded data; the evaluation module is also used to combine the first expanded data and the data in the original data set, and use the average port Jason Shannon divergence for evaluation to obtain a second indicator, which is used to represent the diversity of the original data set after combining the first expanded data; an adding module for adding the first expanded data to the original data set to obtain the first data set when the second indicator is greater than the first indicator; and a replacement module for replacing the original data set with the first data set.

[0014] In one possible embodiment, the apparatus further includes: a feature extraction module for performing feature extraction on data in an original data set; a detection module for detecting the data in the original data set after feature extraction using a local outlier factor (LOF) algorithm to obtain an outlier set; and the first generation module is further configured to input the data in the outlier set into a GAN and generate first augmented data.

[0015] In one possible implementation, the first generation module includes: a generative adversarial network (GAN); a loss function used by the GAN has multiple parameters, wherein the parameters include port Jason Shannon divergence and kernel maximum mean difference.

[0016] In one possible embodiment, the device also includes: an introduction module, which is used to introduce data with the same label as the label in the original data set from the public data set; the feature extraction module is also used to extract features from the data in the public data set with the same label; the evaluation module is also used to combine the data in the public data set with the same label and the data in the original data set, and use the average port Jason Shannon divergence to evaluate to obtain a third indicator, which is used to represent the diversity of the original data set after combining the data in the public data set with the same label; a second generation module is used to use the data in the public data set with the same label as the second expanded data when the third indicator is greater than the first indicator.

[0017] In one possible implementation, having the same label includes: the label in the public dataset is the same as the label in the original dataset; or the label in the public dataset is a subset of the label in the original dataset.

[0018] In one possible implementation, the adding module is further configured to: add the second expanded data to the original data set to obtain a second data set; and the replacing module is further configured to replace the original data set with the second data set.

[0019] In one possible embodiment, the first generation module is further used to: use GAN to generate third augmented data for the second augmented data; the evaluation module is further used to combine the third augmented data and the data in the original data set, and use the average port Jason Shannon divergence to evaluate to obtain a fourth indicator, which is used to represent the diversity of the original data set after combining the third augmented data; the addition module is further used to, when the fourth indicator is greater than the first indicator, add the third augmented data to the original data set to obtain a third data set; the replacement module is further used to replace the original data set with the third data set.

[0020] In a possible implementation, the data type of the data in the original data set, the first data set, the second data set, and / or the third data set is a picture type.

[0021] According to a third aspect, a computer-readable storage medium is provided, in which instructions are stored. The computer-readable storage medium is characterized in that when the instructions are executed on a terminal, the terminal executes the method according to the first aspect.

[0022] In a fourth aspect, a computer system is provided. The computer system includes a processor, the processor is coupled to a memory, and the processor reads and executes instructions in the memory, so that the computer system implements the method of the first aspect.

[0023] The present application discloses a method and device for obtaining diversity data, which extracts the original data set through the local outlier factor LOF algorithm to obtain an outlier set. And uses generative adversarial networks (GAN) to generate expanded new data. By introducing two regularization terms, port Jason Shannon divergence and kernel maximum average difference distance, into the loss function of GAN, the difference between the generated new data and the real data is controlled. And data from a public data set with the same label is obtained, and then the diversity of the new data is increased by evaluating the average distance divergence. Finally, the average distance divergence is used to evaluate whether the newly added data has increased the diversity of the original data set. This prevents the addition of redundant data and avoids negative effects on subsequent training. Through the method provided by the present application, diverse data can be continuously obtained, and a complete and diverse data set can be obtained. At the same time, the complete training data set obtained can effectively improve the generalization ability of the network model. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A schematic diagram of an application scenario provided in an embodiment of the present application;

[0025] Figure 2 A schematic diagram of a data processing system architecture provided in an embodiment of the present application;

[0026] Figure 3 A schematic diagram of a data processing system interaction provided in an embodiment of the present application;

[0027] Figure 4 A flow chart of a data processing method provided in an embodiment of the present application;

[0028] Figure 5 A schematic diagram of diversity evaluation using the average port Jason Shannon divergence provided in an embodiment of the present application;

[0029] Figure 6a A schematic diagram of a two-dimensional distribution of an original data set provided in an embodiment of the present application;

[0030] Figure 6b A schematic diagram of a two-dimensional point set distribution provided in an embodiment of the present application;

[0031] Figure 6c A schematic diagram of a two-dimensional distribution of an outlier set provided in an embodiment of the present application;

[0032] Figure 7 A diagram of a generative adversarial network structure provided in an embodiment of the present application;

[0033] Figure 8 A schematic diagram of data generation by a generative adversarial network according to an embodiment of the present application;

[0034] Figure 9 A schematic diagram of a comparison of classification accuracy of a data set provided in an embodiment of the present application;

[0035] Figure 10 A flow chart of another data processing method provided in an embodiment of the present application;

[0036] Figure 11 A flow chart of another data processing method provided in an embodiment of the present application;

[0037] Figure 12 A schematic diagram of a data processing device provided in an embodiment of the present application;

[0038] Figure 13 A schematic diagram of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0040] This application is primarily used in data processing platforms, which are located within artificial intelligence systems. The following describes these artificial intelligence systems from two perspectives: the "intelligent information chain" and the "IT value chain."

[0041] The "intelligent information chain" reflects the entire process from data acquisition to processing. For example, it can be a general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, and intelligent execution and output. In this process, data undergoes a condensed process of "data-information-knowledge-wisdom."

[0042] The "IT value chain" reflects the value that artificial intelligence brings to the information technology industry, from the underlying infrastructure of human intelligence, information (providing and processing technology implementation) to the system's industrial ecological process.

[0043] (1) Infrastructure:

[0044] Infrastructure provides computing power for AI systems, enabling communication with the outside world and supporting this through a foundational platform. External communication occurs through sensors; computing power is provided by intelligent chips (CPUs, NPUs, GPUs, ASICs, FPGAs, and other hardware accelerators). The foundational platform includes a distributed computing framework and network-related platform guarantees and support, including cloud storage and computing, and interconnected networks. For example, sensors communicate with the outside world to acquire data, which is then fed into the intelligent chips within the distributed computing system provided by the foundational platform for computation.

[0045] (2) Data

[0046] Data above the infrastructure layer represents data sources for AI. This data includes graphics, images, voice, and text, as well as IoT data from traditional devices. This includes business data from existing systems and sensor data such as force, displacement, liquid level, temperature, and humidity.

[0047] (3) Data processing

[0048] Data processing generally includes data training, machine learning, deep learning, search, reasoning, decision-making, etc.

[0049] Among them, machine learning and deep learning can symbolize and formalize data for intelligent information modeling, extraction, preprocessing, and training.

[0050] Reasoning refers to the process of simulating human intelligent reasoning in computers or intelligent systems, using formalized information to perform machine thinking and solve problems based on reasoning control strategies. Typical functions are search and matching.

[0051] Decision-making refers to the process of making decisions after intelligent information is reasoned, and usually provides functions such as classification, sorting, and prediction.

[0052] (4) General ability

[0053] After the data has undergone the data processing mentioned above, some general capabilities can be further formed based on the results of the data processing, such as algorithms or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0054] (5) Smart products and industry applications

[0055] Smart products and industry applications refer to the products and applications of artificial intelligence systems in various fields. They are the encapsulation of the overall artificial intelligence solution, which productizes intelligent information decision-making and realizes practical application. Its application areas mainly include: smart manufacturing, smart transportation, smart home, smart medical care, smart security, autonomous driving, safe cities, smart terminals, etc.

[0056] The data processing platform of this application belongs to the data processing part and is in the early stage of model training. Its purpose is to provide original training data for machine learning, deep learning and other steps in order to train a data model that meets the expectations. Figure 1 As shown, Figure 1 A schematic diagram of an application scenario provided for an embodiment of this application. The data processing platform is an intelligent data platform for human-machine collaboration, with artificial intelligence capabilities that are more efficient, faster to train, and have stronger models. The data processing platform can include data preprocessing, automatic data labeling, automatic data enhancement, or a plug-in data processing framework. Among them, this application is mainly used in the automatic data enhancement section outlined by the dotted line.

[0057] In some approaches, generative adversarial networks (GANs) are primarily used to expand the small category sample set in the original dataset, thereby improving image classification accuracy when the amount of data across different categories in the dataset is imbalanced. The sample data in the original dataset includes one or more categories. For some categories, samples from only a very small portion of the original dataset are referred to as small categories, and the collection of samples from these categories is referred to as the small category sample set. First, sample images from the small category in the dataset are selected to obtain an original small category sample image training set. The original small category sample image training set is then input into a GAN to generate generated image samples from the small category, where these samples are generated by the GAN. The generated generated image samples are then added to the original small category sample image training set to obtain a generated small category sample image training set. Finally, an image classification network is trained using the original small category sample image training set, and the trained image classification network is then trained using the generated small category sample image training set. Ultimately, an optimal image classification network is obtained, which can be used to classify input images. However, this approach also requires a large amount of data to generate high-quality image sets when using GANs to generate images. Directly using a smaller set of images from a smaller category will result in poor quality images generated by the GAN, which will have limited impact on improving the network's classification accuracy.

[0058] To address the above issues, this application has designed a solution that can automatically and continuously acquire diverse data, greatly improving the data enhancement function in the data framework and solving the problem of data redundancy generated in other solutions.

[0059] like Figure 2 As shown, Figure 2 A schematic diagram of a data processing system architecture provided in an embodiment of the present application.

[0060] Figure 2 A schematic diagram of a data processing system architecture is shown in FIG. Figure 2As shown, the automatic data enhancement portion used in this application can be located on a server. The software layer located on the server can include the dataset diversity assessment device 201, the GAN data generation device 202, and the public dataset introduction device 203 involved in this application. The hardware layer located on the server can include a processor 204 and a memory 205. The memory 205 can store the original dataset and the new dataset after continuously updating the diversity data, and can also store computer program code for executing the method of this application. The processor 204 is used to read the corresponding program code from the memory to execute the method of this application. At the same time, the processor 204 reads the original dataset from the memory 205 so that the dataset diversity assessment device 201 of the software layer can perform an assessment based on the read original dataset. Those skilled in the art should note that the processor 204 located in the hardware layer can be a central processing unit (CPU), a graphics processing unit (GPU), a field programmable gate array (FPGA), or any other dedicated chip. And the memory can include internal memory and external memory.

[0061] Previous data processing platforms may have processed data by obtaining it from other data sources via the file transfer protocol (FTP), or by reading data from a file, database, or memory. With the solution provided by this application, it is only necessary to update the data source of the data processing platform to the interface of the automatic data enhancement module of this application. This allows for the continuous acquisition of diverse data and the continuous expansion of the data in the dataset.

[0062] like Figure 3 As shown, Figure 3 A schematic diagram of a data processing system interaction provided in an embodiment of the present application.

[0063] Figure 3 Shown Figure 2 Schematic diagram of the system interaction between the Chinese dataset diversity assessment device 201, the public dataset introduction device 203 and the GAN data generation device 202.

[0064] The data set diversity evaluation device 201 includes a first storage module 2011 and a first average port Jason Shannon divergence module 2012, the GAN data generation device 202 includes a local outliers factor (LOF) module 2021, a GAN module 2022, and a second average port Jason Shannon divergence module 2023, and the public data set introduction device 203 includes a second storage module 2031 and a third average port Jason Shannon divergence module 2032. In one example, the GAN module 2022 may further include a port Jason Shannon divergence submodule 20221 and a kernel maximum average difference submodule 20222. In another example, the system further includes a third storage module 2041. The first storage module 2011 is used to obtain the data from Figure 2 The memory 205 shown in the figure obtains and stores the data of the original data set; the second storage module 2023 is used to obtain the data of the original data set from the memory 205; Figure 2 The memory 205 shown acquires and stores data of the public data set; the third storage module 2032 is used to store data of the new data set.

[0065] The dataset diversity assessment device 201 first determines the diversity of the original dataset using the first average port Jason-Shannon divergence module 2012, and then performs feature extraction on the data in the original dataset. The feature-extracted data is then input into the LOF module 2021 in the GAN data generation device 202. The LOF module 2021 performs outlier detection on the input data based on density and determines a set of outliers. The LOF module then inputs the set of outliers into the GAN module 2022, allowing the GAN module to generate the first augmented data X'. To ensure that the generated data closely resembles the real data, the GAN module 2022 is continuously corrected during training using the port Jason-Shannon divergence submodule and the kernel maximum mean difference submodule to improve the GAN module 2022. Furthermore, for data with labels in the original dataset, the dataset diversity assessment device 201 determines the labels in the original dataset and transmits the label information to the public dataset introduction device 203. Based on the labels in the original dataset, the public dataset introduction device 203 extracts data with the same labels from the public dataset. The same labels can mean that the labels are exactly the same, or the labels in the public dataset are a subset of the labels in the original dataset. The extracted data is used as the second expanded data X". Then X', X", or X'+X" is added to the original dataset to form a new dataset. Finally, the dataset diversity assessment device 201 determines whether the data diversity of the new dataset is greater than the diversity of the original dataset. If the diversity of the new dataset is higher, the new dataset replaces the original dataset. In other words, X', X", or X'+X" is added to the original dataset. In this way, the data in the original dataset is continuously expanded, while ensuring that the diversity of the expanded dataset becomes higher.

[0066] The technical solutions in the embodiments of the present application will be described in detail below with reference to the accompanying drawings in the embodiments of the present application.

[0067] like Figure 4 As shown, Figure 4 A flow chart of a data processing method provided in an embodiment of the present application.

[0068] like Figure 4 As shown, the present application discloses a method for data processing, which describes Figure 3 The process shown is to generate diversity data from the dataset diversity evaluation device 201 through the GAN data generation device 202 and then expand it to the original dataset. The method may include the following steps:

[0069] S401, obtaining data of the original data set.

[0070] In one embodiment, the data diversity assessment device first obtains data from the original dataset. In one example, the dataset diversity assessment device 201 reads the data from the original dataset from the memory 204 and stores it in the first storage module 2011. In another example, the data type of the data in the original dataset can be an image.

[0071] S402: Evaluate the first data set using average distance divergence to obtain a first index.

[0072] In one embodiment, data diversity assessment device 201 uses the mean port Jason-Shannon divergence to assess the original dataset to obtain a first index. The first index is used to indicate the diversity of the original dataset; a higher index value indicates a higher diversity of the original dataset. In one example, the first index can be represented by M1.

[0073] In an example, the data diversity evaluation device 201 may use mean inception jensen-shannon divergence (Mean_Inception_JS) to first evaluate the diversity of the original data set.

[0074] In one embodiment, high-dimensional data typically suffers from data redundancy and the widespread phenomenon of the "curse of dimensionality." The curse of dimensionality refers to the situation in which various problems arise when analyzing and organizing high-dimensional spaces as spatial dimensions increase and the volume exponentially increases. In the space of the original sample set, the distance between samples of the same type is often greater than the distance between samples of different types. Therefore, similarity between data cannot usually be measured directly based on distance. In some solutions, dimensionality reduction methods are often used, such as principal component analysis (PCA) or robust principle components analysis (RPCA). However, these methods also have significant drawbacks. Due to the complex real-world scenarios and the limited expressive power of linear mappings, they are not suitable for nonlinear data. Commonly used nonlinear mappings, such as kernel functions, can calculate nonlinear distances between samples, but they do not know the specific high-dimensional space into which they are projected, resulting in limited applicability.

[0075] In one example, convolutional neural networks have performed very well in multiple computer data fields such as image recognition and classification. Convolutional neural networks are a feedforward neural network whose artificial neurons can respond to surrounding units within a part of the coverage area, and have excellent performance in large-scale image processing. At present, the port (Inception) network is a convolutional neural network model with both width and depth, which has strong classification capabilities and is relatively practical. The inception network provides an effective feature extraction method. For high-dimensional image data, after mapping through the inception network, an effective representation form of the image in the category space can be obtained, and the redundant dimension of the data is reduced. Among them, the category space can be understood as a collection of multiple categories. This application uses the inception network to map data to the category space, and proposes a method to measure the distance between data in the category space, namely the port Jason Shannon divergence (Inception Jensen-Shannon divergence, Inception_JS).

[0076] In one embodiment, the data of the original data set is first fed forward through a pre-trained Inception network to obtain the output vector of the last layer of the Inception network, the softmax layer. Each element in the output vector represents the probability value of the data input to the Inception network belonging to a different category, and the entire vector can be regarded as a discrete probability distribution density function. Therefore, the Jensen-Shannon divergence can be used to measure the distance between probability distributions. In one example, the Jensen-Shannon divergence is a symmetry measure that measures the difference between two probability distributions. Therefore, after the data of the original data set passes through the Inception network to obtain the probability distribution vector in the category space, the Jensen-Shannon divergence can be used to measure the distance between the probability distributions, which gives Inception_JS. In an example, Inception_JS can be shown as follows,

[0077]

[0078] p x =Inception(x),p y =Inception(y)

[0079] Among them, inception() represents the inception network, p x The probability distribution vector in the category space is obtained by mapping the data x through the inception network, p yThe probability distribution vector in the category space is obtained by mapping the data y through the Inception network. Inception_JS(x,y) calculates the distance between the two samples x and y.

[0080] When we want to measure the diversity of a dataset of the same category, we need to calculate the Inception_JS distance between any two samples in the dataset and then calculate the average. Therefore, this application defines the sum of the Inception_JS divergences of any two samples in the dataset, namely Mean_Inception_JS. In an example, Mean_Inception_JS can be expressed as follows:

[0081]

[0082] Among them, (x,y)~p r are any two sample data in the dataset.

[0083] It is understandable that Mean_Inception_JS can be used to describe the diversity of the data set. The larger the value of Mean_Inception_JS, the higher the diversity of the data set.

[0084] like Figure 5 Shown, Figure 5 A schematic diagram of the diversity evaluation of the mean port Jason Shannon divergence provided in the embodiment of the present application. It can be seen that the Inception_JS divergence provides a more effective measurement method. When evaluating a data set, when the number of categories in the data set remains unchanged, as the number of samples increases, the value of Mean_Inception_JS also increases. For example Figure 5 The horizontal axis represents the number of samples, and the vertical axis represents the Mean_Inception_JS value. When the number of evaluated samples reaches a certain value, the dataset diversity reaches saturation. It can be seen that the dataset diversity value also approaches a plateau at this point. Clearly, Mean_Inception_JS is very effective in assessing dataset diversity.

[0085] S403: Extract features from the original data set.

[0086] In one embodiment, the data diversity evaluation device 201 performs feature extraction on the data of the original data set.

[0087] In one example, the data diversity assessment device 201 performs feature extraction on the data in the original dataset. Feature extraction is a concept in vision and image processing that refers to extracting image information using a terminal device and converting it into a digital vector for representation. Because the data in the original dataset is complex and may contain a large amount of irrelevant information, the corresponding features of the data in the original dataset can be extracted to ignore the irrelevant information in the original dataset and achieve data dimensionality reduction, thereby reducing the amount of data required for subsequent calculations.

[0088] In one example, after S403, S405' can be directly performed (not shown in the figure).

[0089] S405 ′: Generate first expanded data using the original data set.

[0090] In one embodiment, the GAN module 2022 in the GAN data generation device 202 generates first augmented data using data in the original dataset. The first augmented data is generated from the original dataset using the GAN. In one example, the first augmented data can be represented by X'.

[0091] S406 may be executed after S405 ′.

[0092] In one example, since the data in the original data set is numerous and complex, there is a certain probability that the generated data cannot well increase the diversity of the original data set. In order to better generate the additional data, the present application may also perform S404 after S403.

[0093] S404 , using a local outlier factor (LOF) algorithm to detect the data in the original dataset after feature extraction to obtain an outlier set.

[0094] In one embodiment, the LOF module 2021 in the GAN data generation device 202 performs outlier detection on the data of the original data set after feature extraction through a local outlier factor LOF algorithm to obtain an outlier set.

[0095] In one example, the LOF module 2021 in the GAN data generation device 202 receives the data of the original dataset after feature extraction sent by the dataset diversity assessment device 201, and then uses the LOF algorithm to perform outlier detection, thereby detecting all outlier data in the data of the original dataset after feature extraction, and forming an outlier set with the detected outlier data. It should be noted that in the feature space, the proximity between an outlier and its nearest neighbor significantly deviates from the proximity between other data in the dataset and its own nearest neighbor; the outlier set is the collection of all outliers, so it can generally be considered that outliers are samples in the dataset that are prone to error in classification methods and are relatively small in number.

[0096] In another example, Figure 6a Shown, Figure 6a This is a schematic diagram of a two-dimensional distribution of an original data set provided in an embodiment of the present application. First, for all images in the original data set, the data is first mapped to a two-dimensional plane using isomap. Among them, isomap is an unsupervised algorithm for nonlinear data dimensionality reduction. Then each image in the original data set is mapped to a point, and the following is formed: Figure 6b The set of points shown. Figure 6b A schematic diagram of a two-dimensional point set distribution provided in an embodiment of the present application is shown. Figure 6b After the feature-extracted data is mapped to a two-dimensional plane, points of the same category are clustered into a point set of that category, for example Figure 6b The different shades of color represent different categories of point sets. Then the LOF algorithm is used to Figure 6b Perform outlier detection on the point set in Figure 6c The two-dimensional distribution of outliers is shown in the coordinate diagram on the right. Figure 6c A schematic diagram of a two-dimensional distribution of outlier points provided in the embodiment of the present application is shown in FIG. Figure 6c All points in the right coordinate graph are displayed with the X-axis as the sample point ID mark and the Y-axis as the coordinate of the outlier factor distribution, that is, Figure 6c The left coordinate diagram shows that, assuming the Y-axis value is 250 as the dividing line, the four points on the top are outliers, which can be corresponded to Figure 6c The four outliers shown in the right coordinate diagram are light grey circles. It is worth noting that Figure 6a-6c Only 20 categories are selected from the original dataset for illustration.

[0097] Continue back Figure 4 .

[0098] S405: Generate first extended data using the outlier point set.

[0099] In one embodiment, the GAN module 2022 in the GAN data generation device 202 generates first augmented data using the outlier point set obtained by the LOF module 2021. The first augmented data is generated from the outlier point set through the GAN. In one example, the first augmented data can be represented by X'.

[0100] In some schemes, image enhancement methods for supervised learning applications are implemented. Supervised learning is the process of adjusting the parameters of a classifier using a set of samples of known categories to achieve the required performance. First, the original data set to be enhanced and the annotation information are obtained; then, according to the supervised learning application task, the data enhancement parameters are set; then, the image data and label data in the original data set are respectively enhanced; then, the enhanced images and annotation information are filtered; finally, the filtered images and annotation information are stored in the original data set. There are many ways to enhance data in this scheme, and the specific enhancement method is strongly related to the specific task. It needs to be set manually and cannot be processed automatically. At the same time, there are many parameters in the various data enhancement methods that need to be set, and adaptive adjustment cannot be achieved.

[0101] Therefore, the present application uses GAN for autonomous learning and continuously corrects parameters through the loss function to achieve adaptive adjustment. Among them, the loss function is a function that maps the value of a random event or its related random variables to a non-negative real number to represent the "risk" or "loss" of the random event. In the field of machine learning, learning through a loss function is a method to evaluate the degree to which a specific algorithm models given data. If the predicted value deviates far from the actual result, the loss function will obtain a very large value. With the assistance of some optimization functions, the loss function gradually learns to reduce the error of the predicted value. Generally, GAN includes two network structures: a generator and a discriminator, which can also be called a generative network and a discriminative network. The generative network is a network structure that uses random variables to generate images. Ideally, the generated image should be very similar to the real image and cannot be distinguished. The discriminative network is a metric network used to distinguish between real images and generated images. The ideal steady state for GAN is to reach a Nash equilibrium so that the probability distribution function of the generated image approaches the real distribution. However, there are still many potential problems with GANs. For example, in high-dimensional space, the distribution function of real images and the distribution function of generated images usually do not intersect. In this case, general distance functions such as relative entropy are usually constant, and the gradient exhibits random directions, making training unstable and difficult to converge. Among them, relative entropy can also be called Kullback-Leibler (KL) divergence, which can be used to measure the asymmetry of the difference between two probability distributions. Existing GANs still have many shortcomings in the task of synthesizing real-world scenes, such as high training difficulty, unstable training process, low generated image quality, weak semantic information, and other problems.

[0102] GAN is a method of unsupervised learning that learns by playing games between two neural networks. Figure 7 Shown, Figure 7A diagram of the structure of a generative adversarial network provided in an embodiment of the present application. GAN consists of a generative network G and a discriminant network D. In one example, GAN randomly samples from a latent space to obtain random data as the input of the generative network G. Among them, the latent space is the eigenspace of high-dimensional data. The output result of the generative network G must imitate the real sample data in the training set as much as possible. For the discriminant network, its input is the real data and the result of the generative network output, and the real data and the result of the generative network output are discriminated to distinguish the result of the generative network output. During the GAN training phase, the loss function will be continuously adjusted, and the parameters will be continuously adjusted through the mutual confrontation between the two networks, so that the discriminant network will eventually be unable to determine whether the data output by the generative network is true or false. Among them, the loss function is a function that maps the value of a random event or its related random variable to a non-negative real number to represent the "risk" or "loss" of the random event.

[0103] As mentioned above, because the distribution function of real images and the distribution function of generated images usually do not intersect in high-dimensional space, the training process of GAN is relatively unstable, making training difficult and prone to producing blurry, low-quality images. In this case, since the distance function in GAN is usually a constant, it causes the gradient to have random directions, making training unstable and difficult to converge. Among them, the distance function usually refers to the loss function in GAN, which is converted into the distance between the image produced by the generator network and the real image in order to obtain the maximum value of the discriminant network D.

[0104] Therefore, this application constructs a new loss function by adding Inception_JS and kernel maximum mean discrepancy (Kernel_MMD) regularization terms to control the degree of difference between the probability distribution of the real image and the probability distribution of the generated image. Among them, the regularization term often leads to overfitting when there is not enough training data or overtraining. At this time, the regularization term introduces additional information into the original model to prevent overfitting and improve the generalization performance of the model. At the same time, this application can measure the similarity of two probability distributions through Inception_JS, which is simple to calculate and solves the asymmetry problem of KL divergence.

[0105] In one example, the GAN module 2022 uses the outlier point set output by the LOF module 2021 to generate a new adversarial image, i.e., the first augmented data, through the GAN. In another example, the GAN module 2022 may further include a port Jason-Shannon divergence submodule 20221 and a kernel maximum mean difference submodule 20222. The port Jason-Shannon divergence submodule 20221 may use Inception_JS, and the kernel maximum mean difference submodule 20222 may use Kernel_MMD to control the difference between the generated image and the real image.

[0106] In another example, the loss function constructed in the GAN module 2022 can be expressed as follows:

[0107]

[0108]

[0109] where z~p z Represents random data, x~p r represents the real data, G() represents the output data of the generator network G, D() represents the output data of the discriminator network D, and K_MMD() is the kernel_MMD mentioned above. β and γ are hyperparameters between [0, 1], where hyperparameters are pre-set parameters that do not change during the training process.

[0110] In one example, the objective function of Kernel_MMD can be used to control the generated image density distribution function p g And the real image density distribution function p r The difference between the two. The generated image density distribution function represents the density distribution function between multiple images in the generated data generated by GAN; the real image density distribution function represents the density distribution function of multiple images in the real data. For example, Kernel_MMD can be expressed as follows:

[0111]

[0112] Wherein, K(,) is a kernel function, which can generally be a Gaussian kernel function or a Laplace kernel function. Kernel functions can include linear kernel functions, polynomial kernel functions, and Gaussian kernel functions. Kernel functions can map data to infinite dimensions and are also called radial basis functions. Of course, those skilled in the art should note that any other equivalent alternative functions can also be used, and this application does not limit them here.

[0113] In one embodiment, the two network structures included in the GAN—the generator network G and the discriminator network D—need to follow certain design principles, such as:

[0114] (1) The network structure of the discriminant network D cannot use pooling layers, only convolution layers (strided convolutions) are used, and the network structure of the generative network G needs to use transposed convolution layers (transposed convolutions).

[0115] (2) Both the generator network G and the discriminator network D need to use batch normalization layers to solve the problem that the distribution of internal node data changes due to changes in network parameters during network training, and try not to use fully connected layers.

[0116] (3) The activation function of the intermediate layer of the generated network G can use the linear rectification function (rectified linear unit, ReLu), and the last layer of the generated network G uses the activation function Tanh.

[0117] (4) Each layer in the discriminant network D uses the optimized ReLu function, that is, the LeakyReLu function.

[0118] In one example, to generate a 64*64 image, the input of the generator network G can be a random vector with a dimension of 100. In another example, the input can be a discrete point group output by the LOF module. The specific network structure of the generator network G and the discriminator network D can be shown in Table 1.

[0119]

[0120] Table 1

[0121] It can be seen that in the generative network G, the 100-dimensional random vector input by the first Convtranspose layer is repeatedly convolved by the BatchNorm layer, the ReLU layer, and the Tanh layer, and finally outputs data of dimension 3. The output of the generative network is then used as the input of the discriminant network D, which is also repeatedly convolved by the BatchNorm layer, the LeakyReLU layer, and the Sigmoid layer, and finally outputs data of dimension 1, that is, it is determined whether the input of the discriminant network D is true or false. Among them, in the network structure shown in Table 1, the length and width of the convolution kernel of the convolution layer can be 4*4. Of course, those skilled in the art should note that the length and width of the convolution kernel can also be other combinations, which are not limited in this application.

[0122] like Figure 8 As shown, Figure 8 This is a schematic diagram of data generation by a generative adversarial network according to an embodiment of the present application. Figure 8It can be seen that the image generated by the GAN module 2022 should ideally be very similar to the real image, that is, the original image, so that the discriminant network D cannot discriminate the true situation of the image generated by the generating network G.

[0123] This application generates new data by inputting a set of outliers into GAN, while the outlier set is calculated from the original dataset, thereby solving the problem that some current FANs are only applicable to specific task scenarios and have low versatility.

[0124] Continue back below Figure 4 .

[0125] S406 , combining the first expanded data and the data in the original data set, and using the average port Jason-Shannon divergence for evaluation to obtain a second indicator.

[0126] In one embodiment, the GAN data generation device 202 combines the first augmented data generated by the GAN module 2022 with the original dataset. Then, the second mean-port Jason-Shannon divergence module 2023 uses the mean-port Jason-Shannon divergence to evaluate the data to obtain a second index. The second index is used to represent the diversity of the original dataset after combining the first augmented data.

[0127] In one example, the GAN data generation device 202 superimposes the first augmented data generated by the GAN module 2022 with the data in the original dataset, and then the second mean port Jason Shannon divergence module 2023 can use Mean_Inception_JS to perform diversity evaluation to obtain a second indicator. The second indicator can be represented by M2.

[0128] S407: Determine whether the second indicator is greater than the first indicator.

[0129] In one embodiment, the second average port Jason-Shannon divergence module determines whether the second indicator is greater than the first indicator. When the second indicator is greater than the first indicator, it is determined that the diversity of the original dataset has been significantly improved after the addition of the first augmented data, and execution continues to S408. In one example, when M2 is greater than M1, it can be determined that the diversity of the original dataset has been significantly improved after the addition of the first augmented data generated by the GAN module 2022. This method allows the addition of new data to the original dataset to continuously maintain the diversity of the dataset, and does not reduce the diversity of the dataset due to the introduction of new data, thereby solving the problem of data redundancy.

[0130] In another embodiment, if it is determined that the second indicator is less than or equal to the first indicator, it is determined that the diversity of the original data set after adding the first expanded data will not be significantly improved. At this time, it is possible to return to S404 and obtain a new outlier set by changing the threshold in the LOF algorithm, and perform calculations to obtain new expanded data.

[0131] S408: Add the first expanded data to the original data set to obtain a first data set.

[0132] In one embodiment, the first expanded data is added to the original data set to form a first data set. In one example, the first data set can also be referred to as a new data set.

[0133] S409: Replace the original data set with the first data set.

[0134] In one embodiment, the first data set is used to replace the original data set. In one example, since the diversity of the original data set is improved due to the addition of the first expanded data, the first expanded data is added to the original data set to form a new data set, and the new data set is used to replace the original data set. In another example, the first expanded data can also be directly added to the original data set to form a new original data set, and used as the original data set in subsequent operations. Those skilled in the art should note that the core purpose of S409 is to add the generated first expanded data to the original data set to continuously expand the original data set. This application does not limit whether there is a specific replacement step.

[0135] In one example, a classifier is used to classify the data in a dataset, and the accuracy of the classifier classification is used to verify the completeness and diversity of the training dataset. The classifier is a classification function generated or a classification model constructed by learning based on existing data. For a relatively complete and diverse training dataset, training the classifier can ensure that the classification results of the classifier have a high accuracy rate. For example Figure 9 Shown, Figure 9 A diagram showing the comparison of classification accuracy of a dataset provided in the embodiment of this application. Figure 9 In this paper, the original data set, the enhanced data set expanded by traditional data enhancement methods, and the enhanced data set using this application are given. Figure 4 The enhanced dataset is augmented and expanded using the method shown above. The extreme gradient boosting (xgboost) classifier is trained using the above different datasets, and the test accuracy curve is obtained. Figure 9 The horizontal axis indicates the number of iterations of the training process, and the vertical axis indicates the classification accuracy. Figure 9The first plotted line shows the accuracy curve for the original dataset, the second plotted line shows the accuracy curve for the traditional enhanced dataset, and the third plotted line shows the accuracy curve for the enhanced dataset of the present application. The original dataset contains 40,000 sample data, the traditional enhanced dataset contains 50,000 sample data, and the enhanced dataset of the present application contains 50,000 sample data. It can be clearly seen that the method described in the present application continuously expands the original dataset, thereby increasing the data in the dataset while maintaining its diversity. When the classifier is trained using the expanded dataset, the accuracy of the classifier can be greatly improved. Under the same number of iterations, the accuracy of the classifier trained with the enhanced dataset of the present application not only greatly exceeds that of the classifier trained with the original dataset, but also, under the same 50,000 sample data, the accuracy of the classifier trained with the enhanced dataset of the present application still exceeds that of the classifier trained with the traditional enhanced dataset. It can be seen that expanding the original dataset using the method described in the present application can increase the completeness of the dataset and effectively improve the generalization ability of the classifier.

[0136] This application introduces the Inception_JS metric and Kernel_MMD as regularization terms. The similarity between the images generated by the generative network G and the real images is evaluated in the loss function of the GAN, and a stable training process is achieved. At the same time, the generated images are evaluated using Mean_Inception_JS to determine whether they are suitable for addition to the original dataset. That is, when new data is added to the original dataset, the newly generated data is added only when the Mean_Inception_JS of the dataset increases. Therefore, the problem of data redundancy can be solved. At the same time, since new data is only added when the diversity of the original dataset increases, the problem of expanding data diversity is also solved.

[0137] like Figure 10 As shown, Figure 10 A flowchart of another data processing method provided in an embodiment of the present application.

[0138] like Figure 10 As shown, after S402, if the data in the original data set has a label, the following steps may be further performed:

[0139] S1001: For the labels in the original dataset, introduce data with the same labels from the public dataset.

[0140] In some schemes, an image data enhancement scheme is implemented. First, an image to be enhanced and an auxiliary image are obtained, wherein the image to be enhanced and the auxiliary image have the same or similar data distribution. Then, the auxiliary image is used to superimpose preset color channels on the image to be enhanced. Finally, the image obtained by superimposing the preset color channels is determined as the generated enhanced image. Among them, the channel that stores the color information of the image is called the color channel. However, in this scheme, simple color channel superposition will cause the original features of the image to change too much, and it is very likely to change the semantic information of the image. At the same time, the color channel superposition method will introduce information that is irrelevant to the category itself, thereby reducing the classification accuracy of the neural network.

[0141] Therefore, the present application can also expand the data by identifying the category labels in the original data set, so that the introduced data does not affect the classification accuracy of the neural network.

[0142] In one embodiment, if the data in the original data set has a label, the public data set introduction device can extract data with the same label from the public data set as the introduced new data. The same label means that the label of the data in the public data set is the same as the label of the data in the first data set; in another example, the same label can also include similar labels, that is, the label of the data in the public data set belongs to a subset of the label of the data in the original data set. In one example, for example, the data label of the first data set is "lamp", then the label of the data introduced from the public data set is also "lamp", or it can be a subset of the label "lamp", such as "desk lamp", "floor lamp", "chandelier", etc. Those skilled in the art should note that whether the labels are exactly the same or the labels are similar, they can be referred to as the same labels in this application.

[0143] In one example, if the data in the original dataset has labels, before executing S1001, in S402, Mean_Inception_JS can be used to perform diversity assessment on the data of each label category in the original dataset. Then, S1001 is executed separately for data of different label categories.

[0144] S1002: Extract features from data in a public dataset with the same label.

[0145] In one embodiment, feature extraction is performed on data in a public dataset with the same label.

[0146] In one example, the public dataset importing device 203 extracts features from data with the same label in the public dataset, thereby ignoring information in the dataset that is irrelevant to the features and achieving data dimensionality reduction, thereby reducing the amount of data for subsequent calculations.

[0147] S1003, combines the data in the public dataset with the same label and the data in the original dataset, and uses the average port Jason Shannon divergence for evaluation to obtain the third indicator.

[0148] In one embodiment, the public dataset importing device 203 combines data with the original dataset that have the same label in the public dataset. Then, the third average port Jason-Shannon divergence module 2032 uses the average port Jason-Shannon divergence to evaluate the data and obtain a third index. The third index is used to represent the diversity of the original dataset after combining the data with the same label in the public dataset.

[0149] In one example, the public dataset introduction device 203 superimposes data with the same label in the public dataset with data in the original dataset, and then the third mean port Jason Shannon divergence module 2032 can use Mean_Inception_JS to perform diversity evaluation to obtain a third indicator. The third indicator can be represented by M3.

[0150] S1004, determining whether the third index is greater than the first index.

[0151] In one embodiment, the third average port Jason Shannon divergence module 2032 determines whether the third indicator is greater than the first indicator. When the third indicator is greater than the first indicator, it is determined that the diversity of the first dataset has been significantly improved after the data with the same label in the public dataset is added, and the process continues to execute S1005. In one example, when M3 is greater than M1, it can be determined that the diversity of the original dataset has been significantly improved after the data with the same label in the public dataset is added. Through the above method, the new data added to the original dataset can continuously maintain the diversity of the dataset, and the diversity of the dataset will not decrease due to the introduction of new data, thereby solving the problem of data redundancy.

[0152] In another embodiment, if it is determined that the third indicator is less than or equal to the first indicator, it is determined that the diversity of the original data set will not be significantly improved after adding data with the same label in the public data set. At this time, it is possible to return to S1001 and select another public data set to obtain new expanded data.

[0153] S1005: Use data in the public dataset with the same label as the second expanded data.

[0154] In one embodiment, if the third indicator is greater than the first indicator, data in the public dataset with the same label is used as the second expanded data. In one example, the second expanded data can be represented by X".

[0155] S1006: Add the second expanded data to the original data set to obtain a second data set.

[0156] In one embodiment, the second expanded data is added to the original data set to form a second data set. In one example, the second data set may also be referred to as a new data set.

[0157] S1007: Replace the original data set with the second data set.

[0158] In one embodiment, the second data set is used to replace the original data set. In one example, since the diversity of the original data set is improved due to the added second expanded data, the second expanded data is added to the original data set to form a new data set, and the new data set is used to replace the original data set. In another example, the second expanded data can also be directly added to the original data set to form a new original data set, and used as the original data set in subsequent operations. Those skilled in the art should note that the core purpose of S1007 is to add the generated second expanded data to the original data set to achieve continuous expansion of the original data set. This application does not limit whether there is a specific replacement step.

[0159] Those skilled in the art should note that Figure 10 The method shown is to perform data expansion for a certain category of labels in the original data set. When there are multiple categories of labels in the original data set, the method is performed for each category of labels. Figure 10 The method shown is sufficient.

[0160] like Figure 11 As shown, Figure 11 A flowchart of another data processing method provided in an embodiment of the present application.

[0161] like Figure 11 As shown, after S1005, the following steps may be further performed for the second extended data:

[0162] S1101: Generate third extended data using GAN on the second extended data.

[0163] In one embodiment, the second augmented data is input into the GAN module 2022 in the GAN data generating device 202, so that the GAN module 2022 generates the third augmented data by taking the second augmented data as input.

[0164] In one example, X" is input to the GAN module 2022, which then generates third augmented data based on the input X". The third augmented data can also be referred to as new data. In one example, the third augmented data can be represented by the same symbol X' as the first augmented data. Those skilled in the art should note that in this application, X' is used to represent all augmented data generated by the GAN module, which can be either the first augmented data or the third augmented data. X" is used to represent all augmented data generated by non-GAN modules, which can be either the second augmented data.

[0165] S1102 , combining the third expanded data and the data in the original data set, and using the average port Jason-Shannon divergence for evaluation to obtain a fourth indicator.

[0166] In one embodiment, the GAN data generation device 202 combines the third augmented data with the original dataset, and then the second mean-port Jason-Shannon divergence module 2023 uses the mean-port Jason-Shannon divergence to evaluate and obtain a fourth index. The fourth index is used to represent the diversity of the original dataset after combining the third augmented data.

[0167] In one example, the GAN data generating device 202 superimposes the third augmented data with the data in the original data set, and then the second mean port Jason Shannon divergence module 2023 can use Mean_Inception_JS to perform diversity evaluation to obtain a fourth indicator. The fourth indicator can be represented by M4.

[0168] S1103, determine whether the fourth index is greater than the first index.

[0169] In one embodiment, the second average port Jason-Shannon divergence module 2023 determines whether the fourth indicator is greater than the first indicator. When the fourth indicator is greater than the first indicator, it is determined that the diversity of the original dataset has been significantly improved after the third augmented data is added, and the process continues with S1104. In one example, when M4 is greater than M1, it can be determined that the diversity of the original dataset has been significantly improved after the third augmented data is added. This method allows the addition of new data to the original dataset to continuously maintain the diversity of the dataset, and does not reduce the diversity of the dataset due to the introduction of new data, thereby solving the problem of data redundancy.

[0170] In another embodiment, if the fourth index is determined to be less than or equal to the first index, then it is determined that the diversity of the first dataset will not be significantly improved after adding the third extended data. In this case, the process can return to S1001 and select another public dataset to obtain new extended data.

[0171] S1104: Add the third expanded data to the original data set to obtain a third data set.

[0172] In one embodiment, the third expanded data is added to the original data set to form a third data set. In one example, the third data set may also be referred to as a new data set.

[0173] S1105: Replace the original data set with the third data set.

[0174] In one embodiment, the third data set is used to replace the original data set. In one example, since the added third expanded data has improved the diversity of the original data set, the third expanded data is added to the original data set to form a new data set, and the new data set is used to replace the original data set. In another example, the third expanded data can also be directly added to the original data set to form a new original data set, and used as the original data set in subsequent operations. Those skilled in the art should note that the core purpose of S1105 is to add the generated third expanded data to the original data set to achieve continuous expansion of the original data set. This application does not limit whether there is a specific replacement step.

[0175] This application Figures 2 to 11 The method shown, by proposing a way to measure the distance between high-dimensional data samples, namely Inception_JS divergence, can measure the distance between samples in the category space and ensure the validity of the distance metric. At the same time, on this basis, an indicator Mean_Inception_JS is proposed to measure the diversity of the data set. This application first uses the LOF algorithm to filter out the outlier point set, and then uses the filtered outlier point set in combination with GAN to generate new data. At the same time, this scheme introduces the regularization terms of Inception_JS and Kernel_MMD in the loss function of GAN to control the difference between the generated image and the real image distribution. When the newly generated data is introduced into the original data set, the Mean_Inception_JS indicator is used to evaluate whether the diversity is improved. This application also adds samples from the public data set to expand the original data set, and uses Mean_Inception_JS as the evaluation indicator to select samples that can increase the diversity of the original data set to add, so as to prevent the addition of redundant data and negatively affect the training of the network model.

[0176] Those skilled in the art should note that the GAN network structure involved in the technical solution of this application can also be used to generate medical images and other scenarios, and the idea of ​​the overall solution can also be used for the continuous acquisition of data in other formats in intelligent data systems, which is not limited in this application.

[0177] Those skilled in the art should note that the data types in the original and new datasets in this application can be images, audio, text, and so on. When the data type is non-image, the Inception network can be replaced with another network suitable for the data type. Furthermore, this application generates augmented data to expand the original dataset, so that the expanded original data can be used to train a more complete classification model, thereby achieving more accurate image classification and image recognition.

[0178] like Figure 12 As shown, Figure 12 A schematic diagram of a data processing device provided in an embodiment of the present application.

[0179] Figure 12 A data processing device 1200 is shown, and the device 1200 includes: an acquisition module 1201, used to obtain an original data set; an evaluation module 1202, used to use the average port Jason Shannon divergence to evaluate the first data set to obtain a first indicator, and the first indicator is used to represent the diversity of the original data set; a first generation module 1205, used to input the data in the original data set into the generative adversarial network (GAN) and generate first expanded data; the evaluation module 1202 is also used to combine the first expanded data and the data in the original data set, and use the average port Jason Shannon divergence to evaluate to obtain a second indicator, and the second indicator is used to represent the diversity of the original data set after combining the first expanded data; an adding module 1206, used to add the first expanded data to the original data set when the second indicator is greater than the first indicator to obtain the first data set; and a replacement module 1207, used to replace the original data set with the first data set.

[0180] In one possible embodiment, the apparatus 1200 further includes: a feature extraction module 1203 for performing feature extraction on the data of the original data set; a detection module 1204 for detecting the data of the original data set after feature extraction using a local outlier factor (LOF) algorithm to obtain an outlier set; and a first generation module 1205 for inputting the data in the outlier set into a GAN and generating first augmented data.

[0181] In one possible implementation, the first generation module 1205 includes: a generative adversarial network (GAN); a loss function used by the GAN has multiple parameters, wherein the parameters include port Jason-Shannon divergence and kernel maximum mean difference.

[0182] In one possible embodiment, the device 1200 also includes: an introduction module 1209, which is used to introduce data with the same label as the label in the original data set from the public data set; the feature extraction module 1203 is also used to extract features from the data in the public data set with the same label; the evaluation module 1202 is also used to combine the data in the public data set with the same label and the data in the original data set, and use the average port Jason Shannon divergence to evaluate to obtain a third indicator, which is used to represent the diversity of the original data set after combining the data in the public data set with the same label; the second generation module 1208 is used to use the data in the public data set with the same label as the second expanded data when the third indicator is greater than the first indicator.

[0183] In one possible implementation, having the same label includes: the label in the public dataset is the same as the label in the first dataset; or the label in the public dataset belongs to a subset of the labels in the original dataset.

[0184] In a possible implementation, the adding module 1206 is further configured to: add the second expanded data to the original data set to obtain a second data set; and the replacing module 1307 is further configured to replace the original data set with the second data set.

[0185] In one possible implementation, the first generation module 1205 is further used to: use GAN to generate third augmented data for the second augmented data; the evaluation module is further used to combine the third augmented data and the data in the original data set, and use the average port Jason Shannon divergence for evaluation to obtain a fourth indicator, which is used to represent the diversity of the original data set after combining the third augmented data; the adding module 1206 is further used to, when the fourth indicator is greater than the first indicator, add the third augmented data to the original data set to obtain a third data set; the replacement module 1207 is further used to replace the original data set with the third data set.

[0186] In a possible implementation, the data type of the data in the original data set, the first data set, the second data set, and / or the third data set is a picture type.

[0187] like Figure 13 As shown, Figure 13 A schematic diagram of a terminal device provided in an embodiment of the present application.

[0188] Figure 13A terminal device 1300 is provided, which may include a processor 1301, a memory 1302, a communication interface 1303, and a bus 1304. The processor 1301, the memory 1302, and the communication interface 1303 in the terminal device may establish a communication connection via the bus 1304. The communication interface 1303 is used to send and receive external information.

[0189] The processor 1301 may be a central processing unit (CPU).

[0190] Memory 1302 may include volatile memory, such as random-access memory (RAM); memory 1302 may also include non-volatile memory (English: non-volatile memory), such as read-only memory (ROM), flash memory, hard disk drive (HDD) or solid state drive (SSD); memory 1302 may also include a combination of the above types of memory.

[0191] above Figures 2 to 11 The embodiment of the present invention provides a method for obtaining diversity data, which is executed by the processor 1301. The file data and / or calculated data in this application will be stored in the memory 1302. In addition, the memory 1302 will also be used to store the processor to implement the above Figures 2 to 11 Program instructions corresponding to a method for obtaining diversity data described in an embodiment, etc.

[0192] This application relates to a data processing method and apparatus. By proposing the Inception_JS divergence, which provides an effective feature extraction method by passing samples through the Inception network, high-dimensional image data can be mapped through the Inception network to obtain an effective representation of the image in the category space and an effective distance between high-dimensional data. Based on this, the Mean_Inception_JS metric is proposed, which can be used to effectively evaluate the completeness and diversity of a dataset. This application proposes first using the LOF algorithm to filter out a set of outliers, then using a GAN to generate new data. Regularization terms of Inception_JS and Kernel_MMD are introduced into the GAN loss function to control the difference between the generated image distribution and the real image distribution. Furthermore, when the newly generated data is introduced into the original dataset, it is also evaluated using the Mean_Inception_JS metric. This application also adds samples from a public dataset with the same label and uses Mean_Inception_JS to select samples that can increase the diversity of the original dataset. This can prevent the addition of redundant data and negatively impact the training of the network model.

[0193] The method provided in this application can continuously acquire diverse data and obtain a complete and diverse dataset. At the same time, the obtained complete training dataset can effectively improve the generalization ability of the network model.

[0194] Those skilled in the art should further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the composition and steps of each example according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0195] Those skilled in the art will appreciate that all or part of the steps in the above-described embodiment method can be performed by instructing a processor through a program, and the program can be stored in a computer-readable storage medium, which is a non-transitory medium, such as a random access memory, a read-only memory, a flash memory, a hard disk, a solid-state drive, a magnetic tape, a floppy disk, an optical disc, and any combination thereof.

[0196] The above description is merely a preferred embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A data processing method, characterized in that: The method comprises: Get the original dataset; The original dataset is evaluated using an average port Jason Shannon divergence to obtain a first indicator, where the first indicator is used to represent the diversity of the original dataset; wherein the average port Jason Shannon divergence is the sum of the port Jason Shannon divergences (Inception_JS) of any two samples in the original dataset; the Inception_JS is obtained by mapping the original dataset to a category space using a port network and measuring the distance between data in the category space using Jason Shannon divergence; Inputting the data in the original data set into a generative adversarial network (GAN) to generate first expanded data; including: extracting features from the data in the original data set; using a local outlier factor (LOF) algorithm to detect the data in the original data set after feature extraction to obtain an outlier set; Inputting the data in the outlier set into GAN to generate the first augmented data; The generative adversarial network (GAN) includes a generative network G and a discriminative network D, wherein the network structure of the generative network G includes a transposed convolution layer, a BatchNorm layer, a ReLU layer, and a Tanh layer; the network structure of the discriminative network D includes a BatchNorm layer, a LeakyReLU layer, and a Sigmoid layer; the loss function adopted by the generative adversarial network (GAN) has multiple parameters, wherein the multiple parameters include port Jason Shannon divergence and kernel maximum mean difference; the loss function controls the degree of difference between the probability distribution of real data and the probability distribution of generated data by adding regularization terms of the port Jason Shannon divergence and kernel maximum mean difference; Combining the first expanded data and the data in the original data set, and evaluating using the average port Jason-Shannon divergence to obtain a second indicator, wherein the second indicator is used to represent the diversity of the original data set after combining the first expanded data; When the second index is greater than the first index, the first expanded data is added to the original data set to obtain a first data set; the data type of the data in the original data set and the first data set is a picture type; The original data set is replaced with the first data set.

2. The method according to claim 1, wherein The method further comprises: For the labels in the original data set, data with the same labels are introduced from a public data set; and feature extraction is performed on the data in the public data set with the same labels; Combining the data in the public dataset with the same label and the data in the first dataset, and evaluating using the average port Jason Shannon divergence to obtain a third indicator, wherein the third indicator is used to represent the diversity of the first dataset after combining the data in the public dataset with the same label; When the third indicator is greater than the first indicator, data in the public dataset with the same label is used as the second augmented data.

3. The method according to claim 2, wherein the having the same label comprises: The labels in the public dataset are the same as the labels in the original dataset; or The labels in the public dataset are a subset of the labels in the original dataset.

4. The method according to claim 2 or 3, wherein: The method further comprises: Adding the second extended data to the original data set to obtain a second data set; the data type of the data in the second data set is a picture type; The original data set is replaced with the second data set.

5. The method according to claim 2 or 3, wherein: The method further comprises: generating third augmented data using the GAN on the second augmented data; Combining the third expanded data with the data in the original data set and evaluating using the average port Jason-Shannon divergence to obtain a fourth index, wherein the fourth index is used to represent the diversity of the original data set after combining the third expanded data; When the fourth indicator is greater than the first indicator, the third extended data is added to the original data set to obtain a third data set; the data type of the data in the third data set is a picture type; The original data set is replaced with the third data set.

6. A data processing device, characterized in that: The device comprises: Acquisition module, used to obtain the original data set; An evaluation module is configured to evaluate the original dataset using an average port Jason Shannon divergence to obtain a first indicator, where the first indicator is used to represent the diversity of the original dataset; wherein the average port Jason Shannon divergence is the sum of the port Jason Shannon divergences (Inception_JS) of any two samples in the original dataset; and wherein the Inception_JS is obtained by mapping the original dataset to a category space using a port network and measuring the distance between data in the category space using Jason Shannon divergence. A first generation module is configured to input the data in the original dataset into a generative adversarial network (GAN) and generate first extended data; the module comprises: performing feature extraction on the data in the original dataset; using a local outlier factor (LOF) algorithm to detect the data in the original dataset after feature extraction to obtain an outlier set; and inputting the data in the outlier set into the GAN to generate the first extended data; The generative adversarial network GAN includes a generator network G and a discriminator network D. The network structure of the generator network G includes a transposed convolution layer, a BatchNorm layer, a ReLU layer, and a Tanh layer; the network structure of the discriminator network D includes a BatchNorm layer, a LeakyReLU layer, and a Sigmoid layer. The loss function used by the generative adversarial network (GAN) has multiple parameters, wherein the multiple parameters include port Jason Shannon divergence and kernel maximum mean difference; the loss function controls the degree of difference between the probability distribution of real data and the probability distribution of generated data by adding regularization terms of the port Jason Shannon divergence and kernel maximum mean difference; The evaluation module is further configured to combine the first expanded data and the data in the original data set and use the average port Jason-Shannon divergence to perform evaluation to obtain a second indicator, wherein the second indicator is used to represent the diversity of the original data set after combining with the first expanded data; an adding module, configured to, when the second index is greater than the first index, add the first expanded data to the original data set to obtain a first data set; the data type of the data in the original data set and the first data set is a picture type; A replacement module is configured to replace the original data set with the first data set.

7. The device according to claim 6, characterized in that The device further comprises: An introduction module is used to introduce data with the same labels as those in the original dataset from a public dataset; The feature extraction module is further used to extract features from data in the public dataset with the same label; The evaluation module is further configured to combine the data in the public dataset with the same label and the data in the original dataset, and perform evaluation using the average port Jason-Shannon divergence to obtain a third indicator, wherein the third indicator is used to represent the diversity of the original dataset after combining the data in the public dataset with the same label; The second generating module is configured to use data in the public dataset with the same label as the second expanded data when the third indicator is greater than the first indicator.

8. The apparatus according to claim 7, wherein the having the same label comprises: The labels in the public dataset are the same as the labels in the original dataset; or The labels in the public dataset are a subset of the labels in the original dataset.

9. The device according to claim 7 or 8, characterized in that The adding module is also used for: Adding the second extended data to the original data set to obtain a second data set; the data type of the data in the second data set is a picture type; The replacement module is further configured to replace the original data set with the second data set.

10. The device according to claim 7 or 8, characterized in that The first generating module is further configured to: generating third augmented data using the GAN on the second augmented data; The evaluation module is further configured to combine the third expanded data and the data in the original data set and perform evaluation using the average port Jason-Shannon divergence to obtain a fourth index, wherein the fourth index is used to represent the diversity of the original data set after combining the third expanded data; The adding module is further configured to, when the fourth indicator is greater than the first indicator, add the third extended data to the original data set to obtain a third data set; the data type of the data in the third data set is a picture type; The replacement module is further configured to replace the original data set with the third data set.

11. A computer-readable storage medium storing instructions, characterized in that: When the instruction is executed on a terminal, the terminal is caused to execute the method according to any one of claims 1 to 5.

12. A computer system, characterized in that: The computer system includes a processor coupled to a memory, and the processor reads and executes instructions in the memory, so that the computer system implements the method according to any one of claims 1 to 5.

Images