Simulation service platform data clustering and generating method, system, equipment and medium

Abstract

The invention relates to a simulation service platform data clustering and generating method and system, electronic equipment and a storage medium, and relates to the technical field of simulation data processing. The method comprises the following steps: acquiring input data, adding the input data and a preset smooth vector to generate smooth data, compressing the smooth data based on a preset nested lattice system to obtain coding information, respectively inputting the coding information into a preset discrimination network and a preset generation network, and performing clustering discrimination on the coding information by the preset discrimination network to obtain a coding result; and generating a clustering result, wherein the preset generation network generates a new data sample based on the clustering result and the coding information. According to the scheme provided by the invention, the data can be processed in parallel based on the nested lattice and the neural network, the clustering discrimination process of the neural network is optimized through nested lattice coding, meanwhile, data generation is performed by utilizing the learning ability of the neural network, the simulation data processing efficiency and accuracy are improved, and the random smoothing technology is combined, so that the simulation efficiency is improved. And the resistance to noise and abnormal values is improved.

Description

Technical Field

[0001] The present application relates to the technical field of simulation data processing, and in particular to a method, system, electronic device and storage medium for clustering and generating data of a simulation service platform. Background Art

[0002] With the rapid development of information technology, data processing and simulation technology has become an indispensable part of industrial design and manufacturing. Especially driven by artificial intelligence (AI) technology, the efficiency and accuracy of data processing and simulation have been continuously improved, providing strong support for industrial innovation and design optimization.

[0003] Data clustering is an important process in data processing, which can divide samples in a data set into several groups or "clusters" that are similar to each other. In the field of industrial design and simulation services, traditional data clustering and generation methods usually rely on a single algorithm model, such as a single neural network model, which may face problems of low efficiency or insufficient generalization ability when processing complex data structures. At the same time, traditional data clustering methods often lack effective integration with the data generation process, which limits their potential in application scenarios such as data enhancement and privacy protection. Summary of the invention

[0004] In order to solve or partially solve the problems existing in the related art, the present application provides a simulation service platform data clustering and generation method, system, electronic device and storage medium, which can process data in parallel based on nested lattices and neural networks, optimize the clustering discrimination process of neural networks through nested lattice coding, and use the learning ability of neural networks to generate data, thereby improving the efficiency and accuracy of simulation data processing.

[0005] The first aspect of the present application provides a simulation service platform data clustering and generation method, comprising: Acquire input data, and add the input data to a preset smoothing vector to generate smoothed data; Compressing the smoothed data based on a preset nested lattice system to obtain encoding information; Inputting the encoded information into a preset discriminant network and a preset generative network respectively; The preset discriminant network performs clustering discrimination based on the coded information to generate a clustering result; The preset generation network generates new data samples based on the clustering result and the encoding information.

[0006] Preferably, the preset nested lattice system includes a fine lattice and a coarse lattice, and the smoothed data is compressed based on the preset nested lattice system to obtain the coded information including: Adjust the smoothed data based on a preset estimation coefficient to generate adjusted data; quantizing the adjustment data to the fine lattice to generate quantization information; The quantized information is mapped to the coarse lattice to obtain the encoded information.

[0007] Preferably, the step of compressing the smooth data based on the preset nested lattice system to obtain the coded information comprises: Decompressing the coded information according to the preset estimation coefficient and the preset smoothing vector to generate reconstructed data; A distortion metric is calculated based on preset target data and the reconstructed data; the distortion metric is used to evaluate the reconstruction quality of the reconstructed data.

[0008] Preferably, the step of inputting the coded information into a preset discriminant network and a preset generative network respectively comprises: Determining a compression distance from the input data to preset target data according to the encoding information; The compression distance is input into the preset discrimination network and the preset generation network respectively.

[0009] Preferably, the preset discriminant network performs clustering discrimination based on the coded information, and generates a clustering result including: The preset discriminant network performs clustering discrimination based on the compression distance to generate the clustering result; The preset generation network generates new data samples based on the clustering result and the encoding information, including: The preset generation network generates the new data sample based on the clustering result and the compression distance.

[0010] Preferably, after adding the input data to a preset smoothing vector to generate smoothed data, the step further includes: A noise vector is added or subtracted based on the smoothed data.

[0011] Preferably, the method further comprises: adjusting the preset generation network according to the clustering result; or, Adjust the preset discriminant network according to the new data sample; or, The preset smoothing vector or the preset estimation coefficient is adjusted according to the new data sample.

[0012] The second aspect of the present application provides a simulation service platform data clustering and generation system, including: A preprocessing module, used for acquiring input data, adding the input data to a preset smoothing vector, and generating smoothed data; A compression module, used for compressing the smooth data based on a preset nested lattice system to obtain encoding information; An input module, used to input the encoding information into a preset discrimination network and a preset generation network respectively; A clustering module, used for the preset discriminant network to perform clustering discrimination on the coded information and generate a clustering result; A generation module is used for the preset generation network to generate new data samples based on the clustering result and the encoding information.

[0013] A third aspect of the present application provides an electronic device, including: Processor; and The memory stores executable codes thereon, and when the executable codes are executed by the processor, the processor is caused to execute the method as described above.

[0014] A fourth aspect of the present application provides a computer-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of an electronic device, the processor is caused to execute the method as described above.

[0015] The technical solution provided by the present application may include the following beneficial effects: The embodiment of the present application discloses a simulation service platform data clustering and generation method, including obtaining input data, adding the input data to a preset smoothing vector to generate smoothed data, compressing the smoothed data based on a preset nested lattice system to obtain coding information, and inputting the coding information into a preset discriminant network and a preset generation network respectively, the preset discriminant network performs clustering discrimination on the coding information to generate a clustering result, the preset generation network generates new data samples based on the clustering result and the coding information, and can process data in parallel based on the nested lattice and the neural network, optimize the clustering discrimination process of the neural network through the nested lattice coding, and use the learning ability of the neural network to generate data at the same time, thereby improving the efficiency and accuracy of simulation data processing, and combining with random smoothing technology to improve the resistance to noise and outliers.

[0016] The technical solution of the present application can also: use nested lattice coding technology to calculate the compression distance between data, and use it as an input feature for neural network discrimination and generation, thereby achieving a close integration of clustering and generation processes and improving the pertinence and effectiveness of data generation.

[0017] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and other objects, features and advantages of the present application will become more apparent by describing in more detail the exemplary embodiments of the present application in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the present application.

[0019] Figure 1 It is a flow chart of a simulation service platform data clustering and generation method shown in an embodiment of the present application; Figure 2 It is another flow chart of the simulation service platform data clustering and generation method shown in the embodiment of the present application; Figure 3 It is a flow chart of a simulation service platform data clustering and generation method shown in an embodiment of the present application; Figure 4 is another flow chart of the simulation service platform data clustering and generation method shown in the embodiment of the present application; Figure 5 It is a structural diagram of a simulation service platform data clustering and generation system shown in an embodiment of the present application; Figure 6 It is a schematic diagram of the structure of an electronic device shown in an embodiment of the present application. DETAILED DESCRIPTION

[0020] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0021] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0022] It should be understood that although the terms "first", "second", "third", etc. may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0023] Data clustering is an important process in data processing, which can divide samples in a data set into several groups or "clusters" that are similar to each other. In the field of industrial design and simulation services, traditional data clustering and generation methods usually rely on a single algorithm model, such as a single neural network model, which may face problems of low efficiency or insufficient generalization ability when processing complex data structures. At the same time, traditional data clustering methods often lack effective integration with the data generation process, which limits their potential in application scenarios such as data enhancement and privacy protection.

[0024] In response to the above problems, an embodiment of the present application provides a simulation service platform data clustering and generation method, which can process data in parallel based on nested lattices and neural networks, optimize the clustering discrimination process of neural networks through nested lattice encoding, and use the learning ability of neural networks to generate data, thereby improving the efficiency and accuracy of simulation data processing.

[0025] The technical solution of the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0026] Figure 1 It is a flow chart of the simulation service platform data clustering and generation method shown in the embodiment of the present application.

[0027] See also Figure 1 , the method comprising: Step 101, obtaining input data, adding the input data to a preset smoothing vector, and generating smoothed data; The input data of this application can be training samples for model training, raw data that needs to be encrypted, or data that needs to be detected for anomalies. The input data can be obtained through sensor measurement, database query, or user input.

[0028] For the input data obtained, the input data can be preprocessed first, and the input data is added to the preset smoothing vector to generate smoothed data, aiming to reduce errors caused by noise or abnormal values ​​and improve the adaptability to the input data. In an example, the input data can be original data X or observed data Y. For example, noise interference can be added to the original data X, and the original data X plus the noise vector N can be obtained to obtain the observed data Y.

[0029] Step 102, compressing the smoothed data based on a preset nested lattice system to obtain encoding information; In the embodiment of the present application, a preset nested lattice system is introduced to compress the smooth data, which improves the efficiency of high-dimensional data compression and also improves the accuracy of high-dimensional data compression. By compressing the smooth data, encoding information can be obtained.

[0030] Step 103, inputting the coded information into a preset discriminant network and a preset generative network respectively; The discriminant network is a machine learning model that is mainly used to determine which category or category probability the input data belongs to. The generative network is a neural network used to generate new data samples.

[0031] The encoded information is respectively input into the preset discriminant network and the preset generative network for subsequent processing of the data.

[0032] Step 104, a preset discriminant network performs cluster discrimination based on the coded information to generate a clustering result; The preset discriminant network performs clustering discrimination based on the encoded information to generate a clustering result. In one example, when the input data is X, the clustering result may be the category of X; when the input data is Y, the clustering result may be the category of Y; when the input data is Z and W, the clustering result may be whether Z and W belong to the same category.

[0033] Step 105: The preset generation network generates new data samples based on the clustering results and the encoding information.

[0034] According to the clustering results and encoding information, the preset generative network can learn and generate new data samples that meet specific clustering characteristics. The new data samples are related to the input data. If the input data is a picture, then the new data samples are also pictures; if the input data is a formula, then the new data samples are also formulas; if the input data is a question and answer, then the new data samples are also questions and answers.

[0035] In the preset generation network, a reward mechanism and a penalty mechanism are also set. If the new data sample and the input data do not belong to the same category, negative feedback can be generated and fed back to the preset generation network and the preset discrimination network; if the new data sample and the input data belong to the same category, positive feedback can be generated and fed back to the preset generation network and the preset discrimination network. By setting up a reward mechanism and a penalty mechanism, the neural networks can be iterated cyclically to continuously optimize and improve the clustering effect and the quality of the generated data.

[0036] The embodiments of the present application can be applied in the process of machine learning training to generate new samples to enhance the training data, thereby improving the generalization ability of the model; it can also be applied in the process of data encryption to compress and encrypt the data while retaining the key information of the data and protecting the privacy of the original data; it can also perform anomaly detection on the data to improve the accuracy and security of the data.

[0037] The embodiment of the present application discloses a simulation service platform data clustering and generation method, including obtaining input data, adding the input data to a preset smoothing vector to generate smoothed data, compressing the smoothed data based on a preset nested lattice system to obtain coding information, inputting the coding information into a preset discrimination network and a preset generation network respectively, the preset discrimination network performs clustering discrimination on the coding information to generate a clustering result, the preset generation network generates new data samples based on the clustering result and the coding information, and can process data in parallel based on the nested lattice and the neural network, optimize the clustering discrimination process of the neural network through the nested lattice coding, and use the learning ability of the neural network to generate data, thereby improving the efficiency and accuracy of simulation data processing, and combining with random smoothing technology to improve the resistance to noise and outliers.

[0038] Figure 2 It is another flow chart of the simulation service platform data clustering and generation method shown in the embodiment of the present application.

[0039] See also Figure 2 , the method comprising: Step 201, obtaining input data, adding the input data to a preset smoothing vector, and generating smoothed data; The input data of this application can be training samples for model training, raw data that needs to be encrypted, or data that needs to be detected for anomalies. The input data can be obtained through sensor measurement, database query, or user input.

[0040] For the input data obtained, the input data can be preprocessed first, and the input data is added to the preset smoothing vector to generate smoothed data, aiming to reduce errors caused by noise or abnormal values ​​and improve the adaptability to the input data. In an example, the input data can be original data X or observed data Y. For example, noise interference can be added to the original data X, and the original data X plus the noise vector N can be obtained to obtain the observed data Y.

[0041] In an optional embodiment of the present application, after step 201, the following steps are further included: Step S11, adding or subtracting a noise vector based on the smoothed data.

[0042] If the input data is original data X, and if the original data has noise interference, after generating smoothed data, a noise vector N may be added to the smoothed data.

[0043] If the input data is observation data Y, and if the observation data is interfered by noise, after generating the smoothed data, the noise vector N may be subtracted from the smoothed data.

[0044] Step 202, compressing the smoothed data based on a preset nested lattice system to obtain encoding information; In the embodiment of the present application, a preset nested lattice system is introduced to compress data, which improves the efficiency of high-dimensional data compression and also improves the accuracy of high-dimensional data compression. By compressing smooth data, encoding information can be obtained.

[0045] In an optional embodiment of the present application, the preset nested lattice system includes a fine lattice and a coarse lattice, and step 202 includes: Sub-step 2021, adjusting the smoothed data based on the preset estimation coefficient to generate adjusted data; The preset nested lattice system includes a fine lattice Lambda_F and a coarse lattice Lambda_C, and the coarse lattice Lambda_C is a subset of the fine lattice Lambda_F.

[0046] The preset estimation coefficient α is multiplied by the smoothed data to perform scaling adjustment on the smoothed data, thereby generating adjusted data.

[0047] Sub-step 2022, quantizing the adjusted data to a fine lattice to generate quantization information; The quantization function may be used to perform a quantization operation on the adjusted data to obtain quantization information. When the input data is the original data X, the quantization information is the quantized value quantized_value; when the input data is the original data Y, the quantization information is the quantized vector quantized_vector.

[0048] Sub-step 2023, mapping the quantized information to the coarse lattice to obtain the encoded information.

[0049] The quantized information is mapped to the coarse lattice Lambda_C by performing a modulo operation of the coarse lattice Lambda_C, thereby obtaining the encoded information. The quantized value is further compressed into the coarse lattice by overmodulo operation for easy storage and transmission.

[0050] In an optional embodiment of the present application, after step 202, the following steps are further included: Step S21, decompressing the coded information according to the preset estimation coefficient and the preset smoothing vector to generate reconstructed data; The coded information can be decompressed according to the preset estimation coefficient α and the preset smoothing vector U to generate reconstructed data.

[0051] Step S22, calculating a distortion metric based on the preset target data and the reconstructed data; the distortion metric is used to evaluate the reconstruction quality of the reconstructed data and is also used to adjust the preset estimation coefficient.

[0052] When the input data is X, the target data may be Y; when the input data is Y, the target data may be X. The preset target data is known or obtained in some way, such as real data, training data or other reference data. The Euclidean distance between the preset target data and the reconstructed data is calculated by the Euclidean distance function, and the Euclidean distance between the preset target data and the reconstructed data is used as the distortion measure of the preset target data and the reconstructed data. The distortion measure can evaluate the reconstruction quality of the reconstructed data. If the distortion measure is within the preset range, it can be considered that the reconstruction quality of the reconstructed data is high. If the distortion measure is not within the preset range, the quality of the reconstructed data is not high, and it can be considered that the decompression performance does not meet the requirements. The decompression performance is adjusted by adjusting the preset estimation coefficient.

[0053] Step 203, determining the compression distance from the input data to the target data according to the encoding information; If the distortion metric is within a preset range, the length of the encoded information or some complexity metric is used as the compression distance from the input data to the target data.

[0054] Step 204, inputting the compression distance into a preset discriminant network and a preset generative network respectively; The compressed distance is input into the preset discriminant network and the preset generative network at the same time, and the preset discriminant network and the preset generative network can process the compressed distance in parallel to improve the efficiency of data processing.

[0055] Step 205, the preset discriminant network performs cluster discrimination based on the compression distance to generate a clustering result; The preset discriminant network uses distance information as input features for cluster discrimination, generates clustering results, and outputs data such as cluster labels, cluster centers, or cluster quality indicators.

[0056] Step 206: The preset generation network generates new data samples based on the clustering results and the compression distance.

[0057] The preset generation network generates data based on the clustering results and the compression distance, generating new data samples similar to the cluster center. The compression distance can be used as an indicator to evaluate the clustering effect and the efficiency of data compression. A lower compression distance usually means better clustering effect and higher data compression efficiency.

[0058] By processing data in parallel with nested lattice coding and neural networks, the compression distance metric based on nested lattice coding is closely combined with the clustering discrimination and generation process based on neural networks, thereby improving the efficiency and accuracy of data processing.

[0059] In an optional embodiment of the present application, the method further includes: Step S31, adjusting the preset generation network according to the clustering result; or, Adjust the preset discriminant network according to the new data samples; or, Adjust the preset smoothing vector or preset estimation coefficients based on the new data samples.

[0060] According to the clustering results generated by the preset discriminant network, the parameters of the preset generation network can be adjusted. According to the generation quality of the new data sample, the clustering algorithm of the preset discriminant network can be adjusted. The preset smoothing vector or preset estimation coefficient can also be adjusted according to the new data sample. The clustering effect and data generation quality can be improved by continuously optimizing and adjusting the neural network. By continuously adjusting the preset smoothing vector or preset estimation coefficient, the best data processing effect can be achieved. In addition, the step size of the fine lattice and the coarse lattice can be adjusted to meet the needs of data processing in different fields and scenarios, with high flexibility and scalability.

[0061] In order to better understand the technical solution of the embodiment of the present application, the following is a detailed description of the solution of the embodiment of the present application by taking the input data as the original data X and the observation data Y as examples. Figure 3 , is a flow chart of a simulation service platform data clustering and generation method shown in an embodiment of the present application. When the input data is original data X, the method includes: Step 301, obtaining input data X, adding the input data X to a preset smoothing vector U, and generating smoothing data X_smooth; Get the original data X, which can be a vector or matrix of any dimension. Use a random number generator to generate a preset smoothing vector U of the same dimension as X. The element distribution of the preset smoothing vector U can be a Gaussian distribution or other suitable distribution. Add the preset smoothing vector U to the original data X to obtain the smoothed data X_smooth = X + U.

[0062] In an optional embodiment of the present application, after step 301, the following steps are further included: Step S31, adding a noise vector N to the smoothed data X_smooth.

[0063] If the original data has noise interference, after generating the smoothed data, a noise vector N can be added to the smoothed data to obtain the smoothed data X1= X + U + N containing noise.

[0064] Step 302: Adjust the smoothed data X_smooth based on the preset estimation coefficient α to generate the adjusted data X t .

[0065] If the original data does not have noise interference, the preset estimation coefficient α is multiplied by the smoothed data to scale the smoothed data X_smooth to generate the adjusted data X t =α×(X + U). If the original data is noisy, the smoothed data containing noise is multiplied by the preset estimation coefficient α to generate the adjusted data X t =α×(X + U + N).

[0066] Step 303: adjust the data X t Quantize to the fine lattice Lambda_F and generate the quantized value quantized_value.

[0067] You can use the quantization function mod(round(...), Lambda_F) to perform modulus operations and rounding operations to get the quantized value quantized_value, that is, quantized_value = mod(round(X t ), Lambda_F), where X t The corresponding formula can be substituted according to whether there is noise interference in the original data.

[0068] Step 304 , mapping the quantized value quantized_value to the coarse lattice Lambda_C to obtain the coding information S_Y.

[0069] The quantized value quantized_value is subjected to a modulo operation of the coarse lattice Lambda_C to map the quantized value quantized_value to the coarse lattice Lambda_C, thereby obtaining the encoding information S_Y, that is, S_Y = mod(quantized_value, Lambda_C).

[0070] Step 305, decompressing the coded information S_Y according to the preset estimation coefficient α and the preset smoothing vector U to generate reconstructed data Y_hat; According to the preset estimation coefficient α and the preset smoothing vector U, the encoded information S_Y can be decompressed to generate reconstructed data Y_hat, that is, Y_hat=X + α×mod(S_Y - U - α×X, Lambda_C).

[0071] Step 306: Calculate the distortion metric D based on the preset target data Y and the reconstructed data Y_hat.

[0072] The preset target data Y is known or obtained in some way. The distortion metric D is calculated based on the preset target data Y and the reconstructed data Y_hat using the Euclidean distance function, that is, D=norm(Y_hat - Y). The distortion metric can evaluate the reconstruction quality of the reconstructed data. If the distortion metric is within the preset range, it can be considered that the reconstruction quality of the reconstructed data is high. If the distortion metric is not within the preset range, the quality of the reconstructed data is not high, and it can be considered that the decompression performance does not meet the requirements. The decompression performance can be adjusted by adjusting the preset estimation coefficient.

[0073] Step 307: Determine the compression distance from the input data X to the target data Y according to the encoding information S_Y.

[0074] The length of the encoded information or some complexity measure is taken as the compression distance from the input data to the target data. In one example, distance=length(S_Y).

[0075] Step 308, inputting the compression distance into the preset discriminant network and the preset generative network respectively; The compressed distance is input into the preset discriminant network and the preset generative network at the same time, and the preset discriminant network and the preset generative network can process the compressed distance in parallel to improve the efficiency of data processing.

[0076] Step 309, the preset discriminant network performs cluster discrimination based on the compression distance to generate a clustering result; The preset discriminant network uses distance information as input features for cluster discrimination, generates clustering results, and outputs data such as cluster labels, cluster centers, or cluster quality indicators.

[0077] Step 3010: The preset generation network generates new data samples based on the clustering results and the compression distance.

[0078] The preset generation network generates data based on the clustering results and the compression distance, generating new data samples similar to the cluster center. At the same time, the compression distance can be used as an indicator to evaluate the clustering effect and the efficiency of data compression. A lower compression distance usually means better clustering effect and higher data compression efficiency.

[0079] See also Figure 4 , is another flow chart of the simulation service platform data clustering and generation method shown in the embodiment of the present application. When the input data is observation data Y, the method includes: Step 401, obtaining input data Y, adding the input data Y to a preset smoothing vector U, and generating smoothing data Y_smooth; Obtain observation data Y, which can be obtained through sensors, databases, or user input. Use a random number generator to generate a preset smoothing vector U with the same dimension as Y. The element distribution of the preset smoothing vector U can be a Gaussian distribution or other suitable distribution. Add the preset smoothing vector U to the observation data Y to obtain the smoothed data Y_smooth =Y + U.

[0080] Step 402: subtract the noise vector N from the smoothed data Y_smooth.

[0081] After generating the smoothed data, the noise vector N may be subtracted from the smoothed data to obtain the noise-removed smoothed data Y1 = Y + U – N, thereby reducing the impact of noise on subsequent processing.

[0082] Step 403: Adjust the smoothed data after removing noise based on the preset estimation coefficient α to generate the adjusted data Y t .

[0083] The preset estimation coefficient α is multiplied by the smoothed data to scale the smoothed data Y1 after the noise is removed, thereby generating the adjusted data Y t =α×(Y + U – N).

[0084] Step 404, adjust the data Y t Quantize to the fine lattice Lambda_F and generate a quantized vector quantized_vector.

[0085] Will adjust the data Y tDivide by the fine lattice quantization step size Lambda_F, then round off, and finally multiply the result by Lambda_F to get the quantized vector quantized_vector. The purpose of this step is to map continuous data values ​​to discrete lattice points.

[0086] Step 405 , mapping the quantized vector quantized_vector to the coarse lattice Lambda_C to obtain the coding information S_X.

[0087] The quantized vector quantized_vector is subjected to a modulo operation of the coarse lattice Lambda_C to map the quantized vector quantized_vector to the coarse lattice Lambda_C, thereby obtaining the encoding information S_X, that is, S_X = mod(quantized_vector, Lambda_C). The quantized vector is further compressed into the coarse lattice through the modulo operation for easy storage and transmission.

[0088] Step 406, decompressing the coded information S_X according to the preset estimation coefficient α and the preset smoothing vector U to generate reconstructed data X_hat; According to the preset estimation coefficient α and the preset smoothing vector U, the encoded information S_X can be decompressed to generate reconstructed data X_hat, that is, X_hat=Y + α×mod(S_X - U - α×Y, Lambda_C).

[0089] Step 407: Calculate the distortion metric D based on the preset target data X and the reconstructed data X_hat.

[0090] The preset target data X is known or obtained in some way. The distortion metric D is calculated based on the preset target data X and the reconstructed data X_hat using the Euclidean distance function, that is, D=norm(X_hat - X). The distortion metric can evaluate the reconstruction quality of the reconstructed data. If the distortion metric is within the preset range, it can be considered that the reconstruction quality of the reconstructed data is high. If the distortion metric is not within the preset range, the quality of the reconstructed data is not high, and it can be considered that the decompression performance does not meet the requirements. The decompression performance can be adjusted by adjusting the preset estimation coefficient.

[0091] Step 408: Determine the compression distance from the input data Y to the target data X according to the encoding information S_X.

[0092] The length of the encoded information or some complexity measure is taken as the compression distance from the input data to the target data. In one example, distance=length(S_X).

[0093] Step 409, inputting the compression distance into the preset discriminant network and the preset generative network respectively; The compressed distance is input into the preset discriminant network and the preset generative network at the same time, and the preset discriminant network and the preset generative network can process the compressed distance in parallel to improve the efficiency of data processing.

[0094] Step 4010, the preset discriminant network performs cluster discrimination based on the compression distance to generate a clustering result; The preset discriminant network uses distance information as input features for cluster discrimination, generates clustering results, and outputs data such as cluster labels, cluster centers, or cluster quality indicators.

[0095] Step 4011: The preset generation network generates new data samples based on the clustering results and the compression distance.

[0096] The preset generation network generates data based on the clustering results and the compression distance, generating new data samples similar to the cluster center. The compression distance can be used as an indicator to evaluate the clustering effect and the efficiency of data compression. A lower compression distance usually means better clustering effect and higher data compression efficiency.

[0097] The embodiment of the present application discloses a simulation service platform data clustering and generation method, including obtaining input data, adding the input data to a preset smoothing vector to generate smoothing data, compressing the smoothing data based on a preset nested lattice system to obtain coding information, determining the compression distance from the input data to the target data according to the coding information, inputting the compression distance into a preset discrimination network and a preset generation network respectively, the preset discrimination network performs clustering discrimination based on the compression distance to generate a clustering result, the preset generation network generates a new data sample based on the clustering result and the compression distance, processing data in parallel based on a nested lattice and a neural network, optimizing the clustering discrimination process of the neural network through nested lattice coding, and generating data by using the learning ability of the neural network, thereby improving the efficiency and accuracy of simulation data processing, using the nested lattice coding technology to calculate the compression distance between data, and using it as an input feature for neural network discrimination and generation, thereby realizing a close combination of clustering and generation processes, and improving the pertinence and effectiveness of data generation, introducing random smoothing technology in the preprocessing stage, adding random noise to the input data, reducing errors caused by noise or outliers, and improving the adaptability and robustness of the system to the input data.

[0098] Corresponding to the aforementioned application function implementation method embodiment, the present application also provides a simulation service platform data clustering and generation system, electronic device and corresponding embodiments.

[0099] Figure 5 It is a structural diagram of a data clustering and generation system 500 shown in an embodiment of the present application.

[0100] See also Figure 5 The system comprises: A preprocessing module 501 is used to obtain input data, add the input data to a preset smoothing vector, and generate smoothed data; A compression module 502, used to compress the smoothed data based on a preset nested lattice system to obtain encoding information; An input module 503 is used to input the encoding information into a preset discriminant network and a preset generator network respectively; A clustering module 504 is used to perform clustering discrimination on the coded information using a preset discrimination network to generate a clustering result; The generation module 505 is used to preset a generation network to generate new data samples based on the clustering results and the encoding information.

[0101] In an optional embodiment of the present application, the preset nested lattice system includes a fine lattice and a coarse lattice, and the compression module 502 includes: An adjustment submodule, used for adjusting the smoothed data based on a preset estimation coefficient to generate adjusted data; A quantization submodule, used to quantize the adjustment data to a fine lattice and generate quantization information; The mapping submodule is used to map the quantized information to the coarse lattice to obtain the encoded information.

[0102] In an optional embodiment of the present application, the system further includes: A decompression module, used to decompress the coded information according to preset estimation coefficients and preset smoothing vectors to generate reconstructed data; The distortion module is used to calculate the distortion metric based on the preset target data and the reconstructed data; the distortion metric is used to evaluate the reconstruction quality of the reconstructed data.

[0103] In an optional embodiment of the present application, the input module 503 includes: A compression distance submodule, used to determine the compression distance from the input data to the preset target data according to the encoding information; The input submodule is used to input the compression distance into the preset discriminant network and the preset generative network respectively.

[0104] In an optional embodiment of the present application, the clustering module 504 includes: The clustering submodule is used to preset the discriminant network to perform clustering discrimination based on the compression distance and generate clustering results; The generation module 505 includes: The generation submodule is used to preset the generation network to generate new data samples based on the clustering results and the compression distance.

[0105] In an optional embodiment of the present application, the system further comprises: The noise module is used to add or subtract a noise vector based on the smoothed data.

[0106] In an optional embodiment of the present application, the system further includes: An adjustment module, used to adjust the preset generation network according to the clustering result; or, Adjust the preset discriminant network according to the new data samples; or, Adjust the preset smoothing vector or preset estimation coefficients based on the new data samples.

[0107] The embodiment of the present application discloses a simulation service platform data clustering and generation system, including obtaining input data, adding the input data to a preset smoothing vector to generate smoothed data, compressing the smoothed data based on a preset nested lattice system to obtain coding information, inputting the coding information into a preset discrimination network and a preset generation network respectively, the preset discrimination network performs clustering discrimination on the coding information to generate a clustering result, the preset generation network generates new data samples based on the clustering result and the coding information, and can process data in parallel based on the nested lattice and the neural network, optimize the clustering discrimination process of the neural network through the nested lattice coding, and use the learning ability of the neural network to generate data, thereby improving the efficiency and accuracy of simulation data processing, and combining with random smoothing technology to improve the resistance to noise and outliers.

[0108] Regarding the system in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated again here.

[0109] Figure 6 It is a schematic diagram of the structure of an electronic device shown in an embodiment of the present application.

[0110] See also Figure 6 , the electronic device 600 includes a memory 610 and a processor 620 .

[0111] The processor 620 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0112] The memory 610 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. Among them, ROM can store static data or instructions required by the processor 620 or other modules of the computer. The permanent storage device may be a readable and writable storage device. The permanent storage device may be a non-volatile storage device that does not lose the stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device uses a large-capacity storage device (such as a magnetic or optical disk, flash memory) as a permanent storage device. In some other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as a dynamic random access memory. The system memory may store some or all instructions and data required by the processor at runtime. In addition, the memory 610 may include any combination of computer-readable storage media, including various types of semiconductor storage chips (such as DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, the memory 610 may include a readable and / or writable removable storage device, such as a laser disc (CD), a read-only digital versatile disc (such as a DVD-ROM, a double-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (such as an SD card, a mini SD card, a Micro-SD card, etc.), a magnetic floppy disk, etc. Computer-readable storage media do not include carrier waves and transient electronic signals transmitted wirelessly or wired.

[0113] The memory 610 stores executable codes, and when the executable codes are processed by the processor 620 , the processor 620 can execute part or all of the above-mentioned methods.

[0114] In addition, the method according to the present application may also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing some or all of the steps in the above method of the present application.

[0115] Alternatively, the present application can also be implemented as a computer-readable storage medium (or non-transitory machine-readable storage medium or machine-readable storage medium) on which executable code (or computer program or computer instruction code) is stored. When the executable code (or computer program or computer instruction code) is executed by a processor of an electronic device (or server, etc.), the processor executes part or all of the steps of the above-mentioned method according to the present application.

[0116] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A simulation service platform data clustering and generation method, characterized in that: The method comprises: Acquire input data, and add the input data to a preset smoothing vector to generate smoothed data; Compressing the smoothed data based on a preset nested lattice system to obtain encoding information; Inputting the encoded information into a preset discriminant network and a preset generative network respectively; The preset discriminant network performs clustering discrimination based on the coded information to generate a clustering result; The preset generation network generates new data samples based on the clustering result and the encoding information.

2. The method according to claim 1, characterized in that The preset nested lattice system includes a fine lattice and a coarse lattice, and the smoothed data is compressed based on the preset nested lattice system to obtain the coded information including: Adjust the smoothed data based on a preset estimation coefficient to generate adjusted data; quantizing the adjustment data to the fine lattice to generate quantization information; The quantized information is mapped to the coarse lattice to obtain the encoded information.

3. The method according to claim 2, characterized in that The step of compressing the smooth data based on the preset nested lattice system to obtain the coded information comprises: Decompressing the coded information according to the preset estimation coefficient and the preset smoothing vector to generate reconstructed data; A distortion metric is calculated based on preset target data and the reconstructed data; the distortion metric is used to evaluate the reconstruction quality of the reconstructed data.

4. The method according to claim 3, characterized in that The step of inputting the coded information into a preset discrimination network and a preset generation network respectively comprises: Determining a compression distance from the input data to preset target data according to the encoding information; The compression distance is input into the preset discrimination network and the preset generation network respectively.

5. The method according to claim 4, characterized in that The preset discriminant network performs clustering discrimination based on the coded information, and generates a clustering result including: The preset discriminant network performs clustering discrimination based on the compression distance to generate the clustering result; The preset generation network generates new data samples based on the clustering result and the encoding information, including: The preset generation network generates the new data sample based on the clustering result and the compression distance.

6. The method according to claim 1, characterized in that After adding the input data to the preset smoothing vector to generate smoothed data, the method further includes: A noise vector is added or subtracted based on the smoothed data.

7. The method according to claim 2, characterized in that The method further comprises: adjusting the preset generation network according to the clustering result; or, Adjust the preset discriminant network according to the new data sample; or, The preset smoothing vector or the preset estimation coefficient is adjusted according to the new data sample.

8. A simulation service platform data clustering and generation system, characterized in that: The system comprises: A preprocessing module, used for acquiring input data, adding the input data to a preset smoothing vector, and generating smoothed data; A compression module, used for compressing the smooth data based on a preset nested lattice system to obtain encoding information; An input module, used to input the encoding information into a preset discrimination network and a preset generation network respectively; A clustering module, used for the preset discriminant network to perform clustering discrimination on the coded information and generate a clustering result; A generation module is used for the preset generation network to generate new data samples based on the clustering result and the encoding information.

9. An electronic device, characterized in that: include: processor; as well as A memory having executable codes stored thereon, which, when executed by the processor, causes the processor to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: An executable code is stored thereon, and when the executable code is executed by a processor of an electronic device, the processor is caused to execute the method as claimed in any one of claims 1 to 7.

Images