Control method of data simulation system, terminal device and readable storage medium
By sorting and dividing the real data and generating simulation data based on the Gaussian distribution function, the problem of reducing the similarity between simulation data and real data in the prior art is solved, and the generation of simulation data with high similarity is achieved.
Patent Information
- Application Number
- CN202411118344.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-15
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-08-15
AI Technical Summary
In the prior art, adding noise to real data or clustering algorithms results in the reduced similarity between the generated simulation data and the real data.
By sorting the received real data and dividing it into equidistant data intervals, the expectations and standard deviations within each data interval are determined, and simulation data is generated based on the Gaussian distribution function to ensure that the real data in the data interval has high similarity and avoiding Outliers affect the accuracy of the Gaussian distribution function.
The similarity between the generated simulation data and the real data is improved, so that the data distribution of the Gaussian distribution function is consistent with the real data distribution, and the accuracy of the simulation data is enhanced.
Smart Images

Figure CN118643682B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing technology, and in particular to a control method, terminal equipment and readable storage medium of a data simulation system. Background Art
[0002] Simulation data refers to a set of data generated by simulation or generation methods, which has similar statistical characteristics and distribution to real data in the real world. Simulation data is widely used in various fields, including scientific research, engineering design, risk assessment, decision analysis, etc. By using simulation data, it can help solve problems such as data acquisition difficulties, privacy protection, algorithm testing and evaluation, and provide a controllable data environment for verifying hypotheses, predicting results, and conducting other forms of data analysis.
[0003] In related simulation technologies, real data is input into a generative model with differential privacy function. The generative model adds noise to the input real data or clustering algorithm, and then clusters the real data to obtain multiple clusters, and then generates corresponding simulation data based on the real data in each cluster. However, adding noise to the input real data or clustering algorithm will increase the difference between the simulation results and the real data, which will reduce the similarity between the generated simulation data and the real data.
[0004] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention
[0005] The present invention aims to solve the technical problem of reduced similarity between generated simulation data and real data by providing a control method for a data simulation system, a terminal device and a computer-readable storage medium.
[0006] To achieve the above object, an embodiment of the present invention provides a control method for a data simulation system, and the control method for the data simulation system includes the following steps:
[0007] After receiving the real data to be simulated, the receiving module sorts the real data and outputs the sorting result of the real data to the partitioning module;
[0008] After the partitioning module receives the sorting result, the sorting result is divided into equidistant data intervals, and the expectation and standard deviation of the real data in each data interval are determined, and the expectation and standard deviation of the real data in each data interval are output to the simulation module;
[0009] After the simulation module receives the expectation and the standard deviation, determining a Gaussian distribution function corresponding to the data interval according to the expectation and the standard deviation;
[0010] Based on the Gaussian distribution function, simulation data of the real data is generated and output.
[0011] Optionally, before the step of generating and outputting the simulation data of the real data based on the Gaussian distribution function, the method further comprises:
[0012] The data in the Gaussian distribution function that are identical to the real data are deleted.
[0013] Optionally, the step of dividing the sorting result into equally spaced data intervals comprises:
[0014] Acquire the total data volume of the real data and the set number of the data intervals;
[0015] Determining the length of the data interval according to the total data volume and the set number;
[0016] The real data is divided into the data intervals of equal distance according to the length of the data intervals.
[0017] Optionally, the step of generating and outputting simulation data of the real data based on the Gaussian distribution function comprises:
[0018] Obtaining a boundary value of the data interval;
[0019] Determining a collection interval of the Gaussian distribution function according to the boundary value;
[0020] The simulation data corresponding to the data interval is selected from the acquisition interval of the Gaussian distribution function.
[0021] Optionally, the step of determining a collection interval of the Gaussian distribution function according to the boundary value includes:
[0022] Obtain an expandable range value corresponding to the data interval;
[0023] Performing a target operation on the boundary value and the expandable range value to determine the boundary value of the acquisition interval;
[0024] Determine the acquisition interval of the Gaussian distribution function according to the boundary value;
[0025] The simulation data corresponding to the data interval is selected from the acquisition interval of the Gaussian distribution function.
[0026] Optionally, the step of selecting the simulation data corresponding to the data interval from the acquisition interval of the Gaussian distribution function includes:
[0027] Dividing the acquisition interval into a plurality of sub-acquisition intervals according to a set probability interval value;
[0028] Obtaining a probability value corresponding to a boundary value of the sub-collection interval, and obtaining a data volume of associated simulation data to be collected according to a probability interval formed by the probability value;
[0029] The simulation data corresponding to the data amount is selected from each of the sub-acquisition intervals of the Gaussian distribution function.
[0030] Optionally, the step of dividing the sorting result into equally spaced data intervals further includes:
[0031] Acquire a data volume of the real data and a pre-generated data volume of the simulation data, wherein the pre-generated data volume is smaller than the data volume of the real data;
[0032] Determining the length of the data interval according to the amount of the real data and the pre-generated amount of the simulation data;
[0033] The real data is divided into the data intervals of equal distance according to the length of the data intervals.
[0034] Optionally, the step of dividing the sorting result into equally spaced data intervals further includes:
[0035] When there is remaining real data, and the amount of the remaining real data is less than a preset amount, the remaining real data is sequentially added to the previous data interval;
[0036] When the remaining real data is greater than a preset amount, the remaining real data is saved as the generation of the next round of simulation data.
[0037] In addition, to achieve the above-mentioned purpose, the present invention also provides a terminal device, which includes: a memory, a processor, and a control program of a data simulation system stored in the memory and executable on the processor, and the control program of the data simulation system, when executed by the processor, implements the steps of the control method of the data simulation system as described above.
[0038] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, on which a control program of a data simulation system is stored. When the control program of the data simulation system is executed by a processor, the steps of the control method of the data simulation system as described above are implemented.
[0039] A control method for a data simulation system, a terminal device and a computer-readable storage medium proposed in the first embodiment of the present invention, after receiving the real data to be simulated, the receiving module sorts the real data, outputs the sorting result of the real data to the partitioning module, after the partitioning module receives the sorting result, divides the sorting result into equidistant data intervals, and determines the expectation and standard deviation of the real data in each data interval, outputs the expectation and standard deviation of the real data in each data interval to the simulation module, after the simulation module receives the expectation and standard deviation, determines the Gaussian distribution function corresponding to the data interval according to the expectation and standard deviation, and generates and outputs the simulation data of the real data based on the Gaussian distribution function. The present invention sorts the real data and divides it into equidistant data intervals, so that the real data in each data interval has high similarity, avoids the existence of abnormal values in the same data interval, which affects the accuracy of the Gaussian distribution function, and then makes the data distribution of the Gaussian distribution function consistent with the distribution of the real data, so that the simulation data generated based on the Gaussian distribution function of each data interval has high similarity with the real data. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] Figure 1 A schematic flow chart of a first embodiment of a control method for a data simulation system of the present invention;
[0041] Figure 2 A schematic flow chart of a second embodiment of a control method for a data simulation system of the present invention;
[0042] Figure 3 A schematic flow chart of a third embodiment of a control method for a data simulation system according to the present invention;
[0043] Figure 4 It is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiment of the present invention.
[0044] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0045] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.
[0046] In the relevant simulation technology, real data is input into a generative model with differential privacy function, and the generative model adds noise to the input real data or clustering algorithm, and then clusters the real data to obtain multiple clusters, and then generates corresponding simulation data according to the real data in each cluster. However, adding noise to the input real data or clustering algorithm will increase the difference between the simulation results and the real data, which will lead to the technical problem of reducing the similarity between the generated simulation data and the real data.
[0047] In order to solve the above defects in the related art, the present invention proposes a control method for a data simulation system, and its main solution steps include the following:
[0048] After receiving the real data to be simulated, the receiving module sorts the real data, outputs the sorting result of the real data to the partitioning module, and after the partitioning module receives the sorting result, divides the sorting result into equidistant data intervals, and determines the expectation and standard deviation of the real data in each data interval, outputs the expectation and standard deviation of the real data in each data interval to the simulation module, and after the simulation module receives the expectation and standard deviation, determines the Gaussian distribution function corresponding to the data interval according to the expectation and standard deviation, and generates and outputs the simulation data of the real data based on the Gaussian distribution function. The present invention sorts the real data and divides it into equidistant data intervals, so that the real data in each data interval has high similarity, avoids the existence of abnormal values in the same data interval, which affects the accuracy of the Gaussian distribution function, and then makes the data distribution of the Gaussian distribution function consistent with the distribution of the real data, so that the similarity between the subsequent simulation data generated based on the Gaussian distribution function of each data interval and the real data is high.
[0049] In order to better understand the above technical solution, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.
[0050] A control method for a data simulation system proposed in the present invention is applied to a data simulation system. The data simulation system can be configured in a terminal device, and the terminal device can be a mobile phone, a computer or other device, which is not specifically limited in the present invention. The data simulation system of the present invention includes a receiving module, a partitioning module and a simulation module. The receiving module is used to receive the real data to be simulated, and after sorting the real data, the sorting result of the real data is output to the partitioning module. The partitioning module is used to divide the sorting result into equidistant data intervals after receiving the sorting result, and determine the expectation and standard deviation of the real data in each data interval, and output the expectation and standard deviation of the real data in each data interval to the simulation module. The simulation module is used to determine the Gaussian distribution function corresponding to each data interval according to the expectation and standard deviation of the real data in each data interval after receiving the expectation and standard deviation of the real data in each data interval, and then generate and output simulation data of the real data according to the Gaussian distribution function corresponding to each data interval.
[0051] Reference Figure 1In a first embodiment of a control method for a data simulation system of the present invention, the control method for a data simulation system comprises the following steps:
[0052] Step S10: after receiving the real data to be simulated, the receiving module sorts the real data and outputs the sorting result of the real data to the partitioning module;
[0053] In this embodiment, the sorting result is the sorted real data. It can be understood that the difference between the sorted real data and the original real data is only the difference in the arrangement order. The arrangement order between the original real data is disordered, and the sorted real data is ordered. The real data can be sorted in ascending order or descending order, which is not specifically limited in this embodiment.
[0054] Optionally, the real data can be any data. For example, an ID card, a bank card, a telephone number, etc. It is understandable that when there are letters and words in the real data, in order to be able to sort and simulate the real data, the numbers mapped by different letters or words can be pre-configured, and then when the receiving module receives the real data, the real data is detected, and the letters or words in the real data are converted into pre-mapped numbers, so as to sort the mapped real data. After the simulation data is generated based on the Gaussian distribution function, the numbers at the corresponding positions are mapped back according to the pre-mapped relationship.
[0055] Optionally, the step of sorting the real data includes: determining the data scale level of the real data according to the data volume of the received real data, judging whether the data annotation of the real data is received, and if the data annotation of the real data is received, determining the corresponding sorting algorithm according to the data annotation, and using the sorting algorithm to sort the real data. If the data annotation of the real data is not received, determining the corresponding sorting algorithm according to the data scale level, and using the sorting algorithm to sort the real data, so as to achieve the effect of improving the sorting rate of the real data.
[0056] In this embodiment, a data set of real data is randomly generated, and different sorting algorithms are used to sort the data sets, and the sorting time is recorded; then the number of data sets is increased at a preset growth rate each time, and different sorting algorithms are continued to be used to sort the data sets, and the sorting time is recorded until the number of data sets reaches the maximum receiving amount of the real data by the receiving module; then the sorting time corresponding to each data set is counted, and the optimal sorting algorithm corresponding to each data set is determined; then the data sets corresponding to the optimal sorting algorithms of adjacent data sets are merged, and the data volume of the merged data set is re-determined; then, according to the law of sequential increase of data volume, each data set is divided into a corresponding data scale level, and the data scale level is associated and bound with the optimal sorting algorithm of the corresponding data set.
[0057] For example, the data set includes data set A, the data volume is 200, and the optimal sorting algorithm is I sorting algorithm; data set B, the data volume is 500, the optimal sorting algorithm is II sorting algorithm; data set C, the data volume is 600, and the optimal sorting algorithm is III sorting algorithm. Data with a volume less than 200 is divided into data scale level 1, data with a volume greater than 200 and less than 500 is divided into data scale level 2, and data with a volume greater than 500 and less than 600 is divided into data scale level 3, and data scale level 1 is associated with I sorting algorithm, data scale level 2 is associated with II sorting algorithm, and data scale level 3 is associated with III sorting algorithm.
[0058] Therefore, the receiving module can determine the data scale level according to the data volume of the received real data, and then when the data annotation of the real data is not received, the real data can be sorted by using the sorting algorithm associated with the data scale level, thereby improving the sorting rate of the real data. When the data annotation of the real data is received, the real data can be sorted by using the sorting algorithm identified by the data annotation. The data annotation represents different sorting algorithms.
[0059] For example, the user's data annotations on real data represent the bubble sort algorithm, so the bubble sort algorithm is used to sort the real data. First, after the receiving module receives the real data and determines that the data annotations of the real data represent the bubble sort algorithm, the bubble sort algorithm is used to sort the real data, and the sorting result is output to the partitioning module. After the partitioning module receives the sorting result, it divides the real data into equidistant data intervals, determines the expectation and standard deviation of the real data in each data interval, and outputs the expectation and standard deviation of the real data in each data interval to the simulation module. After the simulation module receives the expectation and standard deviation of the real data in each data interval, it determines the Gaussian distribution function corresponding to each data interval based on the expectation and standard deviation, thereby generating and outputting simulation data of the real data based on the Gaussian distribution function.
[0060] Step S20: After the partitioning module receives the sorting result, the sorting result is divided into equidistant data intervals, and the expectation and standard deviation of the real data in each data interval are determined, and the expectation and standard deviation of the real data in each data interval are output to the simulation module;
[0061] In this embodiment, after receiving the sorting result, the partitioning module divides the sorted real data into equidistant data intervals so that the data volume of the real data in each data interval is consistent. And calculate the expectation and standard deviation of the real data in each data interval. By dividing the sorted real data into equidistant data intervals, the present invention can ensure that the Gaussian distribution function in each data interval after partitioning is relatively smooth, reduce the probability of the occurrence of abnormal values of the Gaussian distribution function, so that the subsequent simulation data generated based on the Gaussian distribution function has a high similarity with the real data. An abnormal value refers to a value that is far from the expectation.
[0062] It can be understood that the amount of real data in the data interval can be obtained, and then the quotient obtained by dividing the sum of all real data in the data interval by the amount of data is the expectation of the real data in the data interval. The variance of the real data in the data interval can be calculated based on the expectation and the real data of the data interval, and then the standard deviation of the real data in the data interval can be calculated based on the variance.
[0063] The expected expression can be expressed as: ,in is the expectation, Xn is the nth real data in the data interval, and n is the amount of real data in the data interval.
[0064] The expression of standard deviation can be expressed as: ; , where Var is the variance of the real data within the data interval, is the standard deviation of the real data within the data interval.
[0065] Optionally, the total data volume of the real data and the set number of data intervals are obtained, the length of the data interval is determined according to the total data volume and the set number, and the real data is divided into equidistant data intervals according to the length of the data interval.
[0066] In this embodiment, a set number of divided data intervals can be set in advance, and then after the partitioning module receives the real data, the total data volume of the real data can be obtained, and the quotient of the total data volume divided by the set number is determined as the length of the data interval. The length represents the data volume of the real data in the data interval, and then according to the length of the data interval, the real data is divided into a set number of equally spaced data intervals.
[0067] Optionally, the set number of data intervals can be set to 4, and then the partition module can divide the real data into 4 equally spaced data intervals according to the quartile division rule. The quartile can be expressed as (0.25, 0.5, 0.75). The partition module can obtain the total amount of received real data, and then determine the sequence number of the real data corresponding to the boundary value of each data interval according to the total amount of data and the quartile division rule. When the calculated sequence number is a decimal, it can be approximated to an integer.
[0068] The serial number expression of the real data corresponding to the boundary value of each data interval can be expressed as: Q1=1+(n-1)*0.25; Q2=1+(n-1)*0.5; Q3=1+(n-1)*0.75, where Qi is the serial number of the real data corresponding to the boundary value of each data interval, and n is the total amount of real data.
[0069] After obtaining the sequence number of each data interval, the real data corresponding to the sequence number can be obtained to determine the boundary value of each data interval. Assuming that the real data corresponding to the sequence number Qi is Xi, then each data interval can be expressed as [minimum value, X1), [X1, X2), [X2, X3), [X3, maximum value].
[0070] Step S30: after the simulation module receives the expectation and the standard deviation, determining the Gaussian distribution function corresponding to the data interval according to the expectation and the standard deviation;
[0071] In this embodiment, after receiving the expectation and standard deviation of the real data in the data interval, the simulation module can substitute them into the expression of the Gaussian distribution function, so as to calculate and obtain the Gaussian distribution function corresponding to the data interval.
[0072] Gaussian distribution is also called normal distribution. If the random variable X obeys a mathematical expectation , the variance is The normal distribution of ). The probability density function of the Gaussian distribution is a normal distribution, and the expectation is The value determines its location, and its standard deviation Determines the magnitude of the distribution. , standard deviation The normal distribution when is the standard normal distribution.
[0073] The expression of Gaussian distribution function can be expressed as:
[0074]
[0075] It should be noted that since the present invention divides the sorted real data into equally spaced data intervals and then determines the expectation and standard deviation of the real data within the data interval, the data distribution of the Gaussian distribution function determined based on the expectation and standard deviation of the real data within the data interval is consistent with the data distribution of the real data, thereby making the generated simulation data consistent with the distribution of the real data.
[0076] Step S40: Generate and output simulation data of the real data based on the Gaussian distribution function.
[0077] In this embodiment, the independent variable of the Gaussian distribution function is the value of the simulation data, and the dependent variable is the probability density of the corresponding value of the simulation data. The simulation data can be randomly sampled based on the Gaussian distribution function, and the simulation data obtained by random sampling is determined as the simulation data of the real data.
[0078] Optionally, in order to protect the privacy of real data and prevent the real data from being leaked, before generating simulation data of the real data, the simulation data that is identical to the real data in the Gaussian distribution function can be deleted to avoid random sampling of simulation data that is identical to the real data, thereby improving the security of the simulation data.
[0079] In the technical solution provided in this embodiment, after receiving the real data to be simulated, the receiving module sorts the real data, outputs the sorting result of the real data to the partitioning module, and after the partitioning module receives the sorting result, divides the sorting result into equidistant data intervals, and determines the expectation and standard deviation of the real data in each data interval, outputs the expectation and standard deviation of the real data in each data interval to the simulation module, and after the simulation module receives the expectation and standard deviation, determines the Gaussian distribution function corresponding to the data interval according to the expectation and standard deviation, and generates and outputs the simulation data of the real data based on the Gaussian distribution function. The present invention sorts the real data and divides it into equidistant data intervals, so that the real data in each data interval has high similarity, avoids the existence of outliers in the same data interval, which affects the accuracy of the Gaussian distribution function, and then makes the data distribution of the Gaussian distribution function consistent with the distribution of the real data, so that the similarity between the subsequent simulation data generated based on the Gaussian distribution function of each data interval and the real data is high.
[0080] Reference Figure 2 In the second embodiment, based on the first embodiment, the step S40 includes:
[0081] Step S41: obtaining the boundary value of the data interval;
[0082] Step S42: determining a collection interval of the Gaussian distribution function according to the boundary value;
[0083] In this embodiment, in order to make the generated simulation data close to the expectation of the data interval and the size of the simulation data within the data interval, and to improve the similarity between the simulation data and the real data, the boundary value of the data interval can be obtained, and the interval formed by the boundary value is determined as the acquisition interval. The boundary value of the data interval includes an upper limit value and a lower limit value.
[0084] For example, if the data interval is [X1, X2), X1 is the lower limit value and X2 is the upper limit value, then the sampling interval is (X1, X2).
[0085] Step S43: selecting the simulation data corresponding to the data interval from the acquisition interval of the Gaussian distribution function.
[0086] In this embodiment, based on the Gaussian distribution function, simulation data is randomly sampled within the range of a sampling interval, and the simulation data obtained by sampling is determined as simulation data of real data.
[0087] Optionally, when the simulated data identical to the real data in the acquisition interval of the Gaussian distribution function is deleted, the obtainable simulated data in the acquisition interval will be reduced, thereby causing the amount of acquired simulated data to be less than the set pre-generated amount. In another optional implementation, the step of determining the acquisition interval of the Gaussian distribution function based on the boundary value includes: obtaining an expandable range value corresponding to the data interval, performing a target operation on the boundary value and the expandable range value, determining the boundary value of the acquisition interval, determining the acquisition interval of the Gaussian distribution function based on the boundary value, and then selecting the simulated data corresponding to the data interval from the acquisition interval of the Gaussian distribution function.
[0088] In this embodiment, the scalable range value refers to the allowable error precision. The target operation refers to the addition operation with the scalable range value. For example, if the data interval is (X1, X2), X1 and X2 are the boundary values of the data interval, and the scalable range value is + Then, the result of adding X1 and X3 is X4, and the result of adding X2 and X3 is X5, so the acquisition interval is (X4, X5), thereby expanding the range of the acquisition interval without exceeding the allowable error range, ensuring that the randomly collected simulation data and the real data have a high similarity.
[0089] Optionally, the step of selecting the simulation data corresponding to the data interval from the acquisition interval of the Gaussian distribution function includes: dividing the acquisition interval into multiple sub-acquisition intervals according to a set probability interval value, obtaining the probability value corresponding to the boundary value of the sub-acquisition interval, obtaining the data volume of the associated simulation data to be collected according to the probability interval formed by the probability value, and then selecting the simulation data corresponding to the data volume from each sub-acquisition interval of the Gaussian distribution function.
[0090] The 68-95-99.7 rule, also known as the "three standard deviation rule" or "rule of thumb for normal distribution", is an important rule that describes the characteristics of the probability distribution of the Gaussian function. The rule states that in a data set that conforms to a Gaussian distribution, about 68% of the data fall within one standard deviation of the mean: According to the rule, for a data set that conforms to a Gaussian distribution, about 68% of the data points have values that fall within one standard deviation of the mean. About 95% of the data fall within two standard deviations of the mean: According to the rule, about 95% of the data points have values that fall within two standard deviations of the mean. About 99.7% of the data fall within three standard deviations of the mean: According to the rule, about 99.7% of the data points have values that fall within three standard deviations of the mean.
[0091] In this embodiment, the probability interval values are set to 68%, 95% and 99.7%. The probability intervals formed by the probability interval values are (0, 68%), [68%, 95%), [95%, 99.7%), [99.7%, 1). The data volume of the simulation data to be collected corresponding to the probability interval formed by the probability interval values is set, and the data volume is associated with the probability interval formed by the probability interval values and saved. It should be noted that the data volume of the simulation data to be collected associated with the probability intervals (0, 68%), [68%, 95%), [95%, 99.7%), [99.7%, 1) increases successively so that the data distribution of the obtained simulation data is consistent with the data distribution in the data interval.
[0092] In the technical solution provided in this embodiment, by obtaining the boundary value of the data interval, the collection interval of the Gaussian distribution function is determined according to the boundary value, and the simulation data corresponding to the data interval is selected from the collection interval of the Gaussian distribution function, so that the generated simulation data is consistent with the data distribution of the real data in the data interval, and the simulation data is close to the expectation, ensuring that there is a high similarity between the simulation data and the real data.
[0093] Reference Figure 3 In a third embodiment, based on any of the above embodiments, step S20 includes:
[0094] Step S21: acquiring the data volume of the real data and the pre-generated data volume of the simulation data, wherein the pre-generated data volume is smaller than the data volume of the real data;
[0095] In this embodiment, the pre-generated amount of simulation data can be configured by the user according to the needs, and this embodiment does not specifically limit this. It can be understood that the larger the pre-generated amount of simulation data, the more real data is required.
[0096] Step S22: determining the length of the data interval according to the amount of the real data and the pre-generated amount of the simulation data;
[0097] Step S23: Divide the real data into equidistant data intervals according to the length of the data intervals.
[0098] In this embodiment, in order to ensure the uniformity of the generated simulation data, the quotient of the data volume of the real data divided by the pre-generated volume of the simulation data can be used as the number of data intervals to be divided, and then the quotient of the data volume of the real data divided by the number of data intervals can be used as the length of the data interval, so that the real data can be divided into equidistant data intervals according to the length of the data interval. If the quotient is a decimal, it can be approximated to an integer.
[0099] Optionally, the step of dividing the sorting result into equally spaced data intervals further includes:
[0100] When there is remaining real data and the amount of the remaining real data is less than the preset amount, the remaining real data is added to the previous data interval in sequence. When the remaining real data is greater than the preset amount, the remaining real data is saved as the generation of the next round of simulation data.
[0101] In this embodiment, when there is remaining real data, and the remaining real data is less than the preset amount, it indicates that the remaining real data added to the previous data interval will not affect the balance between the various data intervals. In order not to affect the difference between the data amounts of the real data in the various data intervals that have been allocated, the remaining real data can be added to the end of the previous data interval in sequence. When the remaining real data is greater than the preset amount, it indicates that after the remaining real data is added to the previous data interval, the difference between the previous data interval and other data intervals is large, so the remaining real data can be saved and used for the generation of the next round of simulation data.
[0102] In the technical solution provided in this embodiment, by obtaining the data volume of real data and the pre-generated volume of simulation data, the pre-generated volume is smaller than the data volume of real data, and according to the data volume of real data and the pre-generated volume of simulation data, the length of the data interval is determined, and then according to the length of the data interval, the real data is divided into equidistant data intervals. By dividing the real data into multiple data intervals, it can be ensured that the simulation data is uniformly selected from the entire data set of the real data. Each data interval has the same amount of simulation data selected, which avoids the situation where some data intervals are over-selected or ignored, so that the distribution of the generated simulation data is consistent with that of the real data, ensuring that the simulation data has a high similarity with the real data.
[0103] Reference Figure 4 , Figure 4The figure is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiment of the present invention.
[0104] like Figure 4 As shown, the terminal may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), a mouse, etc., and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.
[0105] Those skilled in the art will understand that Figure 4 The terminal structure shown in the figure does not constitute a limitation on the terminal, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.
[0106] like Figure 4 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a control program of a data simulation system.
[0107] exist Figure 4 In the terminal shown, the network interface 1004 is mainly used to connect to the background server and perform data communication with the background server; the processor 1001 can be used to call the control program of the data simulation system stored in the memory 1005 and perform the following operations:
[0108] After receiving the real data to be simulated, the receiving module sorts the real data and outputs the sorting result of the real data to the partitioning module;
[0109] After the partitioning module receives the sorting result, the sorting result is divided into equidistant data intervals, and the expectation and standard deviation of the real data in each data interval are determined, and the expectation and standard deviation of the real data in each data interval are output to the simulation module;
[0110] After the simulation module receives the expectation and the standard deviation, determining a Gaussian distribution function corresponding to the data interval according to the expectation and the standard deviation;
[0111] Based on the Gaussian distribution function, simulation data of the real data is generated and output.
[0112] Further, the processor 1001 may call the control program of the data simulation system stored in the memory 1005, and further perform the following operations:
[0113] The data in the Gaussian distribution function that are identical to the real data are deleted.
[0114] Further, the processor 1001 may call the control program of the data simulation system stored in the memory 1005, and further perform the following operations:
[0115] Acquire the total data volume of the real data and the set number of the data intervals;
[0116] Determining the length of the data interval according to the total data volume and the set number;
[0117] The real data is divided into the data intervals of equal distance according to the length of the data intervals.
[0118] Further, the processor 1001 may call the control program of the data simulation system stored in the memory 1005, and further perform the following operations:
[0119] Obtaining a boundary value of the data interval;
[0120] Determining a collection interval of the Gaussian distribution function according to the boundary value;
[0121] The simulation data corresponding to the data interval is selected from the acquisition interval of the Gaussian distribution function.
[0122] Further, the processor 1001 may call the control program of the data simulation system stored in the memory 1005, and further perform the following operations:
[0123] Obtain an expandable range value corresponding to the data interval;
[0124] Performing a target operation on the boundary value and the expandable range value to determine the boundary value of the acquisition interval;
[0125] Determine the acquisition interval of the Gaussian distribution function according to the boundary value;
[0126] The simulation data corresponding to the data interval is selected from the acquisition interval of the Gaussian distribution function.
[0127] Further, the processor 1001 may call the control program of the data simulation system stored in the memory 1005, and further perform the following operations:
[0128] Dividing the acquisition interval into a plurality of sub-acquisition intervals according to a set probability interval value;
[0129] Obtaining a probability value corresponding to a boundary value of the sub-collection interval, and obtaining a data volume of associated simulation data to be collected according to a probability interval formed by the probability value;
[0130] The simulation data corresponding to the data amount is selected from each of the sub-acquisition intervals of the Gaussian distribution function.
[0131] Further, the processor 1001 may call the control program of the data simulation system stored in the memory 1005, and further perform the following operations:
[0132] Acquire a data volume of the real data and a pre-generated data volume of the simulation data, wherein the pre-generated data volume is smaller than the data volume of the real data;
[0133] Determining the length of the data interval according to the amount of the real data and the pre-generated amount of the simulation data;
[0134] The real data is divided into the data intervals of equal distance according to the length of the data intervals.
[0135] Further, the processor 1001 may call the control program of the data simulation system stored in the memory 1005, and further perform the following operations:
[0136] When there is remaining real data, and the amount of the remaining real data is less than a preset amount, the remaining real data is sequentially added to the previous data interval;
[0137] When the remaining real data is greater than a preset amount, the remaining real data is saved as the generation of the next round of simulation data.
[0138] In addition, to achieve the above-mentioned purpose, the present invention also provides a terminal device, which includes: a memory, a processor, and a control program of a data simulation system stored in the memory and executable on the processor, and the control program of the data simulation system, when executed by the processor, implements the steps of the control method of the data simulation system as described above.
[0139] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, on which a control program of a data simulation system is stored. When the control program of the data simulation system is executed by a processor, the steps of the control method of the data simulation system as described above are implemented.
[0140] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.
[0141] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.
[0142] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer) to execute the methods described in each embodiment of the present invention.
[0143] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.
Claims
1. A control method for a data simulation system, characterized in that: The control method of the data simulation system comprises: By randomly generating a data set of real data, and using different sorting algorithms to sort the data set, and recording the sorting time; Each time, the number of the data sets is increased at a preset growth rate, and the data sets are sorted by using different sorting algorithms, and the sorting duration is recorded until the number of the data sets reaches the maximum receiving amount of the real data by the receiving module; Collecting statistics on the sorting time corresponding to each of the data sets to determine the optimal sorting algorithm corresponding to each of the data sets; Merging the data sets corresponding to the adjacent data sets and having the same optimal sorting algorithm, and re-determining the data volume of the merged data sets; Then, according to the law of sequential increase of the data volume, each of the data sets is divided into corresponding data scale levels, and the data scale level is associated and bound with the optimal sorting algorithm corresponding to the data set; When the receiving module receives the real data to be simulated, the real data is detected and the letters or words in the real data are converted into pre-mapped numbers; Sorting the mapped real data, and outputting the sorting result of the real data to a partitioning module; The step of sorting the mapped real data comprises: Determining a data scale level of the real data according to the data volume of the received real data; Determining whether the data annotation of the real data is received; If data annotations of the real data are received, a corresponding sorting algorithm is determined according to the data annotations, and the real data is sorted using the sorting algorithm; If the data annotation of the real data is not received, determining the corresponding sorting algorithm according to the data scale level, and using the sorting algorithm to sort the real data; After the partitioning module receives the sorting result, the sorting result is divided into equidistant data intervals, and the expectation and standard deviation of the real data in each data interval are determined, and the expectation and standard deviation of the real data in each data interval are output to the simulation module; The step of dividing the sorting result into equidistant data intervals comprises: Acquire the total data volume of the real data and the set number of the data intervals; The quotient of dividing the total data amount by the set number is determined as the length of the data interval; Dividing the real data into equidistant data intervals according to the length of the data intervals; After the simulation module receives the expectation and the standard deviation, determining a Gaussian distribution function corresponding to the data interval according to the expectation and the standard deviation; Deleting the data in the Gaussian distribution function that are identical to the real data; Obtaining a boundary value of the data interval; Obtain an expandable range value corresponding to the data interval; Performing a target operation on the boundary value and the expandable range value to determine the boundary value of the acquisition interval, wherein the expandable range value refers to an allowable error accuracy, and the target operation refers to performing an addition operation on the expandable range value; Determine the acquisition interval of the Gaussian distribution function according to the boundary value; Selecting the simulation data corresponding to the data interval from the acquisition interval of the Gaussian distribution function; The simulation data corresponding to the data interval is selected from the acquisition interval of the Gaussian distribution function, wherein the independent variable of the Gaussian distribution function is the value of the simulation data, and the dependent variable is the probability density of the value corresponding to the simulation data.
2. The method according to claim 1, characterized in that The step of selecting simulation data corresponding to the data interval from the acquisition interval of the Gaussian distribution function comprises: Dividing the acquisition interval into a plurality of sub-acquisition intervals according to a set probability interval value; Obtaining a probability value corresponding to a boundary value of the sub-collection interval, and obtaining a data volume of associated simulation data to be collected according to a probability interval formed by the probability value; The simulation data corresponding to the data amount is selected from each of the sub-acquisition intervals of the Gaussian distribution function.
3. The method according to claim 1, characterized in that The step of dividing the sorting result into equidistant data intervals further includes: Acquire a data volume of the real data and a pre-generated data volume of the simulation data, wherein the pre-generated data volume is smaller than the data volume of the real data; Determining the length of the data interval according to the amount of the real data and the pre-generated amount of the simulation data; The real data is divided into the data intervals of equal distance according to the length of the data intervals.
4. The method according to claim 1, characterized in that The step of dividing the sorting result into equidistant data intervals further includes: When there is remaining real data, and the amount of the remaining real data is less than a preset amount, the remaining real data is sequentially added to the previous data interval; When the remaining real data is greater than a preset amount, the remaining real data is saved as the generation of the next round of simulation data.
5. A terminal device, characterized in that: The terminal device includes: a memory, a processor, and a program for generating simulation data of the terminal device which is stored in the memory and can be run on the processor. When the program for generating simulation data of the terminal device is executed by the processor, the steps of the control method of the data simulation system as described in any one of claims 1 to 4 are implemented.
6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program for generating simulation data of a terminal device, and when the program for generating simulation data of the terminal device is executed by a processor, the steps of the control method of the data simulation system according to any one of claims 1 to 4 are implemented.
