A data optimization method, device and medium based on dimensional constraints

Abstract

The present application provides a data optimization method, device and medium based on dimensional constraints, the method comprising: obtaining original data as the data to be optimized, setting a first preset value list and a second preset value list for the data to be optimized, and constructing a first objective function based on the data to be optimized, the first preset value and the second preset value and setting the target number of dimensions, obtaining a target proportion value under unconstrained conditions, and determining the dimensions that meet the target number of dimensions based on the target proportion value. The present invention can further optimize an existing algorithm, screen and compress the dimensions of the variables to be optimized, and expand the algorithm without changing the core of the original optimization algorithm, which is conducive to improving the applicability of the original optimization algorithm. The data after further optimization is more targeted on the basis of the original data and can meet the minimum threshold of the requirements.

Description

Technical Field

[0001] The present application relates to the field of data processing methods, and specifically to a data optimization method, device and medium based on dimensional constraints. Background Art

[0002] With the advent of the big data era, more and more fields require data optimization. This involves finding a list of values within a given data space that minimizes the value of an objective function. Typically, this optimization involves using optimization algorithms, such as genetic algorithms, particle swarm optimization algorithms, and artificial bee colony algorithms. However, these algorithms require that all dimensions of the variables being optimized are available, meaning there are no dimensional constraints.

[0003] However, many practical data problems often involve constraints. For example, in traffic planning, determining the optimal route for optimal travel time may require limiting the number of intermediate transfers. These constraints restrict the dimensionality of the variables used in data optimization, limiting the application of only a subset of the data within the variable dimensionality. Conventional optimization algorithms cannot be directly applied, limiting their scope of application and resulting in inaccurate optimization results. Summary of the Invention

[0004] In response to the above technical problems, the technical solution adopted in this application includes the following steps:

[0005] S100, obtain the original data list X=(X1, X2, ..., X n ), where X j Refers to the original data of the jth dimension, j = 1...n, n is the number of data dimensions, and according to X j , get the value with X j The corresponding first preset value list H=(H1,H2,……,H n ) and the second preset value list L=(L1, L2, ……, L n ), where L j ≤X j ≤H j ;

[0006] S200, obtain the preset optimization model S and objective function f() and obtain the target proportion value T corresponding to each dimension of X according to the original data list X, the optimization model S and the objective function f() j And based on all T j Get the target proportion list T;

[0007] S300, pre-process X based on S to obtain an intermediate data list X'=(X'1, X'2, ..., X' n ) and when (L j +Hj ) / 2≤X' j ≤H j When , the preset constraint dimension number K and target parameter value C are obtained to compare C with the preset parameter threshold C0;

[0008] S400, if C>C0, perform the first type of data processing on X' and determine whether to execute step S300 or step S600 according to the result of the first type of data processing;

[0009] S500, if C≤C0, perform the second type of data processing on X' and determine whether to execute step S300 or step S600 according to the result of the second type of data processing;

[0010] S600 : According to K, a target data list corresponding to X that is subjected to constraint processing in the K dimension is obtained.

[0011] The present application has at least the following technical effects: obtaining the original data as the data to be optimized, setting a first preset value list and a second preset value list for the data to be optimized, and constructing a first objective function based on the data to be optimized, the first preset value and the second preset value and setting the target number of dimensions, obtaining the target proportion value under unconstrained conditions, and determining the dimension that meets the target number of dimensions based on the target proportion value. The present invention can further optimize a certain existing algorithm and screen and compress the dimensions of the variables to be optimized. In a non-invasive manner, the variables to be optimized are further processed based on the data processing method of the variables to be optimized themselves, so that the expansion is carried out without changing the core of the original optimization algorithm, which is conducive to improving the applicability of the original optimization algorithm. The data after further optimization is more targeted on the basis of the original data and can meet the minimum threshold of the requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0013] Figure 1 A flowchart of a data optimization method based on dimensional constraints provided in an embodiment of the present application. DETAILED DESCRIPTION

[0014] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0015] The embodiment of the present application provides a data optimization method based on dimensional constraints, such as Figure 1 As shown, the method includes the following steps:

[0016] S100, obtain the original data list X=(X1, X2, ..., X n ), where X j Refers to the original data of the jth dimension, j = 1...n, n is the number of data dimensions, and according to X j , get X j The corresponding first preset value list H=(H1,H2,……,H n ) and the second preset value list L=(L1, L2, ……, L n ), where L j ≤X j ≤H j ;

[0017] Specifically, the first preset value is used to represent the upper limit value of the original data in any data dimension and the second preset value is used to represent the lower limit value of the original data in any data dimension. In actual applications, several kinds of raw materials need to be prepared when preparing a certain product, and the quality of the raw materials is used as the original data, and the type of the raw materials is the data dimension, wherein the upper limit value and the lower limit value of the quality of each type of raw material are the first preset value and the second preset value respectively.

[0018] S200, obtain the preset optimization model S and objective function f() and obtain the target proportion value T corresponding to each dimension of X according to the original data list X, the optimization model S and the objective function f() j And based on all T j Get the target proportion list T;

[0019] Specifically, the objective function is a function based on the original data list, the first preset value list and the second preset value list to obtain the target value. It is known to those skilled in the art that the target value can be set according to the requirements of the specific application scenario, that is, based on X, H and L, the target value f(X j , H j , L j), for example, a temperature difference power generation system, wherein the original data is the power generation of the thermoelectric module set in any interval, and the upper limit and lower limit of the power generation of the thermoelectric module are used as the first preset value and the second preset value, and according to the power generation, the upper limit and lower limit of the power generation, the power is obtained as the objective function value, and the function corresponding to the power is obtained as the objective function, and those skilled in the art know that an optimization model can be established according to the objective function, and the optimization model involves optimization algorithms including but not limited to, for example: genetic algorithm, particle swarm algorithm, artificial bee colony algorithm, etc., which will not be repeated here.

[0020] Specifically, step S200 also includes the following steps:

[0021] S201. Obtain a first optimized value V0 corresponding to X based on X, S, and f(). This can be understood as the target value obtained after unconstraining X. This first optimized value V0 represents the ultimate optimized value that can be obtained when all dimensions of X are fully available, and serves as a basis and reference for dimension ranking.

[0022] S203. Delete X from X j , build the second intermediate data list according to S and f(), get The corresponding second optimized value V j ;

[0023] S205, according to V0 and V j , get The corresponding target proportion value T j =V0 / V j ;

[0024] S207: Based on Tj, construct a target proportion value list T=(T1, T2, ..., T n );

[0025] S209, based on the target proportion value T of each dimension j , sort each dimension in descending order and obtain the corresponding ranking P for each dimension j j .

[0026] S300, pre-process X based on S to obtain an intermediate data list X'=(X'1, X'2, ..., X' n ) and when (L j +H j ) / 2≤X' j ≤H j When , the preset constraint dimension number K and target parameter value C are obtained to compare C with the preset parameter threshold C0;

[0027] Specifically, step S300 also includes the following steps:

[0028] When (L j +H j ) / 2≤X' j ≤H j When X' j As the first key data and constructing the first key data list Q=(Q1, Q2, ..., Q m ), where Q r Refers to the rth first key data, r = 1...m;

[0029] When X' j <(L j +H j ) / 2 or X' j >H j When Q is deleted from X' and any intermediate data in X' after Q is deleted is used as the second key data to construct the second key data list in, It refers to the pth second key data, where p=1…nK.

[0030] Specifically, the preprocessing is a single-step update process corresponding to the optimization model. Those skilled in the art can know the specific process of determining the single-step update process of data based on the type of optimization algorithm corresponding to the optimization model, which will not be repeated here.

[0031] Specifically, the value range of the random number C is 0 to 1; preferably, the value of C0 is 0.5. This method of using random numbers can increase the randomness during calculation and avoid entering the local optimization process in advance during data processing.

[0032] In a specific embodiment, when (L j +H j ) / 2≤X' j ≤H j When X' j <(L j +H j ) / 2 or X' j >H j , the dimension is marked as "unchecked".

[0033] S400, if C>C0, perform the first type of data processing on X' and determine whether to execute step S300 or step S600 according to the result of the first type of data processing;

[0034] Specifically, the first type of data processing includes the following steps:

[0035] S401, obtain the preset dimension threshold m, if K < m, divide the data dimension in Q according to the ranking P j Sort and delete the data corresponding to the K+1th data dimension to the mth data dimension in Q to obtain the third intermediate data list X" under the K dimension;

[0036] S403. If K=m, use Q as the third intermediate data list X" under K dimension;

[0037] S405. If K>m, The data dimension is ranked according to P j Sort by Delete the data corresponding to the K+1th data dimension to the data corresponding to the mth data dimension, and obtain the third intermediate data list X" under K dimension;

[0038] S407, based on X" and the objective function f(), obtain the objective function value f(X") and based on f(X") and a preset threshold e, execute step S409;

[0039] S409 , when f(X”) is less than the preset threshold e, execute step S600 ; otherwise, when f(X”) is greater than or equal to the preset threshold e, replace X with X” and execute S300 based on X”.

[0040] Specifically, Pj is in X j The corresponding ranking is used to reflect X j the importance of.

[0041] The above step S400 is to constrain the variables, which can accurately limit the number of dimensions in the variables and force the variables to meet the preset target number of dimensions, thereby improving the efficiency and accuracy of dimension screening and compression.

[0042] S500, if C≤C0, perform the second type of data processing on X' and determine whether to execute step S300 or step S600 according to the result of the second type of data processing;

[0043] Specifically, step S500 also includes the following steps:

[0044] S501. If K=m, based on X' and f(), obtain a second objective function value f(X'), and based on f(X') and a preset threshold e, execute step S505;

[0045] S503. When K≠m, based on X' and the penalty function f'(), obtain the function value f'(X'), and based on f(X') and the preset threshold e, execute step S505;

[0046] S505. According to K=m, when f(X') < the preset threshold e, execute step S600; otherwise, when f(X') ≥ the preset threshold e, replace X with X' and execute step S300;

[0047] Alternatively, when K≠m, when f'(X')<preset threshold e, step S600 is executed; otherwise, when f'(X')≥preset threshold e, X is replaced by X' and step S300 is executed.

[0048] Specifically, e is a preset threshold value and the threshold value can be determined by those skilled in the art based on the inherent characteristics of the product or article, for example, the expected amount of material consumed in the product preparation process, which will not be repeated here.

[0049] Preferably, the penalty coefficient f'() satisfies the following conditions:

[0050] f'(X')=a×(Km) 2 ×f(X'), where a is a preset parameter and a≥1.

[0051] Step S500, while still constraining the variables, further constrains the objective function. This guides the optimization process toward a region that satisfies the target dimension constraints, gradually iterating to the optimal variable constraint conditions. Compared to S400, this step is relatively gentle, avoiding the potential for data divergence and other undesirable consequences of forced constraints, and improving the stability of the variable selection and compression process.

[0052] In a specific embodiment, in step S400, when X is replaced by X' and S300 is executed based on X', it can be understood that the next round of iteration process has begun, and the iteration is completed only when f(X'') is less than e.

[0053] Similarly, in step S500, when X is replaced by X' and step S300 is performed based on X', it can be understood that the next round of iteration has begun, and the iteration is completed only when f(X') or f'(X') is less than e.

[0054] S600 , according to K, obtaining a target data list corresponding to X that is subjected to constraint processing in the K dimension; this can be understood as: X′ or X″ being the target data list.

[0055] The present application has at least the following technical effects: obtaining original data as the data to be optimized, setting a first preset value list and a second preset value list for the data to be optimized, and constructing a first objective function based on the data to be optimized, the first preset value and the second preset value and setting the target number of dimensions, obtaining the target proportion value under unconstrained conditions, and determining the dimension that meets the target number of dimensions based on the target proportion value. The present invention can further optimize a certain existing algorithm and screen and compress the dimensions of the variables to be optimized. In a non-invasive manner, the variables to be optimized are further processed based on the data processing method of the variables to be optimized themselves, so that the optimization algorithm can be expanded without changing the core of the original optimization algorithm, which is conducive to improving the applicability of the original optimization algorithm. The data after further optimization is more targeted on the basis of the original data and can meet the minimum threshold of the demand. For example, when producing a certain product, the further optimized data can also reduce the types of materials required to produce the product, and the ratio of the necessary materials can be clearer, which is more conducive to reducing production consumption.

[0056] In a specific application, for example, animal feed needs to be mixed. There are N premix options available on the market, but due to cost and processing complexity, only K products can be purchased for mixing. The goal is to determine which K products to use and the appropriate mix ratios to maximize the total benefit (i.e., the total amount of protein, minerals, and vitamins) while maintaining a fixed cost.

[0057] S=[x1,x2,...,x N ] is the ratio variable to be solved. Here x is either 0, indicating that this ingredient is not used, or a real number between 0 and 1, indicating the proportion of this ingredient used.

[0058] X* is the result after the final screening of this algorithm and is K-dimensional.

[0059] Assume N = 10, K = 5, that is, there are 10 alternative premix products, and 5 of them need to be selected for blending. This algorithm will give which 5 to choose and their respective proportions.

[0060] The commodities here can also include pharmaceuticals, concrete, and cosmetics. If screening is not required, many optimization algorithms can directly determine the optimal ratio. If screening is required, existing optimization algorithms cannot be directly applied and can only obtain the optimal result after dimensional constraints through traversal. For example, in the above embodiment, it is necessary to traverse and calculate C(10,5) = 252 times. When the number N is large, the number of required calculations increases exponentially, making it difficult to obtain an accurate optimal solution in real-world applications. This algorithm, however, builds on existing optimization algorithms and serves as a suite. When applied to the original algorithm, it can efficiently and accurately solve the problem of optimizing ratios under conditions requiring screening, thereby reducing the number and quantity of materials consumed.

[0061] An embodiment of the present application also provides a non-transitory computer-readable storage medium, which can be set in an electronic device to store at least one instruction or at least one program related to implementing a method in a method embodiment. The at least one instruction or the at least one program is loaded and executed by the processor to implement the method provided in the above embodiment.

[0062] An embodiment of the present application also provides an electronic device, including a processor and the aforementioned non-transitory computer-readable storage medium.

[0063] Although some specific embodiments of the present application have been described in detail by way of example, it should be understood by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present application. It should also be understood by those skilled in the art that various modifications may be made to the embodiments without departing from the scope and spirit of the present application. The scope of the present application is defined by the appended claims.

Claims

1. A data optimization method based on dimensional constraints, characterized in that: The method includes the following steps: S100, obtain the original data list X=(X1, X2, ..., X n ), where X j Refers to the original data of the jth dimension, j = 1...n, n is the number of data dimensions, and according to X j , get X j The corresponding first preset value list H=(H1,H2,……,H n ) and the second preset value list L=(L1, L2, ……, L n ), where L j ≤X j ≤H j ; S200, obtain the preset optimization model S and objective function f() and obtain the target proportion value T corresponding to each dimension of X according to the original data list X, the optimization model S and the objective function f() j And based on all T j Get the target proportion list T; S300, pre-process X based on S to obtain an intermediate data list X'=(X'1, X'2, ..., X' n ) and when (L j +H j ) / 2≤X' j ≤H j When , the preset constraint dimension number K and target parameter value C are obtained to compare C with the preset parameter threshold C0; S400. If C > C0, perform a first type of data processing on X' and, based on the result of the first type of data processing, determine whether to execute step S300 or step S600; In step S400, the first type of data processing includes the following steps: S401, obtain the preset dimension threshold m, if K < m, divide the data dimension in Q according to the ranking P j Sort and delete the data corresponding to the K+1th data dimension to the mth data dimension in Q to obtain the third intermediate data list X" under K dimension; wherein, the first key data list Q = (Q1, Q2, ..., Q m ), where Q r Refers to the rth first key data, r = 1...m; S403. If K = m, use Q as the third intermediate data list X'' in the K dimension; S405. If K>m, The data dimension is ranked according to P j Sort by Delete the data corresponding to the K+1th data dimension to the data corresponding to the mth data dimension, and obtain the third intermediate data list X" under K dimension; among which, the second key data list in, Refers to the pth second key data, p = 1...nK; S407. Based on X'' and the objective function f(), obtain the first objective function value f(X'') and, based on f(X'') and a preset threshold e, execute step S409; S409. When f(X'') < the preset threshold e, execute step S600; otherwise, when f(X'') ≥ the preset threshold e, replace X with X'' and execute S300 based on X''; S500. If C ≤ C0, perform a second type of data processing on X' and, based on the result of the second type of data processing, determine whether to execute step S300 or step S600; the second type of data processing includes the following steps: S501. Obtain a preset dimension threshold m. When K = m, based on X' and f(), obtain the second objective function value f(X') and, based on f(X') and a preset threshold e, execute step S505; S503. When K ≠ m, based on X' and the penalty function f'(), obtain the third objective function value f'(X'); and, based on f'(X') and a preset threshold e, execute step S505; S505. According to K = m, when f(X') < e, execute step S600; otherwise, when f(X') ≥ e, replace X with X' and execute step S300; Or, according to K ≠ m, when f'(X') < e, execute step S600; otherwise, when f'(X') ≥ e, replace X with X' and execute step S300; S600. According to K, obtain the target data list corresponding to X that undergoes constraint processing in the K dimension.

2. The method according to claim 1, characterized in that In step S200, the following steps are further included: S201. Based on X, S, and f(), obtain the first optimization value V0 corresponding to X; S203. Delete X from X j , build the second intermediate data list according to S and f(), get The corresponding second optimized value V j ; S205, according to V0 and V j , get The corresponding target proportion value T j =V0 / V j ; S207, based on T j , construct the target proportion value list T = (T1, T2, ..., T n ); S209, based on the target proportion value T of each dimension j , sort each dimension in descending order and obtain the corresponding ranking P for each dimension j j .

3. The method according to claim 1, characterized in that The value range of C is 0 to 1.

4. The method according to claim 1, wherein [[ID=1 When (L j +H j ) / 2≤X' j ≤H j When X' j As the first key data and constructing the first key data list Q=(Q1, Q2, ..., Q m ), where Q r Refers to the rth first key data, r = 1...m; When X' j <(L j +H j ) / 2 or X' j >H j When Q is deleted from X' and any intermediate data in X' after Q is deleted is used as the second key data to construct the second key data list in, It refers to the pth second key data, where p=1…nK.

5. The method according to claim 1, wherein ​ f'(X')=a×(Km) 2 ×f(X'), where a is a preset parameter and a≥1. ​ 7. An electronic device, characterized in that: ​

Images