Post-processing method and device combining LLM with parallel genetic algorithm

Abstract

The application relates to a post-processing method and device combining an LLM and a parallel genetic algorithm, wherein the method comprises the following steps: generating an initial population of division schemes meeting the conditions of format correctness and individual diversity by using the autoregressive sampling characteristics of a language model LLM; performing global optimization on the initial population of division schemes based on a preset parallel genetic algorithm to obtain an optimization result; and screening a Pareto frontier solution set meeting a preset multi-objective optimization condition by using the optimization result and a preset evaluation function. Thus, the problems that, in the related art, due to the fact that a greedy partition algorithm is prone to falling into local optimization and the output of a deterministic algorithm is unique, an effective global post-processing optimization mechanism is lacked, an initial population with sufficient divergence cannot be provided, and due to the fact that an LLM is prone to introducing format errors when improving diversity, a division scheme cannot be parsed by a subsequent process, and format correctness and scheme diversity cannot be considered are solved.

Description

Technical Field

[0001] This application relates to the field of large language model technology, and in particular to a post-processing method and apparatus that combines LLM with parallel genetic algorithm. Background Technology

[0002] Currently, time-series databases are widely used in scenarios such as the Internet of Things, financial quantitative finance, and IT monitoring. Due to their massive data scale, data tables are usually divided into multiple physical partitions within a single node to optimize query I / O costs.

[0003] In related technologies, deterministic algorithms based on greedy strategies, such as KD-tree (K-Dimensional Tree) and QD-tree (Quad Tree), are used to partition data. Heuristic algorithms such as genetic algorithms can be introduced in the optimization stage to iteratively optimize the partitioning scheme. At the same time, some schemes attempt to use autoregressive sampling of LLM (Large Language Model) to generate partitioning schemes and introduce random strategies such as temperature sampling and top-p (Nucleus Sampling) sampling to improve the diversity of output.

[0004] However, in related technologies, greedy partitioning algorithms are prone to getting trapped in local optima, resulting in a lack of effective global post-processing optimization mechanisms. Deterministic algorithms have a unique output, making it impossible to provide an initial population with sufficient divergence. This makes it difficult for heuristic global optimization methods to explore better solution spaces due to the homogeneity of individuals. Furthermore, LLM is prone to introducing format errors when improving diversity, causing partitioning schemes to be unparseable by subsequent processes. It is difficult to balance format correctness and scheme diversity, which urgently needs improvement. Summary of the Invention

[0005] This application provides a post-processing method and apparatus that combines LLM with parallel genetic algorithm to solve the problems in related technologies, such as the greedy partitioning algorithm being prone to getting trapped in local optima and the deterministic algorithm having a unique output, resulting in the lack of an effective global post-processing optimization mechanism and the inability to provide an initial population with sufficient divergence. Furthermore, LLM is prone to introducing format errors when improving diversity, which makes it difficult for the partitioning scheme to be parsed by subsequent processes, thus making it difficult to balance format correctness and scheme diversity.

[0006] The first aspect of this application provides a post-processing method combining LLM and parallel genetic algorithm, comprising the following steps: using the autoregressive sampling characteristics of the language model LLM to generate an initial population of partitioning schemes that satisfies both format correctness and individual diversity; performing global optimization on the initial population of partitioning schemes based on a preset parallel genetic algorithm to obtain optimization results; and using the optimization results and a preset evaluation function to select Pareto front solution sets that satisfy preset multi-objective optimization conditions.

[0007] Through the above-mentioned technical means, the embodiments of this application can construct an initial population for partitioning schemes by leveraging the autoregressive sampling characteristics of LLM, and perform global optimization by combining the global search advantages of parallel genetic algorithms. This achieves multi-objective collaborative optimization of time-series database partitioning schemes, thereby breaking through the local optimum limitation of greedy algorithms. Furthermore, by employing parallel genetic algorithms to efficiently handle large-scale data partitioning problems, a balance is achieved between the two conflicting optimization objectives of the number of partitions and I / O cost. In this way, while ensuring the correctness of the LLM output format, the random sampling characteristics of LLM are effectively utilized to generate diverse partitioning schemes.

[0008] Optionally, in one embodiment of this application, the step of generating an initial population for a partitioning scheme that satisfies both format correctness and individual diversity by utilizing the autoregressive sampling characteristics of the Language Model LLM includes: encoding the database schema and query load into prompt words, and inputting the prompt words into the Language Model LLM; determining the structural constraint sampling strategy of the Language Model LLM based on the condition of both format correctness and individual diversity; generating a partitioning tree JSON based on the structural constraint sampling strategy and the autoregressive sampling characteristics of the Language Model LLM, and converting the partitioning tree JSON into a partition number vector to constitute the initial population for the partitioning scheme.

[0009] Through the aforementioned technical means, the embodiments of this application can enable LLM to accurately capture the core features of database patterns and query loads by reasonably encoding prompt words. Based on the structural constraint sampling strategy, an initial population of partitioning schemes is generated. Thus, when LLM generates key format words for the partitioning tree, it automatically degenerates into approximately greedy sampling to ensure correct format. During the content filling stage, a diverse pool of candidate words is maintained, thereby effectively avoiding format illusion. An effective balance is achieved between format compliance and output diversity, providing a foundation for the global optimization of subsequent parallel genetic algorithms.

[0010] Optionally, in one embodiment of this application, the step of globally optimizing the initial population of the partitioning scheme based on a preset parallel genetic algorithm to obtain the optimization result includes: inputting the initial population, crossover probability, mutation probability, and maximum number of generations based on the parallel genetic algorithm and the initial population of the partitioning scheme; in each generation, selecting parents to form a mating pool based on NSGA-II (Non-dominated Sorting Genetic Algorithm II), performing crossover operations on the mating pool according to the crossover probability to generate an intermediate population, and performing mutation operations according to the mutation probability to generate a progeny population; after merging the parent population and the progeny population, selecting the next generation population based on NSGA-II until returning to the Pareto front, and determining the optimization result.

[0011] Through the above-mentioned technical means, the embodiments of this application can combine parallel genetic algorithms and NSGA-II algorithms to promote the population to converge toward the Pareto front while maintaining population diversity, so as to determine the optimization results, improve computational efficiency, and achieve fine-grained parallelization of crossover operations and effectively reduce the number of partitions through horizontal parallel crossover and vertical parallel mutation strategies, thereby making full use of multi-core computing resources and efficiently handling population evolution of large-scale data partitioning problems.

[0012] Optionally, in one embodiment of this application, before using the optimization results and the preset evaluation function to select the Pareto front solution set that satisfies the preset multi-objective optimization conditions, the method further includes: determining the evaluation dimension based on minimizing the number of partitions and minimizing the I / O cost; and generating the preset evaluation function based on the evaluation dimension.

[0013] Through the above-mentioned technical means, the embodiments of this application can define the evaluation function according to two dimensions: minimizing the number of partitions and minimizing I / O cost. This makes the evaluation function more in line with the actual needs of time series database partitioning, providing accurate decision-making basis for Pareto front screening, and enabling post-processing optimization to obtain a partitioning scheme that achieves a balance between two mutually constraining objectives.

[0014] Optionally, in one embodiment of this application, the step of using the optimization results and a preset evaluation function to select Pareto front solutions that satisfy preset multi-objective optimization conditions includes: calculating the number of partitions and I / O cost for each individual based on the optimization results and the preset evaluation function; introducing Pareto dominance relations to perform non-dominated sorting based on the number of partitions and I / O cost for each individual to determine multiple non-dominated layers; calculating the congestion distance in each of the multiple non-dominated layers, and selecting the Pareto front solutions that satisfy the preset multi-objective optimization conditions based on the non-dominated layers and the congestion distance.

[0015] Through the above-mentioned technical means, the embodiments of this application can introduce the NSGA-II multi-objective evaluation and screening mechanism, which combines non-dominated sorting and crowding distance to simultaneously optimize the two conflicting objectives of minimizing the number of partitions and minimizing I / O cost, thereby maintaining the convergence and diversity of the population and obtaining a uniformly distributed Pareto front solution set, providing a flexible selection space for actual deployment.

[0016] A second aspect of this application provides a post-processing apparatus combining LLM and a parallel genetic algorithm, comprising: a generation module for generating an initial population of a partitioning scheme that satisfies both format correctness and individual diversity by utilizing the autoregressive sampling characteristics of the language model LLM; an optimization module for globally optimizing the initial population of the partitioning scheme based on a preset parallel genetic algorithm to obtain an optimization result; and a screening module for screening Pareto front solutions that satisfy preset multi-objective optimization conditions using the optimization result and a preset evaluation function.

[0017] Through the above-mentioned technical means, the embodiments of this application can construct an initial population for partitioning schemes by leveraging the autoregressive sampling characteristics of LLM, and perform global optimization by combining the global search advantages of parallel genetic algorithms. This achieves multi-objective collaborative optimization of time-series database partitioning schemes, thereby breaking through the local optimum limitation of greedy algorithms. Furthermore, by employing parallel genetic algorithms to efficiently handle large-scale data partitioning problems, a balance is achieved between the two conflicting optimization objectives of the number of partitions and I / O cost. In this way, while ensuring the correctness of the LLM output format, the random sampling characteristics of LLM are effectively utilized to generate diverse partitioning schemes.

[0018] Optionally, in one embodiment of this application, the generation module includes: an encoding unit, configured to encode the database schema and query load into prompt words, and input the prompt words into the language model LLM; a first determining unit, configured to determine the structural constraint sampling strategy of the language model LLM based on the condition of both format correctness and individual diversity; and a first generation unit, configured to generate a partition tree JSON based on the structural constraint sampling strategy and the autoregressive sampling characteristics of the language model LLM, and convert the partition tree JSON into a partition number vector to form the initial population of the partitioning scheme.

[0019] Through the aforementioned technical means, the embodiments of this application can enable LLM to accurately capture the core features of database patterns and query loads by reasonably encoding prompt words. Based on the structural constraint sampling strategy, an initial population of partitioning schemes is generated. Thus, when LLM generates key format words for the partitioning tree, it automatically degenerates into approximately greedy sampling to ensure correct format. During the content filling stage, a diverse pool of candidate words is maintained, thereby effectively avoiding format illusions. An effective balance is achieved between format compliance and output diversity, providing a foundation for the global optimization of subsequent parallel genetic algorithms.

[0020] Optionally, in one embodiment of this application, the optimization module includes: an input unit, configured to input an initial population, crossover probability, mutation probability, and maximum number of generations based on the parallel genetic algorithm and the initial population of the partitioning scheme; a second generation unit, configured to, in each generation, select parents based on NSGA-II to form a mating pool, perform crossover operations on the mating pool according to the crossover probability to generate an intermediate population, and perform mutation operations according to the mutation probability to generate a offspring population; and a second determination unit, configured to, after merging the parent population and the offspring population, select the next generation population based on NSGA-II until a Pareto front is returned, and determine the optimization result.

[0021] Through the above-mentioned technical means, the embodiments of this application can combine parallel genetic algorithms and NSGA-II algorithms to promote the population to converge toward the Pareto front while maintaining population diversity, so as to determine the optimization results, improve computational efficiency, and achieve fine-grained parallelization of crossover operations and effectively reduce the number of partitions through horizontal parallel crossover and vertical parallel mutation strategies, thereby making full use of multi-core computing resources and efficiently handling population evolution of large-scale data partitioning problems.

[0022] Optionally, in one embodiment of this application, it further includes: a determining module, configured to determine an evaluation dimension based on minimizing the number of partitions and minimizing I / O cost before using the optimization results and the preset evaluation function to select the Pareto front solution set that satisfies the preset multi-objective optimization conditions; and a function generating module, configured to generate the preset evaluation function based on the evaluation dimension before using the optimization results and the preset evaluation function to select the Pareto front solution set that satisfies the preset multi-objective optimization conditions.

[0023] Through the above-mentioned technical means, the embodiments of this application can define the evaluation function according to two dimensions: minimizing the number of partitions and minimizing I / O cost. This makes the evaluation function more in line with the actual needs of time series database partitioning, providing accurate decision-making basis for Pareto front screening, and enabling post-processing optimization to obtain a partitioning scheme that achieves a balance between two mutually constraining objectives.

[0024] Optionally, in one embodiment of this application, the screening module includes: a calculation unit, configured to calculate the number of partitions and I / O cost of each individual based on the optimization result and the preset evaluation function; a sorting unit, configured to introduce Pareto dominance relations to perform non-dominated sorting based on the number of partitions and I / O cost of each individual, so as to determine multiple non-dominated layers; and a screening unit, configured to calculate the congestion distance in each non-dominated layer of the multiple non-dominated layers, and screen out the Pareto front solution set that satisfies the preset multi-objective optimization conditions based on the non-dominated layer and the congestion distance.

[0025] Through the above-mentioned technical means, the embodiments of this application can introduce the NSGA-II multi-objective evaluation and screening mechanism, which combines non-dominated sorting and crowding distance to simultaneously optimize the two conflicting objectives of minimizing the number of partitions and minimizing I / O cost, thereby maintaining the convergence and diversity of the population and obtaining a uniformly distributed Pareto front solution set, providing a flexible selection space for actual deployment.

[0026] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the post-processing method combining LLM and parallel genetic algorithm as described in the above embodiments.

[0027] A fourth aspect of this application provides a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the post-processing method combining LLM and parallel genetic algorithm as described above.

[0028] A fifth aspect of this application provides a computer program product that stores a computer program that, when executed by a processor, implements the post-processing method combining LLM and parallel genetic algorithm as described above.

[0029] This application's embodiments utilize the autoregressive sampling characteristics of LLM to construct an initial population for partitioning schemes. Combined with the global search advantages of parallel genetic algorithms, global optimization is performed, achieving multi-objective collaborative optimization of time-series database partitioning schemes. This overcomes the local optima limitation of greedy algorithms and employs parallel genetic algorithms to efficiently handle large-scale data partitioning problems, achieving a balance between the conflicting optimization objectives of partition number and I / O cost. Furthermore, while ensuring the correctness of the LLM output format, it effectively utilizes the random sampling characteristics of LLM to generate diverse partitioning schemes. This solves the problems in related technologies where greedy partitioning algorithms are prone to getting trapped in local optima, and deterministic algorithms have unique outputs, leading to a lack of effective global post-processing optimization mechanisms and an inability to provide an initial population with sufficient divergence. Additionally, LLM, while increasing diversity, easily introduces format errors, causing partitioning schemes to be unparseable by subsequent processes, making it difficult to balance format correctness and scheme diversity.

[0030] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0031] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart of a post-processing method combining LLM and parallel genetic algorithm according to an embodiment of this application; Figure 2 This is a flowchart of a post-processing method combining LLM and parallel genetic algorithm according to an embodiment of this application; Figure 3 This is a schematic diagram of a post-processing device combining LLM and parallel genetic algorithm according to an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of this application.

[0032] Figure label: 10-Post-processing device combining LLM and parallel genetic algorithm; 100-Generation module, 200-Optimization module, 300-Screening module; 401-Memory, 402-Processor, 403-Communication interface. Detailed Implementation

[0033] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0034] The following describes a post-processing method and apparatus combining LLM and parallel genetic algorithm according to embodiments of this application, with reference to the accompanying drawings. Regarding the related technologies mentioned in the background section, greedy partitioning algorithms are prone to getting trapped in local optima, and deterministic algorithms have unique outputs, resulting in a lack of effective global post-processing optimization mechanisms and an inability to provide an initial population with sufficient divergence. Furthermore, LLM, while improving diversity, easily introduces format errors, causing partitioning schemes to be unparseable by subsequent processes, making it difficult to balance format correctness and scheme diversity. This application provides a post-processing method combining LLM and parallel genetic algorithm. In this method, the autoregressive sampling characteristics of LLM can be used to construct an initial population for partitioning schemes, and the global search advantage of parallel genetic algorithms can be combined for global optimization, achieving multi-objective collaborative optimization of time-series database partitioning schemes. This overcomes the local optima limitation of greedy algorithms and uses parallel genetic algorithms to efficiently process large-scale data partitioning problems, achieving a balance between the conflicting optimization objectives of the number of partitions and I / O cost. Thus, while ensuring the correctness of the LLM output format, the random sampling characteristics of LLM are effectively utilized to generate diverse partitioning schemes. This solves the problems in related technologies, such as the greedy partitioning algorithm being prone to getting trapped in local optima and the deterministic algorithm having a unique output, which leads to the lack of an effective global post-processing optimization mechanism and the inability to provide an initial population with sufficient divergence. Furthermore, LLM is prone to introducing format errors when improving diversity, which makes it difficult for the partitioning scheme to be parsed by subsequent processes, making it difficult to balance format correctness and scheme diversity.

[0035] Specifically, Figure 1 This is a flowchart illustrating a post-processing method combining LLM and parallel genetic algorithm provided in an embodiment of this application.

[0036] like Figure 1 As shown, the post-processing method combining LLM with parallel genetic algorithms includes the following steps: In step S101, the autoregressive sampling characteristics of the language model LLM are used to generate an initial population that satisfies both format correctness and individual diversity.

[0037] It is understood that, in the embodiments of this application, the autoregressive sampling characteristic can refer to the mechanism used in the language model to generate output element by element, which can be understood as the output at the current moment depending on all previously generated outputs. The initial population of the partitioning scheme can refer to the initial input set used for subsequent global optimization by the parallel genetic algorithm, where each individual in the population represents a physical partitioning scheme of a time-series database table.

[0038] In practice, the basic process of LLM autoregressive sampling is as follows: after inputting the input sequence into a multi-layer Transformer, the output is the probability distribution p of the words, where each probability value represents the generation probability of the corresponding word. The conventional sampling process combines temperature sampling and top-p sampling to reshape and truncate the original probability distribution p, which can improve the diversity and robustness of the text.

[0039] Furthermore, the basic sampling strategy described above treats the probability of all terms equally when increasing the temperature to enhance diversity. However, when outputting structured formats (such as curly braces, quotation marks, and colons in JSON), the original probability distribution is extremely biased towards the corresponding format terms. Increasing the temperature can cause incorrect terms to enter the candidate set, resulting in a format illusion.

[0040] Therefore, this application embodiment can employ a min-p (Minimum Probability Sampling) sampling strategy on the model obtained by parameter fine-tuning. Let the preset parameter be β, and let pmax be the maximum value in the current word probability distribution. Then, only when the candidate word u satisfies pu ≥ β·pmax can it enter the candidate set v′.

[0041] This application's embodiments can leverage the autoregressive sampling characteristics of the Language Modeling (LLM) to generate an initial population for a partitioning scheme that satisfies both format correctness and individual diversity. When LLM generates JSON-formatted lexical units, pmax is very close to 1.0, resulting in an extremely high truncation threshold. This automatically eliminates almost all low-probability incorrect options, forcing LLM to behave similarly to greedy sampling, thus ensuring format accuracy. When LLM operates in low-confidence scenarios (such as when filling predicate principal units), pmax is lower, and the truncation threshold decreases accordingly, allowing more candidate lexical units to be included in the sampling pool. This, in turn, allows LLM to generate diverse partitioning trees.

[0042] This application embodiment can ensure the semantic rationality and format compliance of the generated content through the autoregressive sampling characteristics of LLM, and combined with diversity control, generate an initial population of partitioning scheme with correct format and sufficient individual diversity. This effectively solves the problems of homogeneity of the initial population in deterministic algorithms and the format illusion that is easy to occur in conventional sampling of LLM, laying the foundation for improving the global optimality of partitioning scheme.

[0043] Optionally, in one embodiment of this application, the initial population of a partitioning scheme that satisfies both format correctness and individual diversity is generated by utilizing the autoregressive sampling characteristics of the Language Model LLM. This includes: encoding the database schema and query load into prompt words, and inputting the prompt words into the Language Model LLM; determining the structural constraint sampling strategy of the Language Model LLM based on the conditions of both format correctness and individual diversity; generating a partitioning tree JSON based on the structural constraint sampling strategy and the autoregressive sampling characteristics of the Language Model LLM, and converting the partitioning tree JSON into a partition number vector to constitute the initial population of the partitioning scheme.

[0044] It is understood that in this application embodiment, the database schema can refer to the structural information of the data tables in the time-series database, such as column names, data types, and hierarchical relationships between columns; the query load can refer to the set of query statements executed on the database within a certain period and the statistical characteristics of their call frequency. Encoding the database schema and query load into prompt words allows the LLM to accurately capture the actual scenario requirements of the database and generate a partitioning scheme that fits the actual application. The structural constraint sampling strategy can include, but is not limited to, temperature sampling, top-p sampling, and min-p sampling. Temperature sampling, top-p sampling, and min-p sampling can be used in combination, individually, or only a few of these methods can be selected. This can be set by those skilled in the art according to the actual situation, and no specific restrictions are imposed here.

[0045] For example, embodiments of this application can utilize LLM to generate an initial partitioning scheme population. Specifically, embodiments of this application can first encode the database schema and query load into prompt words according to a specified encoding scheme (such as prompt word C format, where prompt word C format only contains the database schema and query load), and input them into the model after parameter fine-tuning.

[0046] In some embodiments, the input sequence can be fed into a multi-layer Transformer to output a probability distribution p of the word elements, where each probability value represents the generation probability of the corresponding word element. The sampling process combines temperature sampling, top-p sampling, and min-p sampling. In this embodiment, the original distribution is first adjusted using a temperature coefficient τ to obtain a smoothed or sharpened new distribution p′. A larger temperature τ results in a smoother distribution and a higher probability of low-probability words being included as candidates; a smaller temperature τ favors high-probability words, and when τ=0, it degenerates into greedy sampling. Subsequently, top-p sampling dynamically selects the minimum word element set v′ whose cumulative probability reaches a threshold p, and output words are randomly drawn from the candidate probability distribution renormalized on v′.

[0047] Of course, in other embodiments, a min-p sampling strategy can be used on the model obtained by parameter fine-tuning. Let the preset parameter be β, and let pmax be the maximum value in the current word probability distribution. Then, a candidate word u can only enter the candidate set v′ if pu ≥ β·pmax. For example, in this embodiment, the parameter β = 0.1 and the temperature τ = 1.5 can be set to generate multiple partition tree JSON outputs. Each partition tree JSON is parsed, and each partition tree is mapped to a partition number vector, forming the initial population H0. The maximum number of partitions M = 128, and the initial population size N = 128.

[0048] To verify the impact of different LLM generation methods on the initial population quality and the final optimization effect, this application also designed six initial population generation schemes, specifically: (1) Randomly generate baselines (randomly fill each dimension of the vector with partition numbers and then renumber).

[0049] (2) Gemini is generated by combining the prompt word A (median sampled CSV data), top-p=0.95, τ=1, and the prompt word is the same for each submission.

[0050] (3) Gemini is generated by combining prompt word B (randomly sampled CSV data), top-p=0.95, τ=1, and the data row of the randomly sampled prompt word changes each time it is submitted.

[0051] (4) Gemini is generated by combining the suggestion word C (Schema and query load only), top-p=0.95, τ=1, and the order of query load in the suggestion words changes with each submission.

[0052] (5) Gemini(w) (with code interpreter) is generated in combination with prompt word C, top-p=0.95, τ=1.

[0053] (6) Fine-tune the reinforcement learning model in combination with the prompt word C, min-p=0.1, τ=1.5.

[0054] The above six initial populations were each input into the same parallel genetic algorithm for optimization. Experimental results show that the I / O cost of all initial populations decreased to varying degrees after post-processing. Among them, (6) fine-tuning the reinforcement learning model + min-p sampling performed the best and was the only initial population that outperformed the QD tree benchmark; (5) Gemini(w) had a high degree of homogeneity in its generation scheme (equivalent to copying several KD trees), and although the quality of the initial solution was acceptable, it lacked distribution, making it difficult for the genetic algorithm to achieve significant improvement. This verifies the effectiveness of the min-p sampling strategy in providing good diversity while ensuring the correctness of the format, and also confirms the key influence of the distribution of the initial population on the optimization effect of the genetic algorithm.

[0055] This application embodiment can enable LLM to accurately capture the core features of database patterns and query load by reasonably encoding prompt words. It generates an initial population of partitioning schemes based on structural constraint sampling strategies, thereby automatically degenerating into approximately greedy sampling when LLM generates key format words for the partitioning tree to ensure format correctness. During the content filling stage, it maintains a diverse pool of candidate words, thus effectively avoiding format illusions and achieving an effective balance between format compliance and output diversity, providing a foundation for the global optimization of subsequent parallel genetic algorithms.

[0056] In step S102, the initial population of the partitioning scheme is globally optimized based on a preset parallel genetic algorithm to obtain the optimization result.

[0057] It is understood that the preset parallel genetic algorithm in the embodiments of this application can refer to a pre-configured evolutionary algorithm framework that can simultaneously evaluate and evolve multiple populations. It can be understood as accelerating the iterative evolution process of the population and improving optimization efficiency by using parallel computing resources through parallel evaluation, crossover, mutation and other operations. The parameters of the parallel genetic algorithm may include parallel granularity, crossover probability, mutation probability and maximum number of generations. The preset parallel genetic algorithm can be set by those skilled in the art according to the actual situation, and no specific restrictions are made here.

[0058] In actual implementation, this embodiment can be based on a preset parallel genetic algorithm to globally optimize the initial population of the partitioning scheme through individual encoding and genetic operations. Specifically, when encoding individuals, this embodiment uses a partition number vector to represent a layout, and relabels this vector to ensure the continuity of partition numbers.

[0059] Furthermore, embodiments of this application can perform horizontal parallel crossover operations and vertical parallel mutation operations. The horizontal parallel crossover strategy involves dividing the individual vectors into segments according to random block size and distributing them to multiple CPU cores. Each core independently performs the crossover operation and then concatenates the segments, achieving fine-grained parallelization of the crossover operation. The vertical parallel mutation strategy involves evenly dividing the population and distributing it to multiple CPU cores to independently perform mutation operations, and combining two mutation strategies (interval reset and partition merging) to effectively reduce the number of partitions.

[0060] The embodiments of this application can perform efficient global optimization of the initial population of the partitioning scheme using a parallel genetic algorithm. By leveraging the parallel processing mechanism and the global search capability of the genetic algorithm, the optimization speed can be improved, overcoming the local optimum limitation of the greedy algorithm, and efficiently handling the population evolution of large-scale data partitioning problems.

[0061] Optionally, in one embodiment of this application, the initial population of the partitioning scheme is globally optimized based on a preset parallel genetic algorithm to obtain the optimization result, including: based on the parallel genetic algorithm and the initial population of the partitioning scheme, inputting the initial population, crossover probability, mutation probability, and maximum number of generations; in each generation of evolution, the parent generation is selected based on NSGA-II to form a mating pool, and the mating pool is subjected to crossover operation according to the crossover probability to generate an intermediate population, and mutation operation is performed according to the mutation probability to generate a offspring population; after merging the parent population and the offspring population, the next generation population is selected based on NSGA-II until the Pareto front is returned, and the optimization result is determined.

[0062] It is understood that in the embodiments of this application, NSGA-II is a genetic algorithm for solving multi-objective optimization problems. It introduces non-dominated sorting, elitist strategy, and crowding ratio algorithm. By dividing the population into multiple non-dominated front levels and sorting them by crowding distance within the same level, it achieves approximation to the Pareto front and uniform distribution of solutions. The Pareto front can refer to the set of all non-dominated solutions in multi-objective optimization, which cannot improve the performance of one objective without degrading the performance of another (e.g., cannot reduce the number of partitions while reducing I / O cost). Returning to the Pareto front completes the global optimization.

[0063] In actual implementation, the embodiments of this application can perform global optimization of the initial population based on a preset parallel genetic algorithm. The overall framework of the algorithm is as follows: input the initial population H0, crossover probability pc, mutation probability pm, and maximum number of generations Gmax; in each generation, the parent generation is selected based on NSGA-II to form a mating pool, the mating pool is subjected to crossover operation according to probability pc to generate an intermediate population, and then a mutation operation is performed according to probability pm to generate the offspring population. After merging the parent and offspring generations, the next generation population is selected based on NSGA-II, and finally the Pareto front is returned.

[0064] Specifically, in the embodiments of this application, when performing individual coding, the partition number vector L=(L1, L2, ...,L...) is used. nt ) represents a layout, where L i This represents the partition number corresponding to the i-th tuple in the data table. The vector is relabeled so that 1, 2, ..., max(L) all appear in L, ensuring the continuity of the partition numbers.

[0065] Furthermore, embodiments of this application can perform horizontal parallel crossover operations and vertical parallel mutation operations. The horizontal parallel crossover operation includes: first, dividing the individual vectors into d sub-vectors according to randomly generated block sizes m1, m2, ..., md (satisfying Σmj=nt), and assigning them to d CPU cores. Each core processes all sub-vectors of all individuals within their corresponding intervals. Before crossover, the numbers of the individuals to be crossovered are selected according to probability pc and randomly paired in pairs. For each pairing relationship (u, v) and each CPU core j, a range [l, r] is selected in the sub-vector dimension corresponding to the current core, and the values ​​of the two individuals within that range are swapped. Finally, the sub-vectors obtained by each core are concatenated sequentially to obtain the crossover offspring individuals.

[0066] The vertical parallel mutation operation involves dividing the population into d groups and assigning each group to d CPU cores. Each core is independently responsible for the mutation operation of its corresponding subpopulation. Two mutation strategies are designed for each individual: Strategy 1 randomly selects a range [l, r] and sets all partition numbers within this range to a random value within that range; Strategy 2 randomly selects two partition numbers and merges them. Both strategies aim to effectively reduce the number of partitions in the layout.

[0067] For example, in this embodiment, the parallel genetic algorithm parameters can be configured first, setting the parallel granularity d=16, crossover probability pc=0.6, mutation probability pm=0.6, and maximum number of generations Gmax=128. Next, this embodiment can perform iterative optimization using the parallel genetic algorithm, calculating the fitness of each individual in the current population (dimensions of partition number and I / O cost). This embodiment can select parent individuals from the current population based on NSGA-II to form a mating pool, and perform horizontal parallel crossover operations on the individuals in the mating pool according to probability pc to generate an intermediate population. This embodiment can perform vertical parallel mutation operations on the individuals in the intermediate population according to probability pm to generate a offspring population, calculate the fitness of each individual in the offspring population, merge the parent and offspring populations, and select the next generation population from the merged population based on NSGA-II until the maximum number of generations is reached, returning to the Pareto front to determine the optimization result.

[0068] The embodiments of this application can combine parallel genetic algorithms and NSGA-II algorithms to promote the population to converge toward the Pareto front while maintaining population diversity, so as to determine the optimization results, improve computational efficiency, and achieve fine-grained parallelization of crossover operations and effectively reduce the number of partitions through horizontal parallel crossover and vertical parallel mutation strategies, thereby making full use of multi-core computing resources and efficiently handling population evolution of large-scale data partitioning problems.

[0069] In step S103, the Pareto front solution set that satisfies the preset multi-objective optimization conditions is selected using the optimization results and the preset evaluation function.

[0070] It is understood that, in the embodiments of this application, the preset evaluation function refers to a function used to evaluate the merits of a partitioning scheme. The preset evaluation function can be based on the needs of the time-series database, setting the evaluation dimensions to minimize the number of partitions and minimize I / O cost to quantify the performance of the partitioning scheme. The preset evaluation function can be set by those skilled in the art according to actual circumstances, and no specific restrictions are imposed here. The preset multi-objective optimization condition refers to simultaneously optimizing multiple conflicting objectives, such as maintaining the convergence and diversity of the population while optimizing the number of partitions and I / O cost. The preset multi-objective optimization condition can be set by those skilled in the art according to actual circumstances, and no specific restrictions are imposed here. The Pareto front solution set refers to the set of non-dominated solutions that satisfy the preset multi-objective optimization condition.

[0071] In actual implementation, the embodiments of this application can extract the Pareto front from the final population based on the optimization results, as the optimized data partitioning scheme set, and introduce the NSGA-II algorithm based on the preset evaluation function to achieve screening under multi-objective optimization, determine the Pareto front solution set that meets the preset multi-objective optimization conditions, and select the final scheme from it according to the actual deployment requirements (such as focusing on low I / O cost or focusing on a small number of partitions).

[0072] Specifically, the embodiments of this application can introduce the NSGA-II multi-objective evaluation and screening mechanism. By combining non-dominated sorting and congestion distance, a uniformly distributed Pareto front solution set can be obtained between the contradictory objectives of minimizing the number of partitions and minimizing I / O cost, providing a flexible scheme selection space.

[0073] The embodiments of this application can evaluate the optimization results through an evaluation function, and combined with multi-objective optimization conditions, select Pareto front solution sets that meet the conditions, thereby ensuring that the partitioning schemes in the solution set are all optimal solutions under multi-objective conditions, providing diverse optimal solution selections for time series databases, and improving the practicality and flexibility of the technical solution.

[0074] Optionally, in one embodiment of this application, before using the optimization results and a preset evaluation function to select the Pareto front solution set that satisfies the preset multi-objective optimization conditions, the method further includes: determining the evaluation dimension based on minimizing the number of partitions and minimizing the I / O cost; and generating a preset evaluation function based on the evaluation dimension.

[0075] It is understood that minimizing the number of partitions in this application embodiment can be understood as reducing the total number of physical partitions to reduce storage overhead and management complexity; minimizing I / O cost can be understood as reducing the number of I / O read / write operations and the time spent during database queries to improve query response speed; evaluation dimensions refer to the core indicators used to evaluate the merits of the partitioning scheme, such as minimizing the number of partitions and minimizing I / O cost.

[0076] In actual implementation, the embodiments of this application can define two dimensions of the evaluation function: minimizing the number of partitions |L| and minimizing the I / O cost f. io The two goals are in conflict; improvement in one goal often leads to a deterioration in the other.

[0077] For example, embodiments of this application may define the evaluation function in two dimensions: minimizing the number of partitions |L| and minimizing the I / O cost f. io Based on these two dimensions, a two-dimensional evaluation function is constructed. During the evaluation function calculation stage, the system still maintains a vertically parallel state, and each CPU core j will independently calculate the evaluation function of each individual, providing a standardized criterion for evaluating the fitness of individuals in the genetic algorithm.

[0078] The embodiments of this application can define the evaluation function based on two dimensions: minimizing the number of partitions and minimizing I / O cost. This makes the evaluation function more in line with the actual needs of time series database partitioning, providing an accurate decision basis for Pareto front screening, and enabling post-processing optimization to obtain a partitioning scheme that achieves a balance between the two mutually constraining objectives.

[0079] Optionally, in one embodiment of this application, the Pareto front solution set that satisfies the preset multi-objective optimization conditions is selected using the optimization results and a preset evaluation function, including: calculating the number of partitions and I / O cost of each individual based on the optimization results and the preset evaluation function; introducing Pareto dominance relations to perform non-dominated sorting based on the number of partitions and I / O cost of each individual to determine multiple non-dominated layers; calculating the congestion distance in each of the multiple non-dominated layers, and selecting the Pareto front solution set that satisfies the preset multi-objective optimization conditions based on the non-dominated layers and the congestion distance.

[0080] It is understood that, in the embodiments of this application, the Pareto dominance relation refers to two individuals x and y, where x is said to dominate y if and only if x is not inferior to y in both objectives and is strictly superior to y in at least one objective. The non-dominated layer refers to the division of individuals in the optimization results into different levels based on the Pareto dominance relation; the first non-dominated layer is the Pareto front of the current population. Crowding distance refers to the sum of the normalized distances between an individual and its neighboring individuals in both objective dimensions within the same non-dominated layer; it can be used as an indicator to measure the differences between individuals within the same non-dominated layer.

[0081] In practical implementation, the embodiments of this application can calculate the number of partitions and I / O cost for each individual based on the optimization results and a preset evaluation function, and introduce Pareto dominance to compare the superiority of individuals. For two individuals x and y, x is said to dominate y if and only if x is not inferior to y in both objectives and is strictly superior to y in at least one objective. Individuals not dominated by other individuals constitute the first layer of the Pareto front, and dominated individuals are successively assigned to higher layers.

[0082] Furthermore, to accommodate the diversity of solutions, embodiments of this application can calculate the individual crowding distance within each non-dominated layer. For each objective function, using the range of values ​​for individuals in that layer on the objective as a normalization factor, the crowding distance components for boundary individuals (crowding distance is infinite) and internal individuals (normalized difference) are calculated, and the total crowding distance of an individual is the sum of all objective components.

[0083] This application's embodiments can perform population screening and mating pool selection. Specifically, in the next generation population screening, the parent and offspring generations are merged, and individuals from the non-dominant layer are added to the next generation sequentially from low to high until the size of a certain layer exceeds a preset value N. Individuals in that layer are sorted in descending order of crowding distance, with priority given to individuals with larger distances. In the mating pool selection stage, a binary tournament selection based on the non-dominant layer and crowding distance is adopted: individuals with smaller layers are given priority, and within the same layer, individuals with larger crowding distances are given priority.

[0084] The embodiments of this application can introduce the NSGA-II multi-objective evaluation and screening mechanism, which combines non-dominated sorting and crowding distance to simultaneously optimize the two conflicting objectives of minimizing the number of partitions and minimizing I / O cost, thereby maintaining the convergence and diversity of the population and obtaining a uniformly distributed Pareto front solution set, providing a flexible selection space for actual deployment.

[0085] Specifically, it can be combined with Figure 2 As shown, a specific embodiment is used to illustrate in detail the working principle of the post-processing method combining LLM and parallel genetic algorithm in this application.

[0086] like Figure 2As shown, in this embodiment, the database table, query load, and partition limit can be used as input data. Subsequently, this embodiment performs initial population construction, guiding the solver to perform multiple samplings via prompts to generate an initial partitioning scheme and encode it as an index vector, forming the initial population. Further, this embodiment can execute a genetic algorithm to perform parallel genetic algorithm optimization and NSGA-II multi-objective screening on the initial population. Specifically, this embodiment can employ horizontal parallel crossover and vertical parallel mutation to perform parallel genetic algorithm optimization, minimizing both the number of partitions and I / O cost to perform NSGA-II multi-objective screening, achieving multi-objective optimization. The optimized population is then convergent to determine if the maximum number of generations has been reached. If not, the parallel genetic algorithm optimization is returned for further iterative optimization; if so, the final layout is output. This scheme is the optimal solution in the Pareto front solution set, balancing the dual optimization objectives of the number of partitions and I / O cost.

[0087] The post-processing method combining LLM and parallel genetic algorithm proposed in this application can construct an initial population for partitioning schemes through the autoregressive sampling characteristics of LLM, and perform global optimization by combining the global search advantages of parallel genetic algorithm. This achieves multi-objective collaborative optimization of time-series database partitioning schemes, thereby overcoming the local optimum limitation of greedy algorithms. Furthermore, the parallel genetic algorithm efficiently handles large-scale data partitioning problems, achieving a balance between the conflicting optimization objectives of partition number and I / O cost. This ensures the correctness of LLM output format while effectively utilizing the random sampling characteristics of LLM to generate diverse partitioning schemes. Therefore, this solves the problems in related technologies where greedy partitioning algorithms are prone to getting trapped in local optima, and deterministic algorithms have unique outputs, leading to a lack of effective global post-processing optimization mechanisms and an inability to provide an initial population with sufficient divergence. Additionally, LLM, while increasing diversity, easily introduces format errors, causing partitioning schemes to be unparseable in subsequent processes, making it difficult to balance format correctness and scheme diversity.

[0088] Next, referring to the accompanying drawings, a post-processing apparatus combining LLM and parallel genetic algorithm according to an embodiment of this application is described.

[0089] Figure 3 This is a schematic diagram of the post-processing device combining LLM and parallel genetic algorithm according to an embodiment of this application.

[0090] like Figure 3 As shown, the post-processing device 10 combining LLM with parallel genetic algorithm includes: generation module 100, optimization module 200 and screening module 300.

[0091] Among them, the generation module 100 is used to generate an initial population of a partitioning scheme that satisfies both format correctness and individual diversity by utilizing the autoregressive sampling characteristics of the language model LLM.

[0092] The optimization module 200 is used to perform global optimization on the initial population of the partitioning scheme based on a preset parallel genetic algorithm to obtain the optimization result.

[0093] The filtering module 300 is used to filter out Pareto front solutions that meet preset multi-objective optimization conditions using optimization results and preset evaluation functions.

[0094] Optionally, in one embodiment of this application, the generation module 100 includes: an encoding unit, a first determining unit, and a first generation unit.

[0095] The encoding unit is used to encode the database schema and query load into prompt words, and input the prompt words into the language model LLM.

[0096] The first determining unit is used to determine the structural constraint sampling strategy of the language model LLM based on the conditions of both format correctness and individual diversity.

[0097] The first generation unit is used to generate a partition tree JSON based on the structural constraint sampling strategy and the autoregressive sampling characteristics of the language model LLM, and convert the partition tree JSON into a partition number vector to form the initial population of the partitioning scheme.

[0098] Optionally, in one embodiment of this application, the optimization module 200 includes: an input unit, a second generation unit, and a second determination unit.

[0099] The input unit is used to initialize the population based on the parallel genetic algorithm and partitioning scheme, and inputs the initial population, crossover probability, mutation probability, and maximum number of generations.

[0100] The second generation unit is used to select parents to form a mating pool based on NSGA-II in each generation of evolution, perform crossover operations on the mating pool according to the crossover probability to generate an intermediate population, and perform mutation operations according to the mutation probability to generate a progeny population.

[0101] The second determining unit is used to select the next generation population based on NSGA-II after merging the parent and offspring populations, until the Pareto front is returned, and to determine the optimization result.

[0102] Optionally, in one embodiment of this application, the post-processing device 10 combining LLM with parallel genetic algorithm further includes: a determination module and a function generation module.

[0103] The determination module is used to determine the evaluation dimension based on minimizing the number of partitions and minimizing I / O cost before using the optimization results and the preset evaluation function to select the Pareto front solution set that meets the preset multi-objective optimization conditions.

[0104] The function generation module is used to generate a preset evaluation function based on the evaluation dimension before using the optimization results and preset evaluation functions to select the Pareto front solution set that satisfies the preset multi-objective optimization conditions.

[0105] Optionally, in one embodiment of this application, the filtering module 300 includes: a calculation unit, a sorting unit, and a filtering unit.

[0106] The computing unit is used to calculate the number of partitions and I / O cost for each individual based on the optimization results and a preset evaluation function.

[0107] The sorting unit is used to perform non-dominated sorting based on the number of partitions and I / O cost of each individual, introducing Pareto dominance relations to determine multiple non-dominated layers.

[0108] The filtering unit is used to calculate the crowding distance in each of a set of multiple non-dominated layers, and based on the non-dominated layers and the crowding distance, to filter out the Pareto front solution set that satisfies the preset multi-objective optimization conditions.

[0109] It should be noted that the foregoing explanation of the post-processing method embodiment combining LLM and parallel genetic algorithm also applies to the post-processing device combining LLM and parallel genetic algorithm in this embodiment, and will not be repeated here.

[0110] The post-processing device combining LLM and parallel genetic algorithm proposed in this application can construct an initial population for partitioning schemes through the autoregressive sampling characteristics of LLM, and perform global optimization by combining the global search advantages of parallel genetic algorithm. This achieves multi-objective collaborative optimization of time-series database partitioning schemes, thereby overcoming the local optimum limitation of greedy algorithms. Furthermore, it uses parallel genetic algorithm to efficiently handle large-scale data partitioning problems, achieving a balance between the conflicting optimization objectives of partition number and I / O cost. This ensures the correctness of LLM output format while effectively utilizing the random sampling characteristics of LLM to generate diverse partitioning schemes. Therefore, it solves the problems in related technologies where greedy partitioning algorithms are prone to getting trapped in local optima, and deterministic algorithms have unique outputs, resulting in a lack of effective global post-processing optimization mechanisms and an inability to provide an initial population with sufficient divergence. Additionally, LLM, while increasing diversity, easily introduces format errors, causing partitioning schemes to be unparseable by subsequent processes, making it difficult to balance format correctness and scheme diversity.

[0111] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include: The memory 401, the processor 402, and the computer program stored on the memory 401 and capable of running on the processor 402.

[0112] When the processor 402 executes the program, it implements the post-processing method that combines LLM and parallel genetic algorithm provided in the above embodiments.

[0113] Furthermore, electronic devices also include: Communication interface 403 is used for communication between memory 401 and processor 402.

[0114] The memory 401 is used to store computer programs that can run on the processor 402.

[0115] Memory 401 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0116] If the memory 401, processor 402, and communication interface 403 are implemented independently, then the communication interface 403, memory 401, and processor 402 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized into address buses, data buses, control buses, etc. For ease of representation, Figure 4 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0117] Optionally, in a specific implementation, if the memory 401, processor 402, and communication interface 403 are integrated on a single chip, then the memory 401, processor 402, and communication interface 403 can communicate with each other through an internal interface.

[0118] Processor 402 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0119] This application also provides a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the post-processing method combining LLM and parallel genetic algorithm as described above.

[0120] This application also provides a computer program product storing a computer program that, when executed by a processor, implements the post-processing method combining LLM and parallel genetic algorithm as described above.

[0121] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0122] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0123] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0124] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0125] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, it can be implemented using any one or more of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0126] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0127] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0128] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A post-processing method combining LLM and parallel genetic algorithm, characterized in that, Includes the following steps: By utilizing the autoregressive sampling characteristics of the language model LLM, an initial population for a partitioning scheme that satisfies both format correctness and individual diversity is generated. Based on a preset parallel genetic algorithm, the initial population of the partitioning scheme is globally optimized to obtain the optimization result; The Pareto front solution set that satisfies the preset multi-objective optimization conditions is selected using the optimization results and the preset evaluation function.

2. The method according to claim 1, characterized in that, The method of generating an initial population for a partitioning scheme that satisfies both format correctness and individual diversity by utilizing the autoregressive sampling characteristics of the Language Modeling (LLM) includes: The database schema and query load are encoded into prompt words, and the prompt words are input into the language model LLM; The structural constraint sampling strategy of the language model LLM is determined based on the conditions of both format correctness and individual diversity. Based on the structural constraint sampling strategy and the autoregressive sampling characteristics of the language model LLM, a partition tree JSON is generated, and the partition tree JSON is converted into a partition number vector to form the initial population of the partitioning scheme.

3. The method according to claim 1, characterized in that, The pre-defined parallel genetic algorithm is used to globally optimize the initial population of the partitioning scheme to obtain the optimization result, including: Based on the parallel genetic algorithm and the partitioning scheme, the initial population, crossover probability, mutation probability, and maximum number of generations are input. In each generation of evolution, a mating pool is formed by selecting the parent generation based on NSGA-II, and a crossover operation is performed on the mating pool according to the crossover probability to generate an intermediate population, and a mutation operation is performed according to the mutation probability to generate a progeny population. After merging the parent and offspring populations, the next generation population is selected based on NSGA-II until the Pareto front is returned, and the optimization result is determined.

4. The method according to claim 1, characterized in that, Before using the optimization results and the preset evaluation function to select the Pareto front solution set that satisfies the preset multi-objective optimization conditions, the process further includes: The evaluation dimensions are determined based on minimizing the number of partitions and minimizing I / O cost; The preset evaluation function is generated based on the evaluation dimensions.

5. The method according to claim 4, characterized in that, The step of using the optimization results and a preset evaluation function to select the Pareto front solution set that satisfies the preset multi-objective optimization conditions includes: Based on the optimization results and the preset evaluation function, the number of partitions and I / O cost for each individual are calculated; Based on the number of partitions and I / O cost of each individual, Pareto dominance relation is introduced for non-dominated sorting to determine multiple non-dominated layers; The congestion distance is calculated in each of the multiple non-dominated layers, and based on the non-dominated layers and the congestion distance, the Pareto front solution set that satisfies the preset multi-objective optimization conditions is selected.

6. A post-processing device combining LLM and parallel genetic algorithm, characterized in that, include: The generation module is used to generate an initial population for a partitioning scheme that satisfies both format correctness and individual diversity by utilizing the autoregressive sampling characteristics of the language model LLM. An optimization module is used to perform global optimization on the initial population of the partitioning scheme based on a preset parallel genetic algorithm to obtain the optimization result; The filtering module is used to filter out Pareto front solutions that satisfy preset multi-objective optimization conditions using the optimization results and preset evaluation functions.

7. The apparatus according to claim 6, characterized in that, The generation module includes: An encoding unit is used to encode the database schema and query load into prompt words, and input the prompt words into the language model LLM; The first determining unit is used to determine the structural constraint sampling strategy of the language model LLM based on the condition of both format correctness and individual diversity. The first generation unit is used to generate a partition tree JSON based on the structural constraint sampling strategy and the autoregressive sampling characteristics of the language model LLM, and to convert the partition tree JSON into a partition number vector to form the initial population of the partitioning scheme.

8. An electronic device, characterized in that, include: The memory, the processor, and the computer program stored in the memory and capable of running on the processor, wherein the processor executes the program to implement the post-processing method combining LLM and parallel genetic algorithm as described in any one of claims 1-5.

9. A non-volatile computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the post-processing method combining LLM with parallel genetic algorithms as described in any one of claims 1-5.

10. A computer program product, comprising a computer program, characterized in that, The computer program is executed to implement the post-processing method combining LLM with parallel genetic algorithms as described in any one of claims 1-5.

Images