Stock selection condition splitting and rewriting method and system

Abstract

The application provides a stock selection condition splitting and rewriting method and system, comprising the following steps: S1, obtaining a text T containing only a user question, and preprocessing the obtained text T; the preprocessed text T is used to obtain a factor set J with a similarity to the text T satisfying a preset requirement based on a stock selection factor library F, so as to obtain a labeled data set; S2, expanding the labeled data set according to a preset requirement to obtain an expanded labeled data set; S3, merging the labeled data set and the expanded labeled data set, and preprocessing the merged labeled data set to obtain a preprocessed labeled data set; S4, training a rewriting model by using the preprocessed labeled data set to obtain a trained rewriting model, and splitting and rewriting the stock selection condition by using the trained rewriting model; the rewriting model is based on a qwen model to split and rewrite the user question text according to the stock selection factor library.

Description

Technical Field

[0001] This invention relates to the field of large language model technology, specifically to a method and system for splitting and rewriting stock selection conditions, and more specifically to a method and system for splitting and rewriting stock selection conditions based on fine-tuning of large model instructions. Background Technology

[0002] Existing technologies typically preprocess user natural language queries into "standard node sequences" and construct node trees to represent "index-condition" combinations; then, the node trees are mapped to backend fields and translated into executable data query instructions. This process emphasizes the engineered processing of dictionaries / configuration dictionaries: first, natural language fragments are mapped to indicators / fields using dictionaries, and then conditional expressions are formed based on node types and predefined rules to generate queries. In engineering implementation, this type of method shifts most of the complexity of semantic parsing to offline dictionary and rule design and maintenance, resulting in high maintenance costs, poor generalization, weak support for new domains or non-standard expressions, and insufficient robustness to fuzzy and complex natural language.

[0003] When user statements contain nested conditions, ambiguous modifiers, or omitted information, rule-based splitting is prone to incorrect segmentation or ambiguity, leading to incorrect node tree construction and incorrect query results.

[0004] Patent document CN117370507A (application number: 202311120222.2) discloses a question-answering method, system, medium, and device based on task-oriented dialogue in the financial field, including: acquiring a question sentence input by a user; performing intent recognition based on the question sentence to obtain a corresponding recognition result; determining the current question sentence type based on the recognition result, querying corresponding data according to the interface corresponding to the question sentence type, and then obtaining integrated data and returning it to the user. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the purpose of this invention is to provide a method and system for splitting and rewriting stock selection criteria.

[0006] A method for splitting and rewriting stock selection criteria according to the present invention includes:

[0007] Step S1: Obtain text T containing only user questions, and preprocess the obtained text T. Based on the stock selection factor library F, obtain a set of factors J that meet the preset requirements for similarity with text T, thereby obtaining the labeled dataset.

[0008] Step S2: Expand the labeled dataset according to preset requirements to obtain an expanded labeled dataset;

[0009] Step S3: Merge the labeled dataset with the augmented labeled dataset, and preprocess the merged labeled dataset to obtain the preprocessed labeled dataset;

[0010] Step S4: Train the rewriting model using the preprocessed labeled dataset to obtain the trained rewriting model, and use the trained rewriting model to decompose and rewrite the stock selection conditions.

[0011] The rewriting model is based on the qwen model, which splits and rewrites user question text according to the stock selection factor library.

[0012] Preferably, step S1 includes:

[0013] Step S1.1: Export each interaction record R from the online log of the stock selection system, clean and preprocess each exported interaction record R, and extract the text T containing only the user's question.

[0014] Step S1.2: Perform preprocessing on the text T, including sentence segmentation, stop word removal, and sentence vector encoding, to obtain the preprocessed text T;

[0015] Step S1.3: Based on the stock selection factor library F, obtain the factor candidate set S related to the preprocessed text T through vector retrieval or a recall method based on preset rules;

[0016] Step S1.4: Input the text T and the factor candidate set S into the large language model so that the large language model can select the factor set J from the factor candidate set S that meets the preset requirements for similarity with the text T, and obtain the labeled dataset.

[0017] Preferably, step S2 includes:

[0018] Step S2.1: Obtain the set M of labeled questions containing multiple factors in the labeled dataset;

[0019] Step S2.2: Divide the stock selection factor library F into a set of factors with parameters A and a set of factors without parameters B;

[0020] Step S2.3: Expand the labeled question set M based on the parameterized factor set A;

[0021] Step S2.4: Expand the labeled question set M based on the set of unparalleled factors B.

[0022] Preferably, step S2.3 includes:

[0023] Step S2.3.1: Select the parameterized factor P based on the parameterized factor set A;

[0024] Step S2.3.2: Randomly sample from the labeled question set M and combine with the parameterized factor P to generate a fusion sentence that conforms to the style of the labeled question set M through a large language model, and generate the corresponding structured annotation;

[0025] Step S2.3.3: Based on preset rules, perform parameter replacement on the parameterized factor P to generate regularized sentences and automatically generate annotations;

[0026] Step S2.3.4: The sentences obtained from rule expansion are used to generate similar sentences through a large language model, and annotations are automatically generated.

[0027] Preferably, step S2.4 includes:

[0028] Step S2.4.1: Select a parameterless factor N based on the parameterless factor set B;

[0029] Step S2.4.2: Expand similar short sentences using a large language model and automatically generate annotations for the unparalleled factor N.

[0030] Preferably, step S3 includes: merging the labeled dataset and the expanded labeled dataset according to a preset ratio, and performing deduplication and sensitive sample oversampling processing on the merged dataset to obtain a preprocessed labeled dataset.

[0031] A stock selection criterion splitting and rewriting system provided by the present invention includes:

[0032] Module M1: Obtains text T containing only user questions, preprocesses the obtained text T, and obtains a set of factors J with similarity to text T that meet the preset requirements based on the stock selection factor library F, thereby obtaining the labeled dataset;

[0033] Module M2: Expands the labeled dataset according to preset requirements to obtain an expanded labeled dataset;

[0034] Module M3: Merges the labeled dataset with the augmented labeled dataset and preprocesses the merged labeled dataset to obtain the preprocessed labeled dataset;

[0035] Module M4: Trains the rewriting model using the preprocessed labeled dataset to obtain the trained rewriting model, and uses the trained rewriting model to decompose and rewrite the stock selection conditions.

[0036] The rewriting model is based on the qwen model, which splits and rewrites user question text according to the stock selection factor library.

[0037] Preferably, the module M1 includes:

[0038] Module M1.1: Export each interaction record R from the online logs of the stock selection system, perform cleaning and preprocessing on each exported interaction record R, and extract the text T containing only user questions;

[0039] Module M1.2: Performs preprocessing on text T, including sentence segmentation, stop word removal, and sentence vector encoding, to obtain preprocessed text T;

[0040] Module M1.3: Based on the stock selection factor library F, obtain the set of factor candidates S related to the preprocessed text T through vector retrieval or a recall method based on preset rules;

[0041] Module M1.4: Input the text T and the factor candidate set S into the large language model so that the large language model can select the factor set J from the factor candidate set S that meets the preset requirements of similarity with the text T, and obtain the labeled dataset.

[0042] Preferably, the module M2 includes:

[0043] Module M2.1: Obtains a set M of labeled questions containing multiple factors from the labeled dataset;

[0044] Module M2.2: Divide the stock selection factor library F into a set of factors with parameters A and a set of factors without parameters B;

[0045] Module M2.3: Expands the labeled question set M based on the parameterized factor set A;

[0046] Module M2.4: Expands the labeled question set M based on the set of unparalleled factors B;

[0047] The module M2.3 includes:

[0048] Module M2.3.1: Selecting a parameterized factor P based on a set of parameterized factors A;

[0049] Module M2.3.2: Randomly samples from the labeled question set M and combines it with a parameterized factor P to generate a fused sentence that conforms to the style of the labeled question set M through a large language model, and generates the corresponding structured annotations;

[0050] Module M2.3.3: Generates regularized sentences by replacing parameters of the parameterized factor P according to preset rules, and automatically generates annotations;

[0051] Module M2.3.4: Sentences obtained from rule expansion generate similar sentences through a large language model and automatically generate annotations;

[0052] The module M2.4 includes:

[0053] Module M2.4.1: Selecting a parameterless factor N based on the parameterless factor set B;

[0054] Module M2.4.2: Uses a large language model to expand similar short sentences and automatically generates annotations for the unparalleled factor N.

[0055] Preferably, module M3 includes: merging the labeled dataset and the expanded labeled dataset according to a preset ratio, and performing deduplication and sensitive sample oversampling processing on the merged dataset to obtain a preprocessed labeled dataset.

[0056] Compared with the prior art, the present invention has the following beneficial effects:

[0057] 1. This invention improves the ability to understand and decompose fuzzy, nested clauses and multi-conditional natural language;

[0058] 2. This invention reduces the reliance on manually maintaining the rule base / mapping table, thereby improving the generalization ability to new indicators and new fields;

[0059] 3. This invention enables the model to output structured and standardized stock selection factors and parameters even under complex statements, improving the accuracy and robustness of NL→Query; ultimately achieving a significant improvement in the recognition accuracy and user experience of user-oriented stock selection functions.

[0060] 4. This invention shifts more of the "semantic understanding" to the data-driven and model fine-tuning stage, and solves the processing of a large number of language variations in the training / expansion stage, thereby reducing the immediate dependence on artificial dictionaries / rules and improving the generalization ability to new indicators and new expressions.

[0061] 5. This invention constructs diverse training sets through rule-based and generative expansion, which can improve robustness to nested and fuzzy conditions and reduce the probability of splitting errors propagating downstream. This invention can more easily utilize the contextual reasoning, reference parsing, and implicit information completion capabilities of large models, thereby achieving higher accuracy in complex syntactic / semantic scenarios. Attached Figure Description

[0062] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0063] Figure 1 A flowchart illustrating the method for splitting and rewriting stock selection criteria.

[0064] Figure 2 A schematic diagram of the stock selection criteria splitting and rewriting system. Detailed Implementation

[0065] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0066] According to the present invention, a method and system for splitting and rewriting stock selection criteria are provided, such as... Figures 1 to 2 As shown, this includes: combining factor-based data retrieval, a hybrid manual and automatic annotation approach, and various rule-based and generative augmentation strategies to generate high-quality paired training samples. By designing prompt word templates for the annotated data and fine-tuning the general large model (Qwen), it can accurately identify and structure user stock selection conditions under complex, fuzzy, or nested natural language conditions, thereby significantly improving the accuracy and robustness of NL→ filtering queries.

[0067] The method for splitting and rewriting the stock selection criteria includes:

[0068] Step 1: Obtain text T containing only user questions, and preprocess the obtained text T. Based on the stock selection factor library F, obtain a set of factors J that meet the preset requirements for similarity with text T, thereby obtaining the labeled dataset.

[0069] Step 2: Expand the labeled dataset according to preset requirements to obtain the expanded labeled dataset;

[0070] Step 3: Merge the labeled dataset with the augmented labeled dataset, and preprocess the merged labeled dataset to obtain the preprocessed labeled dataset;

[0071] Step 4: Train the rewriting model using the preprocessed labeled dataset to obtain the trained rewriting model, and use the trained rewriting model to decompose and rewrite the stock selection conditions.

[0072] The rewriting model is based on the qwen model, which splits and rewrites user question text according to the stock selection factor library.

[0073] Specifically, step 1 includes:

[0074] Step 1.1: Export each interaction record R from the online logs of the stock selection system, clean R, and extract the text T containing only the user's question;

[0075] Step 1.2: For each text T, perform preprocessing based on the stock selection factor library F, including: sentence segmentation, stop word removal, and sentence vector encoding; to obtain the preprocessed text T;

[0076] Among them, the stock selection factor library F is a short name for different stock sets. The factor library is a set that contains all stock selection factors. Here, the stock selection factor refers to the stock pool. For example, if "yesterday's涨停" is a factor, then this factor is the short name for the set of stocks that涨停 yesterday.

[0077] Step 1.3: Use vector retrieval or rule-based recall to obtain a candidate factor set S related to T

[0078] Input T and the candidate set S into DeepSeek V3 together, and let DeepSeek V3 select the most relevant selected set J from S;

[0079] In this embodiment, it further includes: manually reviewing and selecting the factor set that exactly corresponds to T from J, denoted as L (manually confirmed set), and manually supplementing the factors omitted in L or missed by DeepSeek recall to obtain the basic labeled data set.

[0080] Specifically, the said step 2 includes: systematic data augmentation based on the stock selection factor library, aiming to increase multi-condition and multi-style labeled samples through various augmentation means, and improve the generalization ability of the model to parametric factors and natural question styles;

[0081] More specifically,

[0082] Step 2.1: Sampling and sample preparation. Sample a batch of labeled question sets M containing multiple conditions (multi-factors) from the labeled samples obtained in step 1;

[0083] Step 2.2: Factor classification. Divide the factor library F into a parametric factor set A, such as "price-to-earnings ratio < X", "liquidity in the range of Y"; and a non-parametric factor set B, such as "strong profitability", "leading stocks";

[0084] Step 2.3: Augmentation of parametric factors P ∈ A:

[0085] Step 2.3.1: Randomly sample from M and combine with the parameter P, and let DeepSeek V3 generate a fusion sentence that conforms to the style of M, and automatically generate the corresponding structured annotation;

[0086] Step 2.3.2: Use rules to replace the parameters of factor P, such as parameter value intervals, typical thresholds, boundary values, to generate regularized sentences, and automatically generate annotations; in this embodiment, replacing the variables of the template with actual values has specific meanings.

[0087] Step 2.3.3: Use DeepSeek V3 to generate similar sentences for the sentences obtained by rule augmentation to increase variants, and automatically generate annotations;

[0088] Step 2.4: Extension of the unparalleled factor N∈B;

[0089] Step 2.4.1: For each unparalleled factor N, use DeepSeek V3 to expand similar short phrases and automatically generate annotations. Here, the expansion of unparalleled factors only generates similar questions to preserve semantic diversity.

[0090] This embodiment also includes: expanding other data types besides factors in the system application scenario, such as stocks and F10 information. The goal is to introduce stock entities and their F10 (company fundamentals, etc.) information to generate questions and tags that are closer to real queries. This is mainly divided into:

[0091] The system expands upon each dimension using rules, and then generates labels based on those rules.

[0092] The data generated from this information is used to generate similar questions using deepseek v3.

[0093] Step 3: Merge the labeled data. The goal is to combine the augmented data from various sources into a training set while ensuring quality and balance.

[0094] Step 3.1: Merge data according to the source (manual annotation in Step 1, generation / rule expansion in Step 2) at a preset ratio, and perform deduplication, semantic duplication detection, and conflict annotation detection.

[0095] Step 3.2: Oversample samples that are sensitive to model training, such as extreme parameters and rare combinations, to ensure that the model can learn rare but important patterns.

[0096] In this embodiment, manual sampling quality inspection is also included, which involves randomly sampling the merged training set, manually reviewing the labeling quality, and recording and correcting errors.

[0097] Specifically, step 4 includes:

[0098] The goal of model training and deployment is to develop a serviceable stock selection condition analysis model F through prompt design and instruction fine-tuning, and then validate it online.

[0099] Design prompt word templates to transform training samples into training pairs that consist of "input (natural language question + optional context) → expected output (structured factor list)".

[0100] Training code is written, using the qwen model as a base, and fine-tuned according to instructions to obtain model F. Model F is a rewritten model of stock selection criteria obtained by fine-tuning the qwen model. The input is the user's question, and the output is the rewritten and standardized stock selection criteria. For example, if the user inputs: "Select leading stocks with a price greater than 10 yuan and a volume ratio less than 4," the model output would be: "[Closing price > 10, Volume ratio < 4, Industry leader]".

[0101] Deploy the model as a service and test it with sampled data.

[0102] The present invention also provides a stock selection condition splitting and rewriting system, which can be implemented by executing the process steps of the stock selection condition splitting and rewriting method. That is, those skilled in the art can understand the stock selection condition splitting and rewriting method as a preferred embodiment of the stock selection condition splitting and rewriting system.

[0103] Those skilled in the art will understand that, in addition to implementing the system, apparatus, and their modules provided by this invention in purely computer-readable program code, the same program can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system, apparatus, and their modules provided by this invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; alternatively, modules for implementing various functions can be considered both software programs implementing the method and structures within the hardware component.

[0104] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A method for splitting and rewriting stock selection criteria, characterized in that, include: Step S1: Obtain text T containing only user questions, and preprocess the obtained text T. Based on the stock selection factor library F, obtain a set of factors J that meet the preset requirements for similarity with text T, thereby obtaining the labeled dataset. Step S2: Expand the labeled dataset according to preset requirements to obtain an expanded labeled dataset; Step S3: Merge the labeled dataset with the augmented labeled dataset, and preprocess the merged labeled dataset to obtain the preprocessed labeled dataset; Step S4: Train the rewriting model using the preprocessed labeled dataset to obtain the trained rewriting model, and use the trained rewriting model to decompose and rewrite the stock selection conditions. The rewriting model is based on the qwen model to split and rewrite user question text according to the stock selection factor library; Step S1 includes: Step S1.1: Export each interaction record R from the online log of the stock selection system, clean and preprocess each exported interaction record R, and extract the text T containing only the user's question. Step S1.2: Perform preprocessing on the text T, including sentence segmentation, stop word removal, and sentence vector encoding, to obtain the preprocessed text T; Step S1.3: Based on the stock selection factor library F, obtain the factor candidate set S related to the preprocessed text T through vector retrieval or a recall method based on preset rules; Step S1.4: Input the text T and the factor candidate set S into the large language model so that the large language model can select the factor set J from the factor candidate set S that meets the preset requirements for similarity with the text T, and obtain the labeled dataset; Step S2 includes: Step S2.1: Obtain the set M of labeled questions containing multiple factors in the labeled dataset; Step S2.2: Divide the stock selection factor library F into a set of factors with parameters A and a set of factors without parameters B; Step S2.3: Expand the labeled question set M based on the parameterized factor set A; Step S2.4: Expand the labeled question set M based on the set of unparalleled factors B; Step S2.5: Expand the question according to the actual application scenario and generate corresponding tags; Step S2.3 includes: Step S2.3.1: Select the parameterized factor P based on the parameterized factor set A; Step S2.3.2: Randomly sample from the labeled question set M and combine with the parameterized factor P to generate a fusion sentence that conforms to the style of the labeled question set M through a large language model, and generate the corresponding structured annotation; Step S2.3.3: Based on preset rules, perform parameter replacement on the parameterized factor P to generate regularized sentences and automatically generate annotations; Step S2.3.4: The sentences obtained from rule expansion generate similar sentences through a large language model, and annotations are automatically generated; Step S2.4 includes: Step S2.4.1: Select a parameterless factor N based on the parameterless factor set B; Step S2.4.2: Expand similar short sentences using a large language model and automatically generate annotations for the unparalleled factor N; Step S3 includes: merging the labeled dataset and the expanded labeled dataset according to a preset ratio, and performing deduplication and sensitive sample oversampling processing on the merged dataset to obtain the preprocessed labeled dataset.

2. A stock selection criterion splitting and rewriting system, characterized in that, include: Module M1: Obtains text T containing only user questions, preprocesses the obtained text T, and obtains a set of factors J with similarity to text T that meet the preset requirements based on the stock selection factor library F, thereby obtaining the labeled dataset; Module M2: Expands the labeled dataset according to preset requirements to obtain an expanded labeled dataset; Module M3: Merges the labeled dataset with the augmented labeled dataset and preprocesses the merged labeled dataset to obtain the preprocessed labeled dataset; Module M4: Trains the rewriting model using the preprocessed labeled dataset to obtain the trained rewriting model, and uses the trained rewriting model to decompose and rewrite the stock selection conditions. The rewriting model is based on the qwen model to split and rewrite user question text according to the stock selection factor library; The module M1 includes: Module M1.1: Export each interaction record R from the online logs of the stock selection system, perform cleaning and preprocessing on each exported interaction record R, and extract the text T containing only user questions; Module M1.2: Performs preprocessing on text T, including sentence segmentation, stop word removal, and sentence vector encoding, to obtain preprocessed text T; Module M1.3: Based on the stock selection factor library F, obtain the set of factor candidates S related to the preprocessed text T through vector retrieval or a recall method based on preset rules; Module M1.4: Input the text T and the factor candidate set S into the large language model so that the large language model can select the factor set J from the factor candidate set S that meets the preset requirements of similarity with the text T, and obtain the labeled dataset; The module M2 includes: Module M2.1: Obtains a set M of labeled questions containing multiple factors from the labeled dataset; Module M2.2: Divide the stock selection factor library F into a set of factors with parameters A and a set of factors without parameters B; Module M2.3: Expands the labeled question set M based on the parameterized factor set A; Module M2.4: Expands the labeled question set M based on the set of unparalleled factors B; The module M2.3 includes: Module M2.3.1: Selecting a parameterized factor P based on a set of parameterized factors A; Module M2.3.2: Randomly samples from the labeled question set M and combines it with a parameterized factor P to generate a fused sentence that conforms to the style of the labeled question set M through a large language model, and generates the corresponding structured annotations; Module M2.3.3: Generates regularized sentences by replacing parameters of the parameterized factor P according to preset rules, and automatically generates annotations; Module M2.3.4: Sentences obtained from rule expansion generate similar sentences through a large language model and automatically generate annotations; The module M2.4 includes: Module M2.4.1: Selecting a parameterless factor N based on the parameterless factor set B; Module M2.4.2: Expands similar short sentences using a large language model without parameter factor N and automatically generates annotations; The module M3 includes: merging the labeled dataset and the expanded labeled dataset according to a preset ratio, and performing deduplication and sensitive sample oversampling processing on the merged dataset to obtain the preprocessed labeled dataset.

Images