Corpus feature extraction method, device, storage medium and program product

Abstract

This application provides a method, device, storage medium, and program product for extracting corpus features. The method includes: selecting at least one set of corpus sample pairs from a corpus dataset used for corpus application tasks; for each set of corpus sample pairs, analyzing the differences between the feedback corpus of positive corpus samples and the feedback corpus of negative corpus samples under various conditions based on a large language model to determine the corresponding sample pair difference features; and determining the corpus features for the corpus application task based on the sample pair difference features corresponding to each set of corpus sample pairs. Thus, by automatically selecting positive and negative sample pairs from the corpus dataset and automatically analyzing the differences between positive and negative samples under specific conditions based on a large language model, the powerful language cognitive ability of the large language model can be utilized to fully understand and abstract the corpus features of semantic information, enabling rapid adaptation to different tasks or datasets.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, device, storage medium and program product for extracting features from a corpus. Background Technology

[0002] In today's digital age, corpus data is growing explosively, and corpus feature extraction plays a crucial role in many natural language processing tasks, such as text classification, information retrieval, and sentiment analysis. Accurate and efficient extraction of corpus features can significantly improve the performance and effectiveness of these tasks.

[0003] Currently, corpus feature extraction methods typically include expert feature extraction, text semantic feature extraction, and language model-based embedding feature extraction. Expert feature extraction relies on extensive human experience and heuristics specific to particular tasks, designing different features for different cognitive tasks, such as the frequency of interrogative words, the frequency of high-level vocabulary, and keyword extraction counting. However, these features are highly correlated with the dataset organization and task objectives, and the feature design process is heavily dependent on human experience, resulting in high development costs and time consumption, making it difficult to quickly adapt to new cognitive tasks, tests, or dataset changes. Furthermore, the strong subjectivity of feature selection can lead to overfitting.

[0004] In text semantic feature extraction, NLP (Natural Language Processing) algorithms are used to analyze sentence structure, vocabulary usage, and contextual relationships to identify statistical quantities such as lexical category labels and lexical category proportions. Text semantic feature extraction can understand the logic and connections behind text, describing its characteristics from different perspectives such as prosody, grammar, and lexicality. However, the relationship between underlying text features and abstract semantic tasks is complex, and without supervision signals or feature selection methods, it may perform poorly when handling complex semantics and contextual relationships. Furthermore, NLP algorithms are sensitive to text quality; noisy data can affect the accuracy of feature extraction.

[0005] In language model-based embedding feature extraction, pre-trained language models (such as BERT, GPT series, etc.) are used to transform text into vectors in a high-dimensional vector space, i.e., "embedding." The position of each vector reflects the semantic features of words or sentences. Compared with traditional methods, language model-based embedding can better capture complex contextual information, thereby improving the performance of NLP tasks. However, the features it captures are in the form of high-dimensional vector representations, lacking interpretability; in addition, in the absence of effective supervision signals, it can only reflect semantic information, resulting in limited effectiveness on specific tasks.

[0006] Currently, the industry has not proposed a better solution to the above problems. Summary of the Invention

[0007] This application provides a corpus feature extraction method, device, storage medium, and program product to at least solve the problems of strong subjectivity, overfitting, and lack of interpretability in existing feature selection.

[0008] In a first aspect, embodiments of this application provide a corpus feature extraction method, comprising: selecting at least one set of corpus sample pairs from a corpus dataset used for a corpus application task, wherein the corpus sample pairs include positive corpus samples and negative corpus samples; each corpus sample includes feedback corpus for at least one preset conditional corpus and corresponding sample labels, wherein the sample labels are used to indicate whether the corresponding corpus sample is a positive corpus sample or a negative corpus sample; for each set of the corpus sample pairs, analyzing the difference information between the feedback corpus of the positive corpus sample and the feedback corpus of the negative corpus sample for each of the conditional corpus based on a large language model, to determine the corresponding sample pair difference features; and determining the corpus features of the corpus application task based on the sample pair difference features corresponding to each set of corpus sample pairs.

[0009] Secondly, embodiments of this application provide an electronic device, which includes: at least one processor and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the steps of the corpus feature extraction method of any embodiment of this application.

[0010] Thirdly, embodiments of this application provide a storage medium storing a computer program thereon, characterized in that, when the program is executed by a processor, it implements the steps of the corpus feature extraction method of any embodiment of this application.

[0011] Fourthly, embodiments of this application provide a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the corpus feature extraction method of any embodiment of this application.

[0012] The beneficial effects of the embodiments of this application are as follows:

[0013] By automatically selecting positive and negative sample pairs from the corpus dataset and automatically analyzing the differences between positive and negative samples under specific corpus conditions based on large language models, the powerful language cognitive capabilities of large language models can fully understand and abstract the semantic information of the corpus features. This avoids the tedious process of manually designing features, enabling corpus feature extraction to quickly adapt to different tasks or datasets. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 A flowchart illustrating an example of a corpus feature extraction method according to an embodiment of this application is shown;

[0016] Figure 2 It shows according to Figure 1 An example operation flowchart of step S120 in the process;

[0017] Figure 3 The diagram illustrates an example of an automatic feature extraction method for spontaneous language cognitive function based on a large language model according to an embodiment of this application.

[0018] Figure 4 This is a schematic diagram of the structure of an embodiment of the electronic device of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] It should be noted that various feature extraction techniques currently available all have some inherent limitations. Specifically, in task-specific heuristic expert feature extraction techniques, because this method relies on summarizing human experience to guide the feature extraction process, it needs to be re-summarized for different cognitive tasks and dataset formats. In text semantic feature extraction techniques, since NLP methods extract task-independent low-level text features, while the task requires abstract semantic information, the features extracted by this technique often require complex classification models to construct effective supervision signals. Furthermore, calculating low-level text features depends on tasks such as word segmentation and part-of-speech tagging, making it highly sensitive to noise. In language model-based embedding feature extraction techniques, because the pre-trained language model aims to describe semantic information in a high-dimensional vector space, it is impossible to define the meaning of a particular dimension. Additionally, the training objective of the pre-trained language model differs from that of the downstream task, requiring fine-tuning of the model for specific tasks before it can be used.

[0021] Figure 1 A flowchart illustrating an example of a corpus feature extraction method according to an embodiment of this application is shown.

[0022] like Figure 1 As shown, in step S110, at least one set of corpus sample pairs is selected from the corpus dataset used for the corpus application task. The corpus sample pairs include positive corpus samples and negative corpus samples.

[0023] Here, each corpus sample contains feedback corpus for at least one preset conditional corpus and corresponding sample labels, which are used to indicate whether the corresponding corpus sample is a positive corpus sample or a negative corpus sample.

[0024] It should be understood that the corpus application tasks can be diverse, such as text classification, knowledge-based question answering, and sentiment analysis, etc., and are not limited here. Furthermore, the form of the corpus dataset can be diverse, such as corpus datasets maintained by third-party platforms, or corpus datasets constructed according to actual business needs, etc., all of which fall within the scope of the embodiments of this application. For example, the text samples in the corpus dataset need to be predefined or labeled as positive and negative samples according to the needs of the application task. Furthermore, positive and negative corpus samples refer to positive or negative feedback corpora for one or more specific conditional corpora. For example, positive corpus samples are texts that meet the conditions of the target task (e.g., "normal sentiment" text in sentiment analysis), while negative corpus samples are texts that do not meet the conditions of the target task (e.g., "abnormal sentiment" text).

[0025] In step S120, for each group of corpus sample pairs, the difference information of the feedback corpus of positive corpus samples and the feedback corpus of negative corpus samples in each corpus sample pair for each conditional corpus is analyzed based on the large language model to determine the corresponding sample pair difference features.

[0026] Here, the types of large language models can be diverse, such as the GPT series, Qwen, etc., and no restrictions are imposed here. Specifically, large language models can conduct in-depth comparisons of positive and negative corpus samples in their contextual relationships with conditional corpora, such as comparisons of word frequency, semantic association strength, grammatical structure, and sentiment tendencies. Thanks to the spontaneous language cognition capabilities of large language models, they can effectively analyze the differences between corpus sample pairs under specific conditions and derive the difference features of the sample pairs.

[0027] In one example of this application's embodiments, the large language model can be a task-fine-tuned large language model. When the large language model receives corpus sample pairs and various conditional corpora, it can automatically analyze the feedback difference information of positive and negative samples when facing the conditional corpora to obtain the corresponding sample pair difference features. In another example of this application's embodiments, the large language model can also be a general-purpose large language model, which, by constructing specific task prompt words, triggers the large language model to analyze the feedback difference information of positive and negative samples when facing the conditional corpora to obtain the corresponding sample pair difference features.

[0028] It should be understood that since the number of conditional corpora can be multiple, the difference information exhibited by positive and negative samples on different conditional corpora will also be diverse. Therefore, the difference features of sample pairs determined based on a single set of corpus samples can correspond to the difference information distribution of multiple conditional dimensions. Furthermore, thanks to the powerful language cognitive ability of large language models, they can better summarize the difference analysis process and the output logic of the corresponding sample pair difference features, thereby providing the specific meaning of the features and making the features highly interpretable.

[0029] In step S130, the corpus features for the corpus application task are determined based on the sample pair difference features corresponding to each group of corpus sample pairs.

[0030] In one example of this application's embodiments, the difference features of each sample pair can be directly used as corpus features, or the difference features of each group of corpus sample pairs can be integrated and filtered, for example, by removing duplicate or similar difference features, thereby obtaining corpus features. In another example of this application's embodiments, statistical analysis can be performed on the difference features of all sample pairs to remove redundant or noisy features, and the most effective set of corpus features for the corpus application task can be selected through methods such as L1 regularization and information gain.

[0031] As a preferred embodiment of this application, before step S130, the frequency of occurrence of the differential features of each sample can be counted, and the differential features of samples whose corresponding feature occurrence frequency is lower than a preset frequency threshold can be filtered out.

[0032] It should be noted that during feature extraction from the corpus dataset, some difference features may appear frequently in different sample pairs, while others may only appear in a few sample pairs. To ensure the final corpus features are highly representative, the frequency of each difference feature across all sample pairs is first statistically analyzed and then compared with a preset frequency threshold. If the frequency of a feature is below this threshold, it is considered to contribute little to the final task and is therefore filtered out.

[0033] Here, frequency rules are used to remove features that appear too infrequently in the original features. For example, experiments were conducted on the Chinese Mini-MoCA dataset, which has 6806 original features (i.e., sample-pair difference features). Each feature appears an average of 5.45 times, and 788 features appear only once. These features that appear only once can be removed. This improves feature accuracy and also eliminates errors caused by the illusion of a large language model.

[0034] Through the embodiments of this application, frequency statistics and filtering can effectively remove low-frequency differential features. These low-frequency features are often generated by sporadic events or noisy data, which may be irrelevant to the task objective and may even have a misleading impact. By filtering out these noisy features, the quality of subsequent clustering processes can be significantly improved, ensuring that the features reflected in the clustering results are effective features that are highly relevant to the task. In addition, by filtering low-frequency differential features, the final feature set will be more focused on core information, reducing the interference of redundant and irrelevant features. At the same time, high-frequency differential features are often features with high influence in the task, and retaining these features helps improve task performance.

[0035] Regarding the implementation details of step S110 above, in some embodiments, random matching of samples is performed in the corpus dataset based on sample labels to select at least one set of corpus sample pairs. For example, the corpus dataset contains an equal number of positive and negative samples, and random matching of positive and negative samples is performed based on labels to obtain multiple sets of corpus sample pairs.

[0036] Figure 2 It shows according to Figure 1 An example operation flowchart of step S120 in the process.

[0037] like Figure 2 As shown, in step S210, the feedback data of the positive data sample, the feedback data of the negative data sample, and each conditional data are filled into the preset feedback difference analysis task prompt template to determine the corresponding first prompt word.

[0038] In some implementations, the feedback difference analysis task prompt template can be pre-designed. By reserving placeholders for feedback corpora for positive and negative samples and various conditional corpora in the prompt template, these elements are filled into the template in an appropriate format, such as "Please compare the distribution of feedback difference information between __ in the positive corpus sample and __ in the negative corpus sample under the conditional corpus group {___}", thereby guiding the analysis of the differences in feedback corpora of positive and negative corpus samples for the conditional corpus.

[0039] In step S220, the first prompt word is input into the large language model to analyze the feedback difference information between the feedback corpus of positive corpus samples and the feedback corpus of negative corpus samples when processing various conditional corpora.

[0040] It should be noted that large language models possess strong deep semantic understanding capabilities. They can effectively analyze the feedback differences between positive and negative corpora when processing corpora under the same conditions, such as differences in grammatical structure, semantic relevance, fluency, and word selection. This allows them to extract key semantic differences between positive and negative samples under different conditions. For example, the model might analyze the differences between positive and negative comments in sentiment analysis tasks when processing various sentiment conditions, or the feedback differences between positive and negative samples when faced with a specific topic tag in text classification tasks.

[0041] In step S230, each feedback difference information is filled into a preset feedback difference summary template to determine the corresponding second prompt word.

[0042] In some implementations, the feedback difference summary template can be pre-designed. By reserving placeholders for each feedback difference information in the prompt template, the feedback difference information obtained during the feedback difference analysis process can be better filled into the template. For example, "The feedback difference information between positive and negative samples corresponding to each conditional corpus is ___. Based on the above distribution of difference information, please summarize the characteristic variables that can clearly distinguish between positive and negative corpus samples and their corresponding meanings," thereby guiding the extraction of the core feature differences between positive and negative corpus samples.

[0043] In step S240, the second prompt word is input into the large language model to summarize the conditional-feedback difference features and feature definitions corresponding to each feedback difference information, and obtain the corresponding sample pair difference features.

[0044] Here, the large language model further abstracts and generates features for the feedback differences under each conditional corpus, outputting the correlation features between the conditions and feedback differences, such as changes in sentiment intensity, semantic tendencies, and differences in grammatical structure, and then summarizes the final sample pair difference features. Through clear feature definitions, it can achieve interpretability for decision-makers.

[0045] Regarding the implementation details of step S130 above, in some examples of embodiments of this application, the differential features of the corresponding sample pairs of each group of corpus samples are clustered, and the corpus features of the corresponding corpus application task are determined based on the clustering results. Therefore, by clustering the differential features of each group of corpus sample pairs, potential common features are mined, thereby improving the accuracy and robustness of corpus feature extraction.

[0046] It should be noted that the differences in features between different corpus sample pairs may exhibit similarities or certain inherent commonalities. Cluster analysis can help identify these potential commonalities, thereby discovering highly representative task features. Furthermore, clustering helps extract fewer representative features from a large number of differences, reducing model complexity and computational overhead, while avoiding overfitting and redundant feature problems.

[0047] In some examples of embodiments of this application, the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm is used to cluster the sample pair difference features corresponding to each group of corpus samples to determine multiple corresponding clusters. Then, based on the sample pair difference features closest to the cluster center of each cluster, the corpus features for the corpus application task are determined.

[0048] It should be noted that DBSCAN, as a density-based clustering algorithm, can effectively identify sample pairs with similar densities. It can divide data points into different clusters. For example, if the density of data points in a region exceeds a certain threshold, these points will be divided into a cluster.

[0049] For example, key parameters of DBSCAN include ε and MinPts. ε represents the maximum radius of a cluster, which is the maximum distance that two samples must have to be considered to belong to the same cluster. MinPts represents the minimum number of samples within each cluster, determining the minimum density of the cluster. DBSCAN does not require pre-specifying the number of clusters and can effectively cluster sample pairs with similar density features into clusters.

[0050] For each cluster, calculate its cluster center, which is the mean vector of all differential feature vectors in that cluster. Within each cluster, select the sample pair of differential features that is closest to the cluster center. This sample pair of differential features will be considered the "most representative feature" of that cluster, representing the core corpus features of that cluster. Here, distance metrics such as Euclidean distance, Manhattan distance, or cosine similarity can be used.

[0051] Through the embodiments of this application, DBSCAN, using density clustering, can not only find clustering relationships between differential features of corpus sample pairs, but also effectively discover feature differences with density similarity. For example, the differential features of certain corpus sample pairs may appear as high-density groups in sentiment analysis tasks. DBSCAN can identify and label these groups, helping to focus on task-relevant features. This avoids the shortcomings of traditional feature selection methods based on distance or manual rules, and can automatically discover effective feature differences from large-scale data. Furthermore, DBSCAN does not require a preset number of clusters; it dynamically divides clusters according to data density. Whether it's a small-scale corpus or a large-scale, massive dataset, the adaptive clustering method enables the solution to handle corpus data of different sizes and properties.

[0052] Regarding the implementation details of step S130, in some examples of embodiments of this application, the sample pair difference features corresponding to each group of corpus sample pairs are filled into a preset text summarization task prompt template to determine the corresponding third prompt word. Then, the third prompt word is input into a large language model to perform text summarization of the difference features of each sample pair, thereby obtaining the corpus features for the corpus application task.

[0053] In some implementations, a pre-defined prompt template for triggering a text summarization task for a large language model enables the model to better extract corpus features by summarizing the differences in features between each sample pair. For example, the prompt template for the text summarization task could be expressed as: "The following are the differences in features between each sample pair: ____; Please summarize the above differences in features, extract the features most important to the corpus task, and provide the corresponding meanings of these features." This clear and structured prompt format helps the language model focus on feature summarization and extract the core information relevant to the task.

[0054] Through the embodiments of this application, a large language model is instructed to perform a text summarization task by using prompts to summarize all the final corpus features, which can more accurately understand the semantics of the features. However, it should be noted that in the process of directly summarizing corpus features by the large language model, the output results may be affected by outliers because the frequency information of the original features is lost.

[0055] Figure 3 The diagram illustrates an example of an operation flow diagram of a method for automatically extracting key features of spontaneous language cognitive function based on a large language model according to an embodiment of this application.

[0056] like Figure 3As shown, the method for automatic extraction of key features of spontaneous language cognitive function based on large language model includes the following three stages: stage 1—pairing of positive and negative sample samples, stage 2—difference analysis of large language model, and stage 3—hierarchical feature aggregation.

[0057] The following section will use a cognitive function screening task as an example of a corpus application task to elaborate on the details of corpus feature extraction in the embodiments of this application. Specifically, each sample comes from a test subject's answers to a cognitive function screening, including the test subject's text records of answers to multiple questions in the screening, and a label indicating whether the test subject has a cognitive impairment.

[0058] In scenarios where test subjects participate in cognitive function screening, each sample in the dataset contains multiple texts, recording the test subject's answers to multiple questions in the test, and a label that records whether the test subject has cognitive impairment.

[0059] Taking the Mini-MoCA test as an example, in this test, the test taker needs to answer the following 7 questions (i.e., conditional corpus):

[0060] 1. Tell me what year it is now.

[0061] 2. Tell me what month it is now.

[0062] 3. Tell me the exact date today.

[0063] 4. Tell me what day of the week it is today.

[0064] 5. Tell me what this place is and which city it is in.

[0065] 6. Please name as many animals as you know as possible, as quickly as possible. You have one minute. Think about it. Are you ready? Begin.

[0066] 7. I just read you a few words to help you remember them. Please try to recall them and tell me what they are.

[0067] It should be understood that cognitive function screening involves various tasks, and the embodiments of this application should not be limited to the Mini-MoCA test alone. Furthermore, corpus application tasks also have various application directions and should not be limited to cognitive function screening applications. The related descriptions are only used to help the public better understand the solutions in this application and are intended as examples, not as limitations on this application.

[0068] In Phase 1, positive and negative samples from the dataset are randomly matched 1:1 based on the labels, and the two matched samples are placed into a paired dataset. This is to highlight the differences between positive and negative samples.

[0069] In Phase Two, a large language model is used, employing manually designed prompts to guide the LLM (e.g., QWen2.5-72B-instruction) in analyzing and summarizing paired data. First, the large language model analyzes the differences for each question in the paired data, outputting the difference features between positive and negative samples, including feature definitions and their specific manifestations in the paired data. Next, after analyzing all questions in a paired dataset, the large language model summarizes the features. This is because paired data may contain multiple questions, and their difference features may overlap. Ultimately, for each paired dataset, several raw features are obtained, including their definitions and meanings.

[0070] Specifically, the following are examples of prompts used in the analysis process during Phase Two:

[0071] # Role

[0072] Neuropsychological test analyst

[0073] # Profile

[0074] You are a professional neuropsychological testing analyst, specializing in assessing and analyzing individual performance differences in cognitive function tests, and able to discover key cognitive function differences from the details.

[0075] # Background

[0076] It is necessary to analyze the responses of two test subjects to a series of questions to identify and compare the differences in their answers, in order to find certain features to distinguish between test subjects with and without the disease. These features include, but are not limited to, characteristics related to language, grammar, vocabulary, prosody, and morphology. It is also necessary to discover and define new features.

[0077] # Skills

[0078] You possess professional knowledge of neuropsychology and linguistics, enabling you to identify differences among test takers by analyzing their responses.

[0079] # Goals

[0080] Analyze the responses of the two test takers in detail, identifying as many differences as possible.

[0081] # Constrains

[0082] 1. Please note that the analysis should be based on the content and format of the responses, avoiding judgments about personal identities, and ensuring the objectivity and scientific rigor of the analysis.

[0083] 2. Please note that there is no need to determine whether the specific time, date, or location matches the actual situation.

[0084] 3. Please note that these problems may occur in people with normal cognition, and not all patients with cognitive impairment will exhibit all of these characteristics.

[0085] 4. Please note that this assessment is not intended to provide a formal diagnosis, but rather serves as a preliminary screening tool for potential MCI.

[0086] # Format

[0087] ## Input Format

[0088] 1. The input will be provided to you in the following format: {{"question": " <question>", "record":{{"tester1": "<tester1 record> ", "tester2": "<tester2 record> "}}}}

[0089] 2. The input includes the following information: the problem faced by the tester. <question>Tester 1's answers<tester1 record> Tester 2's answers<tester2 record>

[0090] ## Output Format

[0091] 1. First, summarize the differences between the two test takers' answers in one paragraph.

[0092] 2. Next, for each feature that produces a difference, explain its meaning and describe the difference in the two test takers' answers under this feature.

[0093] # Workflow

[0094] 1. Read carefully the questions the test takers faced and their answers, and record the answers of both test takers for each question.

[0095] 2. Compare the two test takers' answers to each question, analyze the differences, and propose features that make the two test takers' answers significantly different under these features.

[0096] 3. For each feature, explain its meaning and describe the differences in the two test takers' answers under this feature.

[0097] 4. Strictly adhere to the Output Format when outputting your answer.

[0098] # Input

[0099] {input}

[0100] # Output

[0101] Input: {input} is replaced with a record of a question.

[0102] {

[0103] "question": "I just read you a few words to help you remember them. Please try to recall them and tell me what they are."

[0104] "record": {

[0105] "tester1": "Can you remember those words he just taught you to memorize? If you can't, try memorizing them and see if you can remember any of them. He asked if you can still recall them."

[0106] "tester2": "Red chrysanthemum at the Face Hotel. Tiger, and what's that cotton lipstick color? The Chrysanthemum Hotel is out of stock, and I don't have any others. Hmm."

[0107] }

[0108] Output: Summarize the differences and list each difference feature.

[0109] {

[0110] "id_question": 18,

[0111] "question": "I just read you a few words to help you remember them. Please try to recall them and tell me what they are."

[0112] "content": "### Summary\nTester 1 and Tester 2 showed significant differences in their answers to the vocabulary recall question. Tester 1's answers were relatively coherent; they attempted to recall the words but ultimately stated they couldn't remember them. Tester 2's answers were disorganized; the words provided were irrelevant to the question, and there were repetitions and interruptions in their expression.\n\n### Analysis of Difference Characteristics\n\n1. **Coherence**\n - **Meaning**: Coherence refers to whether the answer is logically clear, relevant, and effectively communicates the question.\n - **Difference**:\n - **Tester 1**: Although the answer did not completely recall the words, it maintained overall logical coherence and attempted to recall them related to the question.\n - **Tester 2**: The answer was irrelevant to the question, with multiple instances of deviating from the topic, such as mentioning "Old Pig also has some lipstick," indicating a lack of coherence in their answer.\n\n2. **Vocabulary Selection**\n - **Meaning**: Vocabulary selection refers to whether the words used in the answer are appropriate and accurate.\n - **Difference**:\n - **Tester 1:** Used relatively common vocabulary, such as "remember" and "can't recall," indicating appropriate word choice. **Tester 2:** Used irrelevant words, such as "speaker" and "lipstick," indicating impaired vocabulary selection ability. **3. **Grammatical Structure** **Meaning**:** Grammatical structure refers to the correctness and completeness of sentence structure in the answer. **Difference**: **Tester 1:** Sentence structure is relatively complete; although there are interrogative sentences, it generally conforms to grammatical rules. **Tester 2:** Sentence structure is incomplete, with breaks and incoherent expressions, such as "I...I don't have anything else," indicating impaired grammatical structure. **4. **Repetition and Interruption** **Meaning**:** Repetition and interruption refer to whether repeated words or sentences appear in the answer, and whether there are frequent interruptions. **Difference**: **Tester 1:** No obvious repetition or interruption; the answer is relatively fluent. - **Tester 2**: Repeated phrases such as "I...I don't have anything else" appeared multiple times, and interruptions occurred in the response, such as "um.", indicating impaired fluency. \n\n5. **Content Relevance**\n - **Meaning**: Content relevance refers to whether the answer is directly related to the question. \n - **Differences**:\n - **Tester 1**: The answer was relevant to the question; although the words could not be fully recalled, the tester attempted to remember them. \n - **Tester 2**: The answer was irrelevant to the question, mentioning words unrelated to the question, such as "speaker" and "lipstick," indicating impaired content relevance.Based on the above analysis, Tester 2 exhibited significant cognitive impairment across multiple characteristics, while Tester 1, although unable to fully recall the words, maintained relatively good cognitive function overall. These differences can serve as a reference for further assessment of potential MCI.

[0113] }

[0114] Here is an example of a prompt in the summary operation of Phase Two:

[0115] prompt:

[0116] # Role

[0117] Cognitive function analysis expert

[0118] # Profile

[0119] You are a neuropsychologist specializing in cognitive impairment, with in-depth understanding and research on the symptoms and manifestations of cognitive impairment, and are able to accurately identify the characteristics of patients' responses.

[0120] # Background

[0121] It's necessary to distinguish which of the two answers likely comes from a patient with cognitive impairment. These patients typically exhibit symptoms of cognitive dysfunction such as memory loss, orientation, judgment, calculation difficulties, abstract thinking, and language impairment, and may also experience psychiatric symptoms such as personality or behavioral changes, delusions, or hallucinations.

[0122] # Skills

[0123] You possess professional psychological assessment skills, enabling you to analyze and identify signs of cognitive impairment in your responses.

[0124] # Goals

[0125] Based on the responses of the two test subjects and the analysis of the analyst, it was accurately identified which test subject was more likely to have cognitive impairment.

[0126] # Constrains

[0127] 1. Please note that the analysis should be based on the content and format of the responses, avoiding judgments about personal identities, and ensuring the objectivity and scientific rigor of the analysis.

[0128] 2. Please note that there is no need to determine whether the specific date and location are correct.

[0129] 3. Please note that these issues can occur in cognitively normal individuals, and not all patients with cognitive impairment will exhibit all of these characteristics. The final classification should be based on a comprehensive assessment of all indicators.

[0130] 4. Please note that this assessment is not intended to provide a formal diagnosis, but rather serves as a preliminary screening tool for potential MCI.

[0131] # Format

[0132] ## Input Format

[0133] 1. The input is a list, where each element is a question answered by the test taker, along with the test taker's answer and the analyst's analysis record.

[0134] 1. The input will be provided to you in the following format: "[{{"question": <question>, "record":{{"tester1":<tester1 record>, "tester2":<tester2 record>}}, "analysis": <analysis>}}...]"

[0135] 2. The input includes the following information: the problem faced by the tester. <question>Tester 1's answers<tester1 record> Tester 2's answers<tester2 record> Analysts analyzed the differences in the two test takers' answers to the current question. <analysis>

[0136] ## Output Format

[0137] 1. First, state your diagnosis (1 represents that test subject 1 is more likely to have cognitive impairment, 2 represents that test subject 2 is more likely to have cognitive impairment), then summarize your reasoning, and finally, for each differential characteristic, give a definition and explanation.

[0138] 2. Output format: "{{"diagnosis":<diagnosis_1_or_2> "reason": <reason>"features": [{{"feature":<feature_i> "explanation":<explanation_i> ...}}]}}"

[0139] 3. The output includes the following information: your final judgment on which test subject is more likely to have cognitive impairment.<diagnosis_1_or_2> Summarize your judgment criteria. <reason>You summarized the distinguishing features between patients and normal people.<feature_i> Explanation of this feature<explanation_i> .

[0140] # Workflow

[0141] 1. Carefully read the content of the questions under different questions in the input, the answers of the two test takers under different questions, and the differences in characteristics proposed by the analyst.

[0142] 2. Identify possible signs of cognitive impairment in the responses, such as memory errors, logical confusion, and decreased abstract thinking ability.

[0143] 3. Based on the analysis results of the sub-questions, analyze which answer is more likely to come from a patient with cognitive impairment, and provide reasons.

[0144] 4. Summarize the distinguishing features between patients and healthy individuals. For each feature, explain its meaning and describe the differences in the two test subjects' responses to that feature.

[0145] 5. Strictly adhere to the Output Format when outputting your answer.

[0146] # Input

[0147] {input}

[0148] # Output

[0149] Input: {input} Replace with the results of all stage two analyses for this sample.

[0150] Output: Determine which of the two testers is the positive sample; summarize all features, including definitions and explanations.

[0151] {

[0152] "diagnosis": 1,

[0153] "Reason": "Test subject 1 exhibited signs of cognitive impairment in their responses to multiple questions, including a blurred sense of time, repetitive questioning, poor language organization, limited vocabulary, and simple, incoherent sentence structure. In contrast, test subject 2, while also showing some repetition, performed better in areas such as language expression and logical thinking."

[0154] "features": [

[0155] {

[0156] "feature": "fuzzy time concept",

[0157] "explanation": "Test taker 1 showed significant uncertainty and made errors when answering questions about the year and month, such as repeatedly saying 'He asked, he asked,' indicating that he was unclear about the current year and month."

[0158] },

[0159] {

[0160] "feature": "repeated question",

[0161] "Explanation": "Test subject 1 repeatedly asked the same questions, such as 'He asked you what year it is now?', indicating a decline in memory and comprehension abilities."

[0162] },

[0163] {

[0164] "feature": "poor language organization skills",

[0165] "explanation": "Test taker 1's responses contained repeated use of the same phrases, such as 'no, no, no,' indicating poor language organization skills."

[0166] },

[0167] {

[0168] "feature": "poor vocabulary"

[0169] "explanation": "Test taker 1 used a limited vocabulary, repeatedly using simple words such as 'he asked,' indicating a limited vocabulary."

[0170] },

[0171] {

[0172] "feature": "The statement structure is simple and disjointed".

[0173] "explanation": "Test taker 1's responses contained simple and incoherent sentences, such as 'this this this,' indicating weak language expression skills."

[0174] } ]

[0176] }

[0177] In Phase 3, all the original features are aggregated, and the DBSCAN clustering algorithm is used, which clusters based on the density distribution of the data points. Specifically, DBSCAN relies on two parameters: ϵ (epsilon) and minPts. For a given point, if it contains at least minPts points within its ϵ radius, the point is labeled as a corepoint. If a point is a corepoint and other points fall within its ϵ radius, these points are considered density-reachable. Density-connected clusters are formed by recursively visiting the neighborhood of each corepoint. Non-core and isolated points are labeled as noise.

[0178] For example, the DBSCAN clustering method uses parameters ϵ=0.04 and minPts=1, employing cosine similarity of vectors as the distance metric, ultimately producing 50 clusters. Within each cluster, the mean of all keypoints is calculated as the cluster center, and the feature closest to the cluster center is selected as the final output feature set. The features in the output feature set can then comprehensively and accurately reflect the differences between positive and negative samples, thus characterizing cognitive function.

[0179] In this embodiment, the method of randomly organizing 1:1 positive and negative example pairs, analyzing using a large language model, and aggregating hierarchical features breaks away from the traditional approach of manually designing task-related features or designing classification models to generate supervisory signals for general features. This method can extract features with semantic meaning. Furthermore, automatic feature extraction can be performed using the above approach when facing different tasks and dataset formats.

[0180] Through the embodiments of this application, features with high interpretability can be automatically summarized. Specifically, a large language model is first used for diversified analysis, and then the differences between all positive and negative samples are summarized. This allows the large language model to focus on high-level features such as language organization ability, vocabulary ability, cognition and memory ability, rather than low-level statistical features such as word frequency and part-of-speech ratio. This makes the features extracted by the method provided in the embodiments of this application highly interpretable.

[0181] It should be noted that current feature extraction methods often face the challenge of a lack of interpretability in features extracted by neural networks, while features with rigorous definitions and interpretability heavily rely on expert experience, requiring numerous manual steps. If key features could be automatically extracted from datasets and these features were interpretable, this contradiction could be resolved to some extent. This would allow for a more intelligent and automated discovery of key features reflecting cognitive functions, further changing society's understanding of cognitive functions.

[0182] Furthermore, the feature extraction process in this application can be directly applied to open-source or closed-source general-purpose large language models without the need for fine-tuning, thus reducing the cost of model training. Currently, the bottleneck in large language model research lies in the high training cost. If this cost can be reduced, this bottleneck can be overcome to some extent, leading to more intelligent large language models and further changing the way society produces and lives.

[0183] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of combined actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Secondly, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application. In the above embodiments, the descriptions of each embodiment have their own emphasis; for parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0184] In some embodiments, this application provides a non-volatile computer-readable storage medium storing one or more programs including execution instructions, which can be read and executed by electronic devices (including but not limited to computers, servers, or network devices) to perform any of the above-described corpus feature extraction methods of this application.

[0185] In some embodiments, this application also provides a computer program product, the computer program product including a computer program stored on a non-volatile computer-readable storage medium, the computer program including program instructions, which, when executed by a computer, cause the computer to perform any of the above-described corpus feature extraction methods.

[0186] In some embodiments, this application also provides an electronic device, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform a corpus feature extraction method.

[0187] Figure 4 This is a schematic diagram of the hardware structure of an electronic device for performing a corpus feature extraction method according to another embodiment of this application, as shown below. Figure 4 As shown, the device includes:

[0188] One or more processors 410 and memory 420, Figure 4 Take a processor 410 as an example.

[0189] The device for performing the corpus feature extraction method may further include an input device 430 and an output device 440.

[0190] The processor 410, memory 420, input device 430, and output device 440 can be connected via a bus or other means. Figure 4 Taking the example of a connection between China and Israel via a bus.

[0191] The memory 420, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the corpus feature extraction method in the embodiments of this application. The processor 410 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions, and modules stored in the memory 420, thereby implementing the corpus feature extraction method of the above-described method embodiments.

[0192] The memory 420 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the electronic device. Furthermore, the memory 420 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 420 may optionally include memory remotely located relative to the processor 410, and these remote memories can be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0193] Input device 430 can receive input digital or character information and generate signals related to user settings and function control of the electronic device. Output device 440 may include display devices such as a display screen.

[0194] The one or more modules are stored in the memory 420, and when executed by the one or more processors 410, they perform the corpus feature extraction method in any of the above method embodiments.

[0195] The above-described product can perform the methods provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects for performing the methods. Technical details not described in detail in this embodiment can be found in the methods provided in the embodiments of this application.

[0196] The electronic devices in this application embodiments exist in various forms, including but not limited to:

[0197] (1) Mobile communication devices: These devices are characterized by their mobile communication capabilities and primarily aim to provide voice and data communication. These terminals include smartphones, multimedia phones, feature phones, and low-end phones.

[0198] (2) Ultra-mobile personal computer devices: These devices fall under the category of personal computers, possessing computing and processing capabilities, and generally also have mobile internet access features. These terminals include: PDAs, MIDs, and UMPCs, etc.

[0199] (3) Portable entertainment devices: These devices can display and play multimedia content. This category includes audio and video players, handheld game consoles, e-book readers, as well as smart toys and portable car navigation devices.

[0200] (4) Other airborne electronic devices with data interaction capabilities, such as vehicle-mounted systems installed on vehicles.

[0201] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0202] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0203] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.< / reason> < / reason> < / analysis> < / question> < / analysis> < / question> < / question> < / question>

Claims

1. A corpus feature extraction method, comprising: From the corpus dataset used for corpus application tasks, at least one set of corpus sample pairs is selected, the corpus sample pairs containing positive corpus samples and negative corpus samples; Each corpus sample includes feedback corpus for at least one preset conditional corpus and corresponding sample labels, wherein the sample labels are used to indicate whether the corresponding corpus sample is a positive corpus sample or a negative corpus sample; the corpus application task includes a cognitive function screening task. For each group of corpus sample pairs, the difference information between the feedback corpus of positive corpus samples and the feedback corpus of negative corpus samples in each corpus sample pair for each conditional corpus is analyzed based on the large language model to determine the corresponding sample pair difference features. Based on the sample pair difference features corresponding to each group of corpus sample pairs, the corpus features for the corpus application task are determined. Specifically, for each group of corpus sample pairs, the difference information between the feedback corpus of positive corpus samples and the feedback corpus of negative corpus samples in each corpus sample pair for each conditional corpus is analyzed based on a large language model to determine the corresponding sample pair difference features, including: The feedback data of positive corpus samples, the feedback data of negative corpus samples, and each of the aforementioned conditional data are filled into a preset feedback difference analysis task prompt template to determine the corresponding first prompt word; The first prompt word is input into the large language model to analyze the feedback difference information between the feedback corpus of positive corpus samples and the feedback corpus of negative corpus samples when processing corpus under various conditions; the large language model can effectively analyze the feedback difference between positive and negative corpus when processing corpus under the same conditions, thereby extracting the key semantic differences between positive and negative samples under different conditions. Fill each of the feedback difference information into a preset feedback difference summary template to determine the corresponding second prompt word; The second prompt word is input into the large language model to summarize the condition-feedback difference features and feature definitions corresponding to each feedback difference information, and obtain the corresponding sample pair difference features. The large language model further abstracts and generates features for the feedback differences under each conditional corpus, outputs the correlation features between the conditions and feedback differences, and then summarizes the final sample pair difference features. Through clear feature definitions, interpretability for decision-makers can be achieved. The step of determining the corpus features for the corpus application task based on the sample pair difference features corresponding to each group of corpus sample pairs includes: Count the frequency of occurrence of the differential features for each sample, and filter out the differential features from samples whose corresponding feature frequency is lower than a preset frequency threshold; Based on the DBSCAN clustering algorithm, the sample pairs in each group of corpus sample pairs whose corresponding features appear at a frequency not lower than the preset frequency threshold are clustered to determine the corresponding multiple clusters; For each cluster, the cluster center position is calculated, and the difference features of the sample pair closest to the cluster center position are selected from the cluster. The difference features of the selected sample pair are used as the corpus features of the corresponding cluster. The step of determining the corpus features for the corpus application task based on the sample pair difference features corresponding to each group of corpus sample pairs includes: Fill the sample pair difference features corresponding to each group of corpus samples into the preset text summarization task prompt template to determine the corresponding third prompt word; The third prompt word is input into the large language model to summarize the differences in features between each sample and obtain the corpus features for the corpus application task.

2. The method according to claim 1, wherein, The step of selecting at least one pair of corpus samples from the corpus dataset used for corpus application tasks includes: Random matching of samples is performed in the corpus dataset based on the sample labels to select at least one corpus sample pair.

3. A storage medium having a computer program stored thereon, wherein, When the program is executed by the processor, it implements the steps of the method described in claim 1 or 2.

4. An electronic device comprising: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the steps of the method of claim 1 or 2.

5. A computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the method of claim 1 or 2.

Images