Corpus feature extraction method and device, storage medium and program product

Abstract

The invention provides a corpus feature extraction method and device, a storage medium and a program product, and the method comprises the steps: selecting at least one group of corpus sample pairs from a corpus data set for a corpus application task; for each group of corpus sample pairs, analyzing difference information of feedback corpora of positive corpus samples and feedback corpora of negative corpus samples in the corpus sample pairs for each condition corpus based on a large language model to determine corresponding sample pair difference features; and determining corpus features of the corpus application task based on the sample pair difference features corresponding to each group of corpus sample pairs. Therefore, positive and negative sample pairs are automatically selected from a corpus data set, difference features between positive and negative samples in corpus samples under specific condition corpora are automatically analyzed on the basis of large language model analysis, and corpus features of corpus semantic information can be fully understood and abstracted by utilizing the powerful language cognitive ability of a large language model; and different tasks or data sets can be quickly adapted.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a corpus feature extraction method, device, storage medium and program product. Background Art

[0002] In today's digital age, corpus data is growing explosively. Corpus feature extraction plays a vital role in many natural language processing tasks, such as text classification, information retrieval, sentiment analysis, etc. Accurate and efficient extraction of corpus features can significantly improve the performance and effectiveness of these tasks.

[0003] At present, the methods of extracting corpus features usually include expert feature extraction, text semantic feature extraction, and embedded feature extraction based on language models. In the expert feature extraction method, through a large amount of manual experience, heuristic expert feature extraction based on specific tasks is used to design different features for different cognitive function tasks, such as the frequency of question words in the text, the frequency of high-level vocabulary in the text, and the count of keyword extraction. However, these features are highly related to the organization form of the dataset and the task objectives, and the feature design process is highly dependent on manual experience. The development cost is high and time-consuming, and it is difficult to quickly adapt to new cognitive function tasks, tests, or changes in datasets. In addition, the feature selection is highly subjective and may lead to overfitting problems.

[0004] In the text semantic feature extraction method, the sentence structure, vocabulary usage and contextual relationship are analyzed through the NLP (Natural Language Processing) algorithm to identify statistics such as vocabulary category labels and vocabulary category ratios. Text semantic feature extraction can understand the logic and associations behind the text and describe the characteristics of the text from different angles such as rhythm, grammar, and vocabulary. However, the relationship between the underlying text features and the abstract semantic tasks is complex. In the absence of supervision signals or feature screening methods, it may not perform well when processing complex semantics and contextual relationships. In addition, the NLP algorithm is sensitive to text quality, and noisy data may affect the accuracy of feature extraction.

[0005] In the embedding feature extraction method based on language models, pre-trained language models (such as BERT, GPT series, etc.) are used to convert text into vectors in a high-dimensional vector space, namely "embedding". The position of each vector reflects the semantic characteristics of words or sentences. Compared with traditional methods, embedding based on language models can better capture complex contextual information, thereby improving the effect of NLP tasks. However, the features it captures are in the form of high-dimensional vector representations, which lack interpretability; in addition, in the absence of effective supervision signals, it can only reflect semantic information and has limited effect on specific tasks.

[0006] The industry has not yet proposed a better solution to the above problems. Summary of the invention

[0007] The present application provides a corpus feature extraction method, device, storage medium and program product to at least solve the problems of strong subjectivity, overfitting and lack of interpretability of existing feature selection.

[0008] In a first aspect, an embodiment of the present application provides a corpus feature extraction method, comprising: selecting at least one group of corpus sample pairs from a corpus data set used for a corpus application task, the corpus sample pairs comprising positive corpus samples and negative corpus samples; each corpus sample comprises a feedback corpus for at least one preset conditional corpus and a corresponding sample label, the sample label being used to indicate whether the corresponding corpus sample is a positive corpus sample or a negative corpus sample; for each group of the corpus sample pairs, analyzing the difference information of the feedback corpus of the positive corpus sample and the feedback corpus of the negative corpus sample in the corpus sample pair for each of the conditional corpora based on a large language model to determine the corresponding sample pair difference features; based on the sample pair difference features corresponding to each group of the corpus sample pairs, determining the corpus features of the corpus application task.

[0009] In a second aspect, an embodiment of the present application provides an electronic device, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the steps of the corpus feature extraction method of any embodiment of the present application.

[0010] In a third aspect, an embodiment of the present application provides a storage medium on which a computer program is stored, characterized in that when the program is executed by a processor, the steps of the corpus feature extraction method of any embodiment of the present application are implemented.

[0011] In a fourth aspect, an embodiment of the present application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the corpus feature extraction method of any embodiment of the present application.

[0012] The beneficial effects of the embodiments of the present application are: By automatically selecting positive and negative sample pairs from the corpus data set, and automatically analyzing the difference characteristics between positive and negative samples in the corpus samples under specific conditions based on the large language model analysis, the powerful language cognition ability of the large language model can be used to fully understand and abstract the corpus features of the corpus semantic information. In this way, the tedious process of manually designing features is avoided, and the corpus feature extraction can quickly adapt to different tasks or data sets. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0014] Figure 1 A flowchart of an example of a corpus feature extraction method according to an embodiment of the present application is shown; Figure 2 Shown according to Figure 1 An exemplary operation flow chart of step S120 in FIG. Figure 3 A schematic diagram of an operation flow of an example of a method for automatically extracting key features of spontaneous language cognitive functions based on a large language model according to an embodiment of the present application is shown; Figure 4 It is a schematic structural diagram of an embodiment of an electronic device of the present application. DETAILED DESCRIPTION

[0015] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0016] It should be noted that there are some corresponding technical defects in various feature extraction technologies in the current related technologies. Specifically, in the heuristic expert feature extraction technology for specific tasks, since this method needs to summarize manual experience to guide the feature extraction process, it is necessary to re-summarize it in the face of different cognitive function tasks and data set forms. In the text semantic feature extraction technology, since the features extracted by the NLP method are task-independent underlying text features, and the tasks to be solved require abstract semantic information, the features extracted by this technology usually require complex classification models to construct effective supervision signals. In addition, the calculation of underlying text features depends on tasks such as word segmentation and part-of-speech tagging, so it is very sensitive to noise. In the embedded feature extraction technology based on the language model, since the training goal of the pre-trained language model is to describe semantic information in a high-dimensional vector space, it is impossible to define the meaning of a certain dimension. In addition, the training goal of the pre-trained language model is different from the downstream task, which requires the model to be fine-tuned before it can be used on specific tasks.

[0017] Figure 1A flowchart of an example of a corpus feature extraction method according to an embodiment of the present application is shown.

[0018] like Figure 1 As shown, in step S110, at least one set of corpus sample pairs is selected from a corpus data set for a corpus application task, and the corpus sample pairs include positive corpus samples and negative corpus samples.

[0019] Here, each corpus sample includes feedback corpus for at least one preset conditional corpus and a corresponding sample label, and the sample label is used to indicate whether the corresponding corpus sample is a positive corpus sample or a negative corpus sample.

[0020] It should be understood that the corpus application tasks can also be diversified, such as text classification, knowledge questions and answers, and sentiment analysis, etc., which are not limited here. In addition, the form of the corpus data set can be diversified, such as a corpus data set maintained by a third-party platform, or a corpus data set constructed according to actual business needs, etc., and all belong to the implementation scope of the embodiment of the present application. Exemplarily, the text samples in the corpus data set need to be pre-defined or labeled as positive samples and negative samples according to the needs of the application task. In addition, positive corpus samples and negative corpus samples refer to positive feedback corpus or negative feedback corpus for specific one or more conditional corpus corpus, for example, a positive corpus sample is a text that meets the requirements of the target task conditions (such as, "emotionally normal" text in sentiment analysis), and a negative corpus sample is a text that does not meet the requirements of the target task conditions (such as, "emotionally abnormal" text).

[0021] In step S120, for each group of corpus sample pairs, the difference information of the feedback corpus of the positive corpus sample and the feedback corpus of the negative corpus sample in the corpus sample pair for each conditional corpus is analyzed based on the large language model to determine the corresponding sample pair difference features.

[0022] Here, the types of large language models can be diverse, such as the GPT series, Qwen, etc., which are not limited here. Specifically, the large language model can make an in-depth comparison between the positive corpus samples and the negative corpus samples in the contextual relationship with the conditional corpus, such as the comparison of vocabulary usage frequency, semantic association strength, grammatical structure, emotional tendency, etc., and thanks to the spontaneous language cognition ability of the large language model, the large language model can effectively analyze the differences between corpus sample pairs under specific conditions and obtain the difference characteristics of the sample pairs.

[0023] In one example of an embodiment of the present application, the large language model may be a large language model that has been fine-tuned for a task. When the large language model receives a corpus sample pair and each conditional corpus, it can automatically analyze the feedback difference information of the positive and negative samples when facing the conditional corpus to obtain the corresponding sample pair difference features. In another example of an embodiment of the present application, the large language model may also be a general large language model, which triggers the large language model to analyze the feedback difference information of the positive and negative samples when facing the conditional corpus by constructing a specific task prompt word to obtain the corresponding sample pair difference features.

[0024] It should be understood that since there can be multiple conditional corpora, the difference information expressed by positive and negative samples on different conditional corpora will also be diverse. Therefore, the difference features of sample pairs determined based on a single set of corpus sample pairs can be the difference information distribution corresponding to multiple conditional dimensions. In addition, thanks to the powerful language cognition ability of the large language model, it can better summarize the difference analysis process and the output logic of the difference features of the corresponding sample pairs, thereby giving the specific meaning of the features and making the features more interpretable.

[0025] In step S130, based on the sample pair difference features corresponding to each group of corpus sample pairs, the corpus features of the corpus application task are determined.

[0026] In one example of the embodiment of the present application, the difference features of each sample pair can be directly used as corpus features, or the difference features of the sample pairs of each group of corpus sample pairs can be integrated and filtered, for example, repeated or similar difference features can be screened out to obtain corpus features. In another example of the embodiment of the present application, the difference features of all sample pairs can also be statistically analyzed to remove redundant or noise features, and a group of corpus features that are most effective for corpus application tasks can be screened out through L1 regularization, information gain, etc.

[0027] As a preferred implementation of the embodiment of the present application, before step S130, the feature occurrence frequency of each sample pair difference feature can also be counted, and the sample pair difference features whose corresponding feature occurrence frequency is lower than a preset frequency threshold are filtered out.

[0028] It should be noted that in the feature extraction process for the corpus dataset, some sample pair difference features may appear frequently in different sample pairs, while some features only appear in a few sample pairs. In order to ensure that the final corpus features are highly representative, the frequency of occurrence of each difference feature in all sample pairs is first counted, and then compared with the preset frequency threshold. If the frequency of occurrence of a feature is lower than the threshold, it is considered that the feature contributes less to the final task, so it is filtered out.

[0029] Here, by using the frequency rule, we remove features that appear too rarely in the original features. For example, in the experiment on the Chinese Mini-MoCA dataset, there are 6806 original features (i.e., sample pair difference features), each of which appears 5.45 times on average, and there are 788 features that appear only once. These features that appear only once can be removed. In this way, the feature accuracy is higher and the errors caused by the hallucination of the large language model can be eliminated.

[0030] Through the embodiments of the present application, frequency statistics and filtering are performed to effectively remove difference features with low frequency of occurrence. These low-frequency features are often generated by accidental events or noise data, which may have little to do with the task objectives and may even have misleading effects. By filtering out these noise features, the quality of the subsequent clustering process can be significantly improved to ensure that the features reflected in the clustering results are effective features that are highly relevant to the task. In addition, by filtering low-frequency difference features, the feature set that is ultimately retained will be more focused on core information, reducing the interference of redundant and irrelevant features. At the same time, high-frequency difference features are often highly influential features in the task, and retaining these features helps improve the performance of the task.

[0031] Regarding the implementation details of the above step S110, in some implementations, samples are randomly matched in the corpus data set according to the sample labels to select at least one set of corpus sample pairs. Exemplarily, an equal number of positive samples and negative samples are provided in the corpus data set, and positive and negative samples are randomly matched according to the labels to obtain multiple sets of corpus sample pairs.

[0032] Figure 2 Shown according to Figure 1 An example operation flow chart of step S120 in FIG.

[0033] like Figure 2 As shown, in step S210, the feedback corpus of the positive corpus sample, the feedback corpus of the negative corpus sample and each conditional corpus are filled into the preset feedback difference analysis task prompt template to determine the corresponding first prompt word.

[0034] In some embodiments, the prompt template for the feedback difference analysis task can be pre-designed, by reserving placeholders for the feedback corpus and each conditional corpus for the positive and negative samples in the prompt template, and these elements are filled into the template in a suitable formatting manner, such as "Please compare the distribution of feedback difference information between __ in the positive corpus sample and __ in the negative corpus sample under the conditional corpus group {___}", thereby guiding the analysis of the differences in the feedback corpus of the positive and negative corpus samples for the conditional corpus.

[0035] In step S220, the first prompt word is input into the large language model to analyze feedback difference information between the feedback corpus of the positive corpus sample and the feedback corpus of the negative corpus sample when processing each conditional corpus.

[0036] It should be noted that the large language model has a strong ability to understand deep semantics. It can effectively analyze the feedback differences between positive and negative corpora when processing the same conditional corpora, such as grammatical structure, semantic relevance, fluency, vocabulary selection differences, etc., so as to extract the key semantic differences between positive and negative samples when processing different conditions. For example, the model may analyze the differences between positive and negative comments in sentiment analysis tasks when processing various sentiment conditions, or the feedback differences between positive and negative samples when facing a certain topic label in text classification tasks.

[0037] In step S230, each piece of feedback difference information is filled into a preset feedback difference summary template to determine a corresponding second prompt word.

[0038] In some embodiments, the feedback difference summary template may be pre-designed. By reserving placeholders for each piece of feedback difference information in the prompt template, each piece of feedback difference information obtained during the feedback difference analysis process can be better filled into the template, such as "the feedback difference information between the positive and negative samples corresponding to each conditional corpus is ___ respectively. Please summarize the feature variables and corresponding meaning descriptions that can clearly distinguish the positive and negative corpus samples based on the above difference information distribution", thereby guiding the extraction of the core feature differences between the positive and negative corpora.

[0039] In step S240, the second prompt word is input into the large language model to summarize the condition-feedback difference features and feature definitions corresponding to each feedback difference information, and obtain the corresponding sample pair difference features.

[0040] Here, the large language model further abstracts and generates features for the feedback differences under each conditional corpus, and outputs the associated features between the conditions and the feedback differences, such as changes in sentiment intensity, changes in semantic tendencies, differences in grammatical structures, etc., and then summarizes the final sample pair difference features, and achieves interpretability for decision makers through clear feature definitions.

[0041] Regarding the implementation details of the above step S130, in some examples of the embodiments of the present application, the sample pair difference features corresponding to each group of corpus sample pairs are clustered, and the corpus features of the corresponding corpus application task are determined based on the clustering results. Therefore, by clustering the difference features of each group of corpus sample pairs, potential common features are mined to improve the accuracy and robustness of corpus feature extraction.

[0042] It should be noted that the differential features of different corpus sample pairs may have similarities or some inherent commonalities. Cluster analysis can help identify these potential commonalities and thus discover highly representative task features. In addition, clustering helps to extract fewer representative features from a large number of differential features, reducing model complexity and computational overhead, while avoiding overfitting and redundant feature problems.

[0043] In some examples of the embodiments of the present application, the sample pair difference features corresponding to each group of corpus sample pairs are clustered based on the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm to determine multiple corresponding clusters. Furthermore, the corpus features of the corpus application task are determined based on the sample pair difference features that are closest to the cluster center position of the corresponding cluster in each cluster.

[0044] It should be noted that DBSCAN, as a density-based clustering algorithm, can effectively identify sample pair features with similar density. It can divide data points into different clusters. For example, if the density of data points in a region exceeds a certain threshold, these points will be divided into a cluster.

[0045] For example, the key parameters of DBSCAN include ε and MinPts. ε represents the maximum radius of the cluster, that is, the maximum distance between two samples to be considered as the same cluster; MinPts represents the minimum number of samples in each cluster, which determines the minimum density of the cluster. DBSCAN does not need to specify the number of clusters in advance, and can effectively cluster sample pairs with similar density into clusters.

[0046] For each cluster, calculate its cluster center, that is, the mean vector of all difference feature vectors in the cluster. In each cluster, select the sample pair difference feature that is closest to the cluster center. This sample pair difference feature will be used as the "most representative feature" of the cluster and can represent the core corpus feature of the cluster. Here, the distance metric can use distance metric methods such as Euclidean distance, Manhattan distance or cosine similarity.

[0047] Through the embodiments of the present application, DBSCAN can not only find the clustering relationship between the difference features of corpus sample pairs by density clustering, but also effectively discover feature differences with density similarity. For example, the difference features of certain corpus sample pairs may appear as high-density groups in sentiment analysis tasks. DBSCAN can identify these groups and mark them to help focus on task-related features. Thus, the shortcomings of traditional feature selection methods based on distance or artificial rules are avoided, and effective feature differences can be automatically discovered from large-scale data. In addition, DBSCAN does not need to preset the number of clusters, and dynamically divides clusters according to data density. Whether it is a small-scale corpus set or a large-scale massive data set, the scheme can handle corpus data of different scales and properties through adaptive clustering.

[0048] Regarding the implementation details of step S130, in some examples of the embodiments of the present application, the sample pair difference features corresponding to each group of corpus sample pairs are filled into the preset text summary task prompt template to determine the corresponding third prompt word. Then, the third prompt word is input into the large language model to perform text summary summary on the difference features of each sample pair to obtain the corpus features of the corpus application task.

[0049] In some embodiments, by presetting a prompt template for triggering the text summarization task of the large language model, the large language model can better obtain the corpus features by summarizing the difference features of each sample pair. Exemplarily, the prompt template of the text summarization task can be expressed as "The following are the difference features of each sample pair: ____; please summarize the above difference features, extract the most important features for the corpus task and give the corresponding feature meanings." In this way, through a clear and structured prompt format, the language model is helped to focus on feature summarization and extract the core information related to the task.

[0050] Through the embodiment of the present application, the large language model is required to perform the text summarization task through the prompt word (prompt), and all the final corpus features are summarized, which can more accurately understand the semantics of the features. However, it should be noted that in the process of directly summarizing the corpus features by the large language model, due to the loss of the frequency information of the original features, its output results may be interfered by outliers.

[0051] Figure 3 A schematic diagram of the operational flow of an example of a method for automatically extracting key features of spontaneous language cognitive functions based on a large language model according to an embodiment of the present application is shown.

[0052] like Figure 3As shown, the method for automatically extracting key features of spontaneous language cognitive functions based on a large language model includes the following three stages: stage one - pairing of positive and negative samples, stage two - difference analysis of the large language model, and stage three - hierarchical feature aggregation.

[0053] The following will take the cognitive function screening task as an example of the corpus application task to expand the details of the corpus feature extraction in the embodiment of this application. Specifically, each sample comes from the answer of a tester when facing the cognitive function screening, including the text record of the tester's answer to multiple questions in the screening, and the label of whether the tester has cognitive dysfunction.

[0054] In the scenario where the tester participates in cognitive function screening, each sample in the dataset contains multiple paragraphs of text, recording the tester's answers to multiple questions in the test and a label, which records whether the tester has cognitive dysfunction.

[0055] Take the Mini-MoCA test as an example. In this test, the test taker needs to answer the following seven questions (i.e., conditional corpus): 1. Tell me what year it is.

[0056] 2. Tell me what month it is now.

[0057] 3. Tell me the exact date today.

[0058] 4. Tell me what day it is today.

[0059] 5. Tell me what this place is and which city it is in.

[0060] 6. Please name as many animals as you can as quickly as possible. You have one minute to think about it. Are you ready? Start.

[0061] 7. I just read a few words to you and asked you to remember them. Please try your best to recall them and tell me what these words are.

[0062] It should be understood that there are many tasks for cognitive function screening, and the embodiments of the present application should not be limited to the Mini-MoCA test. In addition, there are many types of application directions for corpus application tasks, and they should not be limited to cognitive function screening applications. The relevant description is only used to make it easier for the public to understand the solution of the present application. It is used as an example and is not regarded as a limitation on the present application.

[0063] In the first stage, the positive and negative samples in the dataset are randomly matched 1:1 according to the labels, and the two matched samples are put into the paired dataset. This is to highlight the difference between positive and negative samples.

[0064] In the second stage, the large language model is used to guide the LLM (for example, QWen2.5-72B-instruct) to analyze and summarize the paired data using manually designed prompt words. First, the large language model analyzes the differences for each question in the paired data and outputs the difference features of the positive and negative samples, including the definition of the features and their specific performance on the paired data; then, after all the questions in a pair of data are analyzed separately, the large language model is used to summarize the features. This is because the paired data may contain multiple questions, and their difference features may be repeated. Finally, for a pair of data, several original features are obtained, including the definition and meaning of the features.

[0065] Specifically, examples of prompts in the analysis operation of stage 2 are as follows: # Role Neuropsychological Test Analyst # Profile You are a professional neuropsychological test analyst who specializes in evaluating and analyzing individual performance differences in cognitive function tests and is able to discover key cognitive function differences from the details.

[0066] # Background It is necessary to analyze the responses of two test takers to a series of questions in order to identify and compare the differences between the responses of the two test takers and to find certain features to distinguish the test takers with the disease from the test takers without the disease. These features include but are not limited to features in language, grammar, vocabulary, prosody, morphology, etc. It is also necessary to discover and define new features.

[0067] # Skills You have professional knowledge of neuropsychology and linguistics, and can discover the differences between test takers by analyzing their responses.

[0068] #Goals Analyze the responses of the two test takers in detail and analyze as many differences as possible.

[0069] # Constrains 1. Please note that the analysis should be based on the content and form of the answers, avoiding judgment on personal identities and ensuring the objectivity and scientific nature of the analysis.

[0070] 2. Please note that there is no need to judge whether the specific time, date and place are consistent with the actual situation.

[0071] 3. Please note that these problems may occur in cognitively normal people, and not all people with cognitive impairment will display all of these characteristics.

[0072] 4. Please note that this assessment is not intended to provide a formal diagnosis but rather serve as an initial screening tool for potential MCI.

[0073] # Format ## Input Format 1. Input will be provided to you in this format: {{"question": " <question>", "record":{{"tester1": "<tester1 record> ", "tester2": "<tester2 record> "}}}} 2. The input includes the following information: the problem faced by the tester <question>、Tester 1's answer<tester1 record> 、Tester 2's answer<tester2 record> ## Output Format 1. First, use a paragraph to summarize the differences between the two test takers’ answers 2. Next, for each feature that produces a difference, explain its meaning and describe the difference in the two test takers’ answers under this feature.

[0074] # Workflow 1. Read the questions faced by the test takers, the test takers’ answers in detail, and record the answers of the two test takers to each question.

[0075] 2. Compare the answers of the two test takers on each question, analyze the differences, and propose features so that the answers of the two test takers are obviously different under this feature.

[0076] 3. For each feature, explain its meaning and describe the difference between the two test takers’ answers on this feature.

[0077] 4. Output your answer strictly according to the Output Format # Input {input} # Output Input: {input} is replaced by a record of a question { "question": "I just read a few words to you and asked you to remember them. Please try to recall them and tell me what they are.", "record": { "tester1": "Can you remember the words he just taught you to remember? Otherwise, remember them and see if you can't remember any of them? He asked you if you can still remember them?", "tester2": "The red chrysanthemum in the face hotel. What other cotton lipstick red does Tiger have? The chrysanthemum hotel is out of stock, and I don't have any other ones. Hmm." } Output: Summarize the differences and list each difference feature { "id_question": 18, "question": "I just read a few words to you and asked you to remember them. Please try to recall them and tell me what they are.", "content": "### Summary\nTester 1 and Tester 2 showed significant differences in answering the question of memorizing words. Tester 1's answer was relatively coherent, and he was able to try to recall the words, but he finally said that he could not remember them. Tester 2's answer was confusing, the words he provided were irrelevant to the question, and there were repetitions and interruptions in his expression.\n\n### Analysis of difference characteristics\n\n1. **Coherence**\n - **Meaning**: Coherence refers to whether the answer is logically clear, relevant in content, and able to communicate effectively around the question.\n - **Difference**:\n - **Tester 1**: Although the answer failed to recall the words completely, it maintained logical coherence as a whole and was able to try to recall around the question.\n - **Tester 2**: The answer content was irrelevant to the question, and there were many deviations from the topic, such as mentioning "What lipstick does Lao Zhu have", etc., indicating that his answer lacked coherence.\n\n2. **Vocabulary Choice**\n - **Meaning**: Vocabulary choice refers to whether the words used in the answer are appropriate and accurate.\n - **Difference**:\n - **Tester 1**: Used relatively common words, such as "remember", "can't remember", etc., and the word selection was relatively appropriate. \n - **Tester 2**: Used words that were not related to the question, such as "speaker", "lipstick", etc., indicating that his vocabulary selection ability was impaired. \n\n3. **Grammatical structure**\n - **Meaning**: Grammatical structure refers to whether the sentence structure in the answer is correct and complete. \n - **Difference**: \n - **Tester 1**: The sentence structure is relatively complete. Although there are interrogative sentences, it conforms to grammatical rules as a whole. \n - **Tester 2**: The sentence structure is incomplete, with broken sentences and incoherent expressions, such as "I don't have anything else", etc., indicating that his grammatical structure is impaired. \n\n4. **Repetition and interruption**\n - **Meaning**: Repetition and interruption refers to whether there are repeated words or sentences in the answer, and whether there are frequent interruptions. \n - **Difference**: \n - **Tester 1**: There is no obvious repetition or interruption, and the answer is relatively fluent. \n - **Tester 2**: There were multiple repetitions, such as "I don't have anything else", and there were interruptions in the answer, such as "Um.", indicating that his language fluency was impaired. \n\n5. **Content relevance**\n - **Meaning**: Content relevance refers to whether the answer is directly related to the question. \n - **Difference**: \n - **Tester 1**: The answer was related to the question. Although the words could not be fully recalled, they were trying to recall them. \n - **Tester 2**: The answer was irrelevant to the question. Words irrelevant to the question were mentioned, such as "speaker" and "lipstick", indicating that the content relevance was impaired.\n\nThrough the above analysis, test subject 2 showed obvious cognitive impairment in multiple features, while test subject 1, although unable to recall the words completely, maintained good cognitive function overall. These differential features can be used as a reference for further assessment of potential MCI. ". } An example of a prompt in the summary operation of stage 2 is as follows: prompt: # Role Cognitive function analysis expert # Profile You are a neuropsychologist specializing in cognitive dysfunction. You have a deep understanding and research on the symptoms and manifestations of cognitive dysfunction and can accurately identify the characteristics of your patients' responses.

[0078] # Background It is necessary to distinguish which of the two answers may come from patients with cognitive dysfunction. Such patients usually show symptoms of cognitive dysfunction such as memory loss, orientation, judgment, calculation, abstract thinking, language, etc., and may also be accompanied by mental symptoms such as personality or behavior changes, delusions or hallucinations # Skills You have professional psychological assessment skills and are able to analyze and identify signs of cognitive dysfunction in responses.

[0079] #Goals Based on the answers of the two testers and the analyst's analysis, it is accurately identified which tester is more likely to be a patient with cognitive dysfunction.

[0080] # Constrains 1. Please note that the analysis should be based on the content and form of the answers, avoiding judgment on personal identities and ensuring the objectivity and scientific nature of the analysis.

[0081] 2. Please note that there is no need to judge whether the specific date and place are correct.

[0082] 3. Please note that these problems may occur in cognitively normal people, and not all people with cognitive impairment will show all of these characteristics. The final classification should be based on the combination of all indicators.

[0083] 4. Please note that this assessment is not intended to provide a formal diagnosis but rather serve as an initial screening tool for potential MCI.

[0084] # Format ## Input Format 1. The input is a list, each element of which is a question for the tester, the tester's answer and the analyst's analysis record.

[0085] 1. Input will be provided to you in this format: "[{{"question": <question>, "record":{{"tester1":<tester1 record>, "tester2":<tester2 record>}}, "analysis": <analysis>}}...]" 2. The input includes the following information: the problem faced by the tester <question>、Tester 1's answer<tester1 record> 、Tester 2's answer<tester2 record> , Analysts analyze the differences between the two testers' answers to the current question <analysis> ## Output Format 1. First, give your diagnosis (1 means that test subject 1 is more likely to have cognitive impairment, and 2 means that test subject 2 is more likely to have cognitive impairment), then summarize your reasons, and finally, for each differential feature, give a definition and explanation.

[0086] 2. Output format: "{{"diagnosis":<diagnosis_1_or_2> , "reason": <reason>, "features": [{{"feature":<feature_i> , "explanation":<explanation_i> ...}}]}}".

[0087] 3. The output includes the following information: Your final judgment on which test subject is more likely to have cognitive impairment<diagnosis_1_or_2> , summarize your judgment <reason>, you summarize the characteristics that distinguish patients from normal people<feature_i> Explanation of this feature<explanation_i> .

[0088] # Workflow 1. Carefully read the content of the questions under different questions in the input, the answers of the two testers under different questions, and the difference features proposed by the analyst.

[0089] 2. Identify possible signs of cognitive dysfunction in responses, such as memory errors, confusion, and decreased abstract thinking ability.

[0090] 3. Based on the analysis results of the sub-questions, analyze which answer is more likely to come from a patient with cognitive impairment and give your reasons.

[0091] 4. Summarize the characteristics that distinguish patients from normal people. For each characteristic, explain its meaning and describe the difference between the two test takers' answers under this characteristic.

[0092] 5. Output your answer strictly according to the Output Format # Input {input} # Output Input: {input} is replaced with the results of all phase 2 analyses of this sample. Output: Determine which of the two testers is a positive sample; summarize all features, including definitions and explanations.

[0093] { "diagnosis": 1, "reason": "Test subject 1 showed signs of cognitive dysfunction in his answers to multiple questions, including vague time concepts, repeated questions, poor language organization, poor vocabulary, simple and incoherent sentence structure, etc. In contrast, although test subject 2 also had some repetitive phenomena, he performed better in language expression and logical thinking.", "features": [ { "feature": "The concept of time is fuzzy", "explanation": "When responding to questions about the year and month, test subject 1 showed obvious uncertainty and errors, such as repeating 'he asked, he asked' several times, indicating that he was not clear about the current year and month." }, { "feature": "Repeat questions", "explanation": "The tester 1 repeatedly asked questions in multiple problems, such as 'He asked you what year it is now?', showing a decline in memory and comprehension abilities." }, { "feature": "Poor language organization ability", "explanation": "In the tester 1's answers, the same phrases were repeatedly used many times, such as 'No no no', indicating poor language organization ability." }, { "feature": "Lack of vocabulary", "explanation": "The vocabulary used by the tester 1 is relatively single, and simple words are repeatedly used, such as 'He asked', showing a limited vocabulary." }, { "feature": "Simple and incoherent sentence structure", "explanation": "In the tester 1's answers, the sentence structure is simple and lacks coherence, such as 'This this this', indicating weak language expression ability." } } In stage three, all the original features are aggregated, and the DBSCAN clustering algorithm can be used. The clustering is based on the density distribution of data points. Specifically, DBSCAN depends on two parameters: ϵ (epsilon) and minPts. For a given point, if there are at least minPts points within its ϵ radius, then this point is marked as a core point. If a point is a core point and there are other points within its ϵ range, these points are considered directly density-reachable. By recursively visiting the neighborhood of each core point, clusters that are density-connected can be formed. Non-core and isolated points are marked as noise.

[0094] Exemplarily, the parameters of the DBSCAN clustering method are ϵ = 0.04 and minPts = 1. The cosine similarity of vectors is used as the distance metric, and finally 50 clusters are produced. In each cluster, the mean of all key points is calculated as the cluster center, and the feature closest to the cluster center is selected as the finally output feature and put into the output feature set. Then the features in the output feature set can comprehensively and accurately reflect the differences between positive and negative samples and are used to characterize cognitive functions.

[0095] ​In the embodiment of the present application, the method of randomly organizing 1:1 positive and negative example pairing data, using a large language model for analysis, and aggregating hierarchical features breaks away from the idea of ​​manually designing task-related features or designing classification models to generate supervisory signals for general features, and can extract features with semantic meanings. In addition, when facing different tasks and data set forms, automatic feature extraction can be performed in the above manner.

[0096] Through the embodiments of the present application, features with high-level interpretability can be automatically summarized. Specifically, a large language model is first used to perform diversified analysis, and then the differences between all positive and negative samples are summarized, so that the large language model can focus on high-level features such as language organization ability, vocabulary ability, cognition and memory ability, rather than low-level statistical features such as word frequency and part of speech ratio, which makes the features extracted by the method provided in the embodiments of the present application have strong interpretability.

[0097] It should be noted that current feature extraction methods often face the problem that a large number of features extracted by neural networks lack interpretability, while strictly defined and interpretable features rely heavily on expert experience and require the introduction of a large number of manual steps. If key features can be automatically extracted from a data set and the features are interpretable, this contradiction can be resolved to a certain extent, thereby more intelligently and automatically discovering key features that reflect cognitive functions, and further changing society's understanding of cognitive functions.

[0098] In addition, the feature extraction process of the embodiment of the present application can be directly applied to the open source or closed source general large language model, without having to fine-tune the large language model, which can reduce the cost of model training. At present, the bottleneck of large language model research lies in the high training cost. If the cost can be reduced, this bottleneck can be broken to a certain extent, thereby forming a more intelligent large language model and further changing the production and lifestyle of society.

[0099] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of actions combined, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application. In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0100] In some embodiments, an embodiment of the present application provides a non-volatile computer-readable storage medium, which stores one or more programs including execution instructions, and the execution instructions can be read and executed by an electronic device (including but not limited to a computer, a server, or a network device, etc.) to execute any of the above-mentioned corpus feature extraction methods of the present application.

[0101] In some embodiments, the embodiments of the present application also provide a computer program product, which includes a computer program stored on a non-volatile computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes any one of the above-mentioned corpus feature extraction methods.

[0102] In some embodiments, the embodiments of the present application also provide an electronic device, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute a corpus feature extraction method.

[0103] Figure 4 FIG. 1 is a schematic diagram of the hardware structure of an electronic device for executing a corpus feature extraction method provided by another embodiment of the present application. Figure 4 As shown, the device includes: One or more processors 410 and memory 420, Figure 4 A processor 410 is taken as an example.

[0104] The device for executing the corpus feature extraction method may further include: an input device 430 and an output device 440 .

[0105] The processor 410, the memory 420, the input device 430 and the output device 440 may be connected via a bus or other means. Figure 4 The example of connecting through bus is taken in the following.

[0106] The memory 420 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions / modules corresponding to the corpus feature extraction method in the embodiment of the present application. The processor 410 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions and modules stored in the memory 420, that is, the corpus feature extraction method in the above method embodiment is implemented.

[0107] The memory 420 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 420 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 420 may optionally include a memory remotely arranged relative to the processor 410, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0108] The input device 430 may receive input digital or character information and generate signals related to user settings and function control of the electronic device. The output device 440 may include a display device such as a display screen.

[0109] The one or more modules are stored in the memory 420, and when executed by the one or more processors 410, the corpus feature extraction method in any of the above method embodiments is executed.

[0110] The above-mentioned product can execute the method provided in the embodiment of the present application, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not fully described in this embodiment, please refer to the method provided in the embodiment of the present application.

[0111] The electronic devices of the embodiments of the present application exist in various forms, including but not limited to: (1) Mobile communication equipment: This type of equipment is characterized by having mobile communication functions and its main purpose is to provide voice and data communications. This type of terminal includes: smart phones, multimedia phones, functional phones, and low-end phones.

[0112] (2) Ultra-mobile personal computer devices: These devices belong to the category of personal computers, have computing and processing functions, and generally also have mobile Internet access features. These terminals include: PDA, MID and UMPC devices, etc.

[0113] (3) Portable entertainment devices: These devices can display and play multimedia content. They include audio and video players, handheld game consoles, e-books, smart toys, and portable car navigation devices.

[0114] (4) Other onboard electronic devices with data interaction functions, such as on-board devices installed in vehicles.

[0115] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0116] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the relevant technology, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0117] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.< / reason> < / reason> < / analysis> < / question> < / analysis> < / question> < / question> < / question>

Claims

1. A corpus feature extraction method, comprising: Selecting at least one set of corpus sample pairs from a corpus data set for a corpus application task, wherein the corpus sample pairs include positive corpus samples and negative corpus samples; Each corpus sample includes feedback corpus for at least one preset conditional corpus and a corresponding sample label, wherein the sample label is used to indicate whether the corresponding corpus sample is a positive corpus sample or a negative corpus sample; For each group of the corpus sample pairs, analyzing the difference information of the feedback corpus of the positive corpus sample and the feedback corpus of the negative corpus sample in the corpus sample pair for each of the conditional corpora based on the large language model to determine the difference features of the corresponding sample pairs; Based on the sample pair difference features corresponding to each group of corpus sample pairs, the corpus features of the corpus application task are determined.

2. The method according to claim 1, wherein: For each group of the corpus sample pairs, analyzing the difference information of the feedback corpus of the positive corpus sample and the feedback corpus of the negative corpus sample in the corpus sample pair for each of the conditional corpora based on the large language model to determine the corresponding sample pair difference features, including: Filling the feedback corpus of the positive corpus sample, the feedback corpus of the negative corpus sample and each of the conditional corpora into a preset feedback difference analysis task prompt template to determine a corresponding first prompt word; Inputting the first prompt word into the large language model to analyze feedback difference information between the feedback corpus of the positive corpus sample and the feedback corpus of the negative corpus sample when processing each conditional corpus; Filling each of the feedback difference information into a preset feedback difference summary template to determine a corresponding second prompt word; The second prompt word is input into the large language model to summarize the condition-feedback difference features and feature definitions corresponding to each of the feedback difference information to obtain corresponding sample pair difference features.

3. The method according to claim 1, wherein: The determining of the corpus features of the corpus application task based on the sample pair difference features corresponding to each group of corpus sample pairs includes: The sample pair difference features corresponding to each group of corpus sample pairs are clustered, and the corpus features corresponding to the corpus application task are determined according to the clustering result.

4. The method according to claim 3, wherein: The step of clustering the sample pair difference features corresponding to each group of corpus sample pairs, and determining the corpus features of the corpus application task according to the clustering result, includes: Based on the DBSCAN clustering algorithm, the sample pair difference features corresponding to each group of corpus sample pairs are clustered to determine the corresponding multiple clusters; The corpus features of the corpus application task are determined according to the difference features of the sample pairs that are closest to the cluster center position of the corresponding cluster in each cluster.

5. The method according to claim 1, wherein: Before determining the corpus features of the corpus application task based on the sample pair difference features corresponding to each group of corpus sample pairs, the method further includes: Count the frequency of occurrence of the difference features of each sample; The samples whose corresponding feature occurrence frequency is lower than the preset frequency threshold are filtered out for the difference features.

6. The method according to claim 1, wherein: The determining of the corpus features of the corpus application task based on the sample pair difference features corresponding to each group of corpus sample pairs includes: Filling the sample pair difference features corresponding to each group of corpus sample pairs into the preset text summarization task prompt template to determine the corresponding third prompt word; The third prompt word is input into the large language model to perform a text summary on the difference features of each sample pair to obtain the corpus features of the corpus application task.

7. The method according to claim 1, wherein: The step of selecting at least one set of corpus sample pairs from a corpus data set used for a corpus application task comprises: Randomly match samples in the corpus dataset according to the sample labels to select at least one set of corpus sample pairs.

8. A storage medium having a computer program stored thereon, wherein: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 7 are implemented.

9. An electronic device, comprising: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the method described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program / instruction, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 7.

Images