Method and system for evaluating influence of social attributes on large model diagnostic reasoning process

Abstract

The invention relates to the technical field of natural language processing, and discloses a method and system for evaluating the influence of social attributes on a large model diagnostic reasoning process. The method comprises the following steps: standardizing an original medical record data set, removing samples associated with population attributes, and arranging the samples into a hierarchical standard format to obtain a standardized data set; the standardized data set is neutralized, population attribute information is replaced, and complete neutrality is ensured through backstepping verification; for various social attributes, attribute texts are generated based on multiple sub-dimensions, corresponding fields of medical records in the neutral data set are embedded through semantic fusion, and a fusion data set is constructed; obtaining a diagnosis result by using a large language model; and calculating a performance index and a reasoning style change index based on the diagnosis result, and evaluating the influence of social attributes on the big language model diagnosis reasoning process. According to the method, the real effects of different social attributes on diagnostic reasoning can be comprehensively measured.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, specifically to a method and system for assessing the impact of social attributes on the diagnostic reasoning process of large models. Background Technology

[0002] In recent years, the application of Large Language Models (LLMs) in medical reasoning tasks has deepened, and their use of multidimensional patient information in clinical decision-making has gradually attracted attention. Existing research indicates that, without explicit constraints, patient sensitivity attributes may influence the diagnostic reasoning of LLMs, thereby altering decision outcomes. However, current research primarily focuses on demographic attributes such as gender, age, and race, lacking a systematic examination of the role of social attributes such as culture, education, and economic status. Therefore, systematically assessing the impact of social attributes on diagnostic reasoning is crucial for further improving the safety and clinical usability of LLMs.

[0003] Current methods for assessing social attributes suffer from two main bottlenecks: first, they struggle to accurately depict the behavioral characteristics of large language models during diagnostic reasoning, for example, overly simplistic task design, data detached from standard clinical context, and assessment procedures that do not strictly adhere to standard diagnostic principles; second, the methods for introducing attributes are crude, lacking fair and nuanced semantic connections with medical record content. These two bottlenecks significantly restrict the accuracy and objectivity of the assessment process.

[0004] Specifically, regarding the first bottleneck, traditional assessment methods often employ true / false, multiple-choice, or open-ended preference questions. These tasks have short reasoning chains and limited complexity, making it difficult to capture the multi-factor comprehensive considerations involved in real diagnosis, resulting in a significant gap with clinical practice. Furthermore, these methods generally rely on medical data derived from textbooks or exam questions, and the content is mostly based on medical common sense, lacking standardized medical record structures and complete clinical contexts. This makes it difficult for assessment results to fully reflect the behavior of the large language model in clinical diagnosis. Although recent assessment methods have attempted to conduct diagnostic tasks based on medical record data, significant limitations remain: some data lacks systematic cleaning and structured organization, resulting in inconsistent quality; the assessment process does not strictly adhere to the standard diagnostic principle centered on major and urgent health problems, and the distinction between primary and accompanying diagnoses is unclear. These problems directly interfere with the assessment results, making it difficult to accurately attribute the performance of the large language model.

[0005] Regarding the second bottleneck, existing methods typically introduce attribute values ​​through explicit annotation or text concatenation during the diagnostic process. This direct approach ignores the fine-grained semantic connections between attribute text and medical record content, not only deviating from the real clinical context but also affecting the objective understanding of attributes by large language models. To alleviate this problem, some advanced studies have attempted to introduce multi-agent collaborative mechanisms and construct assessment data at the semantic level through a "generation-fusion" paradigm, which is considered a more promising technical direction. However, existing related work still has shortcomings at the methodological and implementation levels: First, attribute texts have limited length, prioritizing expressive power; second, attribute characterization is coarse-grained, with highly similar expressions across different attribute texts, making it difficult to reflect their impact on diagnostic reasoning in a detailed and comprehensive manner from multiple perspectives; third, some methods rely on "aggressive" or "escape-style" sample construction, which can amplify attribute effects but also introduces implicit guiding signals that are close to or deviate from the standard answer, making it difficult to fully decouple the model's inherent response to attributes from the construction strategy effect; fourth, the superiority of the multi-agent collaborative paradigm lacks systematic demonstration.

[0006] In summary, current technologies are insufficient to conduct a detailed, comprehensive, and systematic assessment of the impact of social attributes on the diagnostic reasoning process while adhering to standard clinical diagnostic paradigms. There is an urgent need to establish a fine-grained and highly reliable assessment scheme. Summary of the Invention

[0007] To address the aforementioned technical problems, this invention proposes a method and system for evaluating the impact of social attributes on the diagnostic reasoning process of large models.

[0008] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0009] In a first aspect, the present invention provides a method for assessing the impact of social attributes on the diagnostic reasoning process of large models, comprising: The original medical record dataset was standardized by removing samples associated with demographic attributes and organizing it into a hierarchical standard format to obtain a standardized dataset. The standardized dataset is neutralized by replacing the population attribute information and verifying it through reverse calculation to obtain a neutral dataset. For multiple social attributes, attribute text is generated based on multiple sub-dimensions, and corresponding fields of medical records in the neutral dataset are embedded through semantic fusion to construct a fused dataset corresponding to each social attribute. A large language model is used to perform a constrained diagnostic task that outputs only a unique primary diagnosis on the neutral dataset and each fused dataset to obtain diagnostic results. Based on the diagnostic results, performance indicators and reasoning style change indicators are calculated to assess the impact of social attributes on the diagnostic reasoning process of large language models.

[0010] In one embodiment, the standardization process of the original medical record dataset, which involves removing samples associated with demographic attributes and organizing them into a hierarchical standard format to obtain a standardized dataset, specifically includes the following steps: S11, based on rules and regular expressions, remove medical records from the original medical record dataset that are irrelevant to the diagnostic task, and label the source of the remaining medical records; S12, treat medical records from each source as a group, use a text encoding model to convert the medical records from the current source into semantic vectors, calculate the cosine similarity between the semantic vectors of any two medical records, and determine that the two cases are semantic duplicate pairs when the cosine similarity between the two cases is higher than a set threshold; model the case deduplication problem as an undirected graph, identify duplicate case clusters by calculating connected components, and perform in-group deduplication. S13, using large language models to identify and filter medical records that are associated with demographic attributes; S14. Using a large language model, the filtered medical records are organized into a standard medical record format, all clinically significant information is extracted, and the information is classified and organized into a hierarchical form that includes patient behavior information, physical examination results, and auxiliary examination results. S15, Perform an integrity check on each medical record with a standard medical record format, and remove medical records with empty top-level fields or missing information in lower-level fields. S16 integrates medical records from different sources that have undergone integrity checks and deduplication again to obtain a standardized dataset.

[0011] In one embodiment, modeling the deduplication problem of cases as an undirected graph, identifying clusters of duplicate cases by calculating connected components, and performing intra-group deduplication specifically includes: Each medical record is treated as a node in an undirected graph. If two medical records are semantically duplicated, there is an undirected edge between the corresponding nodes. Duplicate case clusters are identified by calculating the connected components of the undirected graph. Only one representative medical record is retained for each connected component, and the rest are removed.

[0012] In one embodiment, the neutralization process of the standardized dataset, replacing population attribute information and verifying it through reverse engineering, to obtain a neutral dataset specifically includes: The explicit or implicit demographic attribute values ​​of medical records in the standardized dataset are replaced with neutral placeholders, pronoun elimination is performed, and the processed medical records are input into a large language model for back-inference verification. Medical records that can still be inferred from demographic attributes are removed to obtain a neutral dataset.

[0013] In one embodiment, the social attributes include: positive economic attributes, negative economic attributes, positive educational attributes, negative educational attributes, Eastern cultural attributes, Western cultural attributes, positive economic and positive educational cross attributes, and negative economic and negative educational cross attributes.

[0014] In one embodiment, the step of generating attribute text based on multiple sub-dimensions for various social attributes, and embedding corresponding fields from medical records in a neutral dataset through semantic fusion to construct a fused dataset corresponding to each social attribute, specifically includes: A large language model is used as a neutral agent to generate attribute text for each social attribute. During generation, conditional constraints are applied based on the value of the social attribute: when the economic and educational attributes are positive, the corresponding attribute text reflects favorable social characteristics; when the economic and educational attributes are negative, the corresponding attribute text reflects unfavorable social characteristics. For cultural attributes, corresponding characteristics are presented according to regional differences. The attribute text contains only social attribute-related content and does not include medical facts, disease information, treatment results, or prognostic implications. The attribute text is systematically characterized based on multiple sub-dimensions, including: educational attributes such as education level, education quality, education content, functional skills, and education continuity; economic attributes such as economic level, income stability, income source, purchasing power, wealth status, and asset status; and cultural attributes such as lifestyle, social norms, traditional festivals, community participation, values, hobbies, and social identity. When generating attribute text, the patient behavior field of medical records in the neutral dataset is used as auxiliary context; When generating attribute text for cross attributes, it is generated based on the common values ​​of the cross attributes; The generated attribute text is assigned to the corresponding fields of medical records in the neutral dataset and integrated with the original content of the fields to form a fused dataset. The semantic preservation of the original content and the semantic absorption of the attribute text are analyzed based on cosine similarity during the integration process.

[0015] In one embodiment, the degree of semantic preservation of the original content and the degree of semantic absorption of the attribute text during the cosine similarity analysis-based integration process specifically includes: Each medical record in the neutral dataset is directly concatenated with its corresponding attribute text to construct a concatenated dataset; By using a text vectorization model, the collection of attribute texts, the fused dataset, the neutral dataset, and the concatenated dataset are uniformly mapped to the same vector space; The cosine similarity sim1 between the neutral dataset and the fused dataset, and the cosine similarity sim2 between the neutral dataset and the spliced ​​dataset are calculated to measure the degree of semantic preservation of the original content in the process of introducing social attribute information. The cosine similarity injection1 between the attribute text set and the fused dataset, the cosine similarity injection2 between the attribute text set and the neutral dataset, and the cosine similarity injection3 between the attribute text set and the concatenated dataset are calculated to measure the degree to which the attribute text is semantically absorbed.

[0016] In one embodiment, the method further includes: for each diagnostic result, first comparing it directly with the corresponding standard answer; if they are completely consistent, the diagnosis is considered correct; if they are inconsistent, a large language model is used to map the diagnostic result and the standard answer to names that conform to the International Classification of Diseases (ICD) standard, and then comparing them again; if they are completely consistent, the diagnosis is considered correct, otherwise the diagnosis is considered incorrect.

[0017] In one embodiment, the performance metrics include accuracy and relative accuracy; the inference style variation metrics include consistency rate, KL divergence, multidimensional scaling analysis (MDS), and t-distributed random neighborhood embedding (t-SNE) method.

[0018] In a second aspect, the present invention provides a computer system including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method of any embodiment of the first aspect.

[0019] Compared with the prior art, the beneficial technical effects of the present invention are: This invention covers the entire lifecycle of analyzing the impact of social attributes on diagnostic reasoning, effectively balancing data standardization, fine-grained semantic association, and multi-agent collaboration mechanisms, thereby comprehensively measuring the true role of different social attributes in diagnostic reasoning. The construction of this evaluation framework presents significant technical challenges and is crucial for promoting the safe and reliable application of large language models in clinical diagnosis. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the method flow of the present invention.

[0021] Figure 2 This is a flowchart illustrating the standardized portion of the present invention.

[0022] Figure 3 This is a flowchart of the neutralization process of the present invention.

[0023] Figure 4 This is a schematic diagram of the workflow of the data construction part of the present invention.

[0024] Figure 5 This is a schematic diagram of the workflow of the constraint diagnosis and evaluation section of the present invention.

[0025] Figure 6 The graph shows the evaluation results of the cosine similarity-based analysis method of the present invention on positive economic attributes, negative economic attributes, positive educational attributes, and negative educational attributes.

[0026] Figure 7 The graph shows the evaluation results of the cosine similarity-based analysis method of the present invention on Eastern cultural attributes, Western cultural attributes, the intersection of positive economic and positive education attributes, and the intersection of negative economic and negative education attributes.

[0027] Figure 8 This is a graph showing the diagnostic accuracy results of the present invention.

[0028] Figure 9 This is a graph showing the relative accuracy results of the present invention.

[0029] Figure 10 This is a graph showing the consistency rate of the large language model of the present invention across positive economic attributes, negative economic attributes, positive educational attributes, and negative educational attributes.

[0030] Figure 11 This is a graph showing the consistency rate of the large language model of this invention across Eastern cultural attributes, Western cultural attributes, and the intersection of positive economy and positive education attributes.

[0031] Figure 12 This is a KL divergence diagram showing the results of different large language models under the conditions of positive economic attributes, negative economic attributes, positive educational attributes, and negative educational attributes.

[0032] Figure 13 This is a KL divergence diagram showing the results of different large language models under the conditions of Eastern cultural attributes, Western cultural attributes, and the intersection of positive economic and positive educational attributes.

[0033] Figure 14 This is a diagram showing the model characterization analysis results based on the multidimensional scaling (MDS) method of this invention.

[0034] Figure 15 The figure shows the model characterization analysis results based on the t-distributed random neighborhood embedding (t-SNE) method of this invention. Detailed Implementation

[0035] A preferred embodiment of the present invention will now be described in detail with reference to the accompanying drawings.

[0036] like Figure 1As shown, a method for assessing the impact of social attributes on the diagnostic reasoning process of large models according to the present invention includes the following steps: S1. Standardize the original medical record dataset, remove samples associated with population attributes, and organize it into a hierarchical standard format to obtain a standardized dataset. S2, neutralizes the standardized dataset, replaces population attribute information, and verifies it through reverse calculation to obtain a neutral dataset; S3 generates attribute text based on multiple sub-dimensions for various social attributes, and embeds the corresponding fields of medical records in the neutral dataset through semantic fusion to construct a fused dataset corresponding to each social attribute. S4. Utilize the large language model to perform a constrained diagnostic task that outputs only a unique primary diagnosis on the neutral dataset and each fused dataset to obtain diagnostic results. S5. Based on the diagnostic results, calculate performance indicators and reasoning style change indicators to assess the impact of social attributes on the diagnostic reasoning process of the large language model.

[0037] The overall workflow of this invention will be introduced first, followed by the four parts of standardization, neutralization, data construction, and constraint diagnosis and evaluation.

[0038] 1. Overall workflow.

[0039] The input to this invention is the original medical record dataset, where each data point represents a patient's medical record. The format is not limited, but a standard answer should be included. All languages ​​used in the process are English. First, the original medical record dataset is standardized, outputting a standardized dataset. Then, the standardized dataset is neutralized, outputting a neutral dataset. Next, based on the neutral dataset and all social attributes, multiple fused datasets containing corresponding social attribute content are constructed, outputting multiple datasets. Social attributes include positive economic attributes, negative economic attributes, positive education attributes, negative education attributes, Eastern cultural attributes, Western cultural attributes, positive economic and positive education cross attributes, and negative economic and negative education cross attributes. In addition, the output includes a concatenated dataset and an analysis method based on cosine similarity. Finally, constraint diagnosis and evaluation are performed based on the neutral dataset and all fused datasets, resulting in multiple evaluation indicators.

[0040] 2. Standardization.

[0041] The standardized workflow is as follows: Figure 2 As shown below, each step will be explained in detail.

[0042] The first step, for any input raw medical record dataset in any format, involves using rules and regular expressions to remove data irrelevant to the diagnostic task. For the removed medical records, this section explicitly labels their source (e.g., from a medical website, from an open-source medical record dataset, etc.), and for each source, steps two through five are performed sequentially.

[0043] The second step involves performing deduplication based on text similarity and graph structure (essentially, intra-group deduplication) on medical records from each source. Specifically, each input medical record is fed into an open-source, high-quality text encoding model, converted into semantic vectors, and the cosine similarity between the semantic vectors of different medical records is calculated. Subsequently, using the FAISS tool, several samples with the most semantic similarity are retrieved for each case. When the cosine similarity between the semantic vectors of two cases is higher than a preset threshold, they are determined to be semantically duplicated pairs. To avoid ignoring the transitivity of duplication relationships, this invention further models the deduplication problem as an undirected graph: each medical record is considered a node, and semantic duplication relationships are considered as undirected edges between nodes. Duplicate case clusters composed of direct or indirect semantic similarity are identified by calculating the connected components of the graph. Finally, only one representative medical record is retained for each connected component, and the remaining samples are removed, thereby achieving systematic deduplication of highly semantically similar medical records globally. Compared to traditional traversal deduplication methods that require pairwise comparisons of samples, this strategy significantly reduces the number of candidate combinations, greatly reduces running time, and provides an efficient and reliable data foundation for subsequent steps.

[0044] The third step involves using the high-performance GPT-4.1 model, which is highly aligned with human medical experts, to identify and remove samples that are highly correlated with demographic attributes (e.g., gender, age, race). This step is necessary because this invention focuses on the independent roles of different social attributes in diagnostic reasoning. If medical record information is significantly correlated with other demographic characteristics, it will be difficult to decouple the variables, thereby weakening the effectiveness and interpretability of subsequent attribute effect analysis.

[0045] The fourth step involves organizing the filtered medical records into a standard format using the GPT-4.1 model. Specifically, this section explicitly prompts the GPT-4.1 model to extract all clinically significant information from the medical records, ensuring complete coverage of the medical content and avoiding any omission of crucial details. Subsequently, the GPT-4.1 model categorizes the extracted information according to clinical reasoning logic and organizes it into a hierarchical and structurally standardized form. The top-level fields include three categories: patient behavior information, physical examination results, and auxiliary examination results. Patient behavior information is further subdivided into medical history, primary symptoms (i.e., chief complaint), secondary symptoms, present illness, past medical history, social history, and systemic review. Physical examination results and auxiliary examination results do not require a fixed template; they are autonomously organized by the GPT-4.1 model according to the corresponding logic of "examination item - examination result." This hierarchical organization makes the medical record information clearly structured, enabling the evaluated model to efficiently locate and utilize key clinical information during diagnostic reasoning.

[0046] The fifth step is to perform a completeness check on each standardized medical record. If any top-level field is empty, or if too many lower-level fields are missing information, the medical record will be discarded.

[0047] Finally, the processed medical records from different sources are integrated together, and the deduplication process is followed in step two to obtain a standardized dataset.

[0048] The standardization process can be viewed as a unified interface capable of accepting medical record inputs of any format and standardizing them into a uniform format. This design addresses common shortcomings in previous work: traditional methods often assume the input is already in a standard format, or simply remove non-standard medical records through data cleaning, resulting in the unnecessary discarding of valuable medical data. Furthermore, this unified format not only reduces data heterogeneity but also reveals key insights that are difficult to find in otherwise disorganized formats, such as missing necessary diagnostic links, omitted important examinations, or lack of crucial diagnostic information, thus ensuring high-quality data input and laying a solid foundation for downstream tasks. Unlike traditional data cleaning, this section not only focuses on the accuracy and completeness of the data but also emphasizes structure and reusability, enabling the processed medical records to be directly used in different datasets or research scenarios. Simultaneously, automated processing significantly reduces manual processing costs, saves time, and improves overall efficiency.

[0049] 3. Neutralization.

[0050] The standardization process has removed medical records highly correlated with demographic attributes, but this is still insufficient to completely eliminate potential influences. Even if the disease itself is not strongly correlated with demographic attributes, the demographic information contained in the medical records may still affect the model. To achieve full decoupling, a neutralization component needs to be introduced. Figure 3This section demonstrates the overall workflow of the neutralization process. Specifically, the GPT-4.1 model first identifies all explicit or implicit references to attributes such as gender and age in the input medical records and replaces them with neutral placeholders. This step eliminates demographic bias while ensuring the overall coherence and semantic integrity of the text. Next, this section explicitly instructs the GPT-4.1 model to perform pronoun elimination, performing fine-grained replacements on all gender-related pronouns and relational terms in the medical records to prevent gender information leakage from affecting model inference and further enhance the neutrality of the medical records. For example, pronouns referring to males or females are uniformly replaced with "patient," and gender-related relational terms (such as husband, wife, boyfriend, girlfriend) are replaced with "spouse" or "partner." Finally, this section introduces a reverse inference mechanism: the processed medical records are input into the GPT-4.1 model, and the model attempts to infer gender or age based on the current content. If the model can still successfully infer demographic information from medical records, it indicates that neutralization has not completely eliminated relevant hints, and such medical records will be discarded. Conversely, if the model cannot infer demographic attributes, it indicates that neutralization has been successful, and the medical record will be retained. Through this process, a neutral dataset conforming to a standard format is obtained. The fused dataset built on this basis for evaluation, whose social attribute impressions are entirely determined by subsequently injected attribute information, achieves full decoupling between the original medical record information and demographic attributes.

[0051] 4. Data construction.

[0052] The workflow of the data construction part is as follows: Figure 4 As shown, its core function is to generate corresponding attribute text for each social attribute using the GPT-4.1 model based on a neutral reference dataset, and then fuse the attribute text with standardized medical records based on clinical semantics to construct a fused dataset. Simultaneously, using the neutral dataset, the fused dataset, and all intermediate data generated during the process, the effectiveness of the fusion strategy is systematically analyzed and validated based on semantic similarity. It should be clarified that the social attribute types focused on in this section mainly cover three categories: economic, educational, and cultural. Economic and educational attributes are further divided into positive and negative, while cultural attributes are divided into Eastern and Western cultures. Furthermore, to characterize the cumulative effect of multiple social attributes, this section also introduces the cross-attribute setting of economic and educational attributes. To avoid semantic conflicts and the introduction of confounding factors, the cross-attribute only considers two combinations: "positive with positive" and "negative with negative." Based on the above discussion, the social attributes involved in this section include: positive economic attributes, negative economic attributes, positive educational attributes, negative educational attributes, Eastern cultural attributes, Western cultural attributes, as well as the cross-attribute of positive economic and positive educational attributes and the cross-attribute of negative economic and negative educational attributes.

[0053] After clarifying the types and combinations of social attributes involved, the workflow of this part will be described in detail, starting with attribute text generation.

[0054] For each social attribute, this invention uses GPT-4.1 to generate corresponding text based on the attribute value. Specifically, for economic and educational attributes, when the attribute value is positive, the text needs to reflect relatively favorable social characteristics in the corresponding dimension; when the attribute value is negative, it should reflect relatively restrictive or unfavorable characteristics; for cultural attributes, it only needs to present corresponding characteristics based on East-West differences. Crucially, during the generation process, GPT-4.1 is not provided with a standard answer and is explicitly instructed to act as a fair and neutral generator. This design constitutes one of the key differences between this method and existing multi-agent collaborative methods. Existing methods often rely on "aggressive" or "escape-style" sample construction strategies, which, while amplifying the social attribute effect, inevitably introduce implicit guiding signals that approach or deviate from the standard answer, making it difficult to fully decouple the model's inherent response to social attributes from the influence of the construction strategy itself. In contrast, this method, through controlled and neutral attribute text generation, avoids the injection of additional inference signals, allowing subsequent observed changes in model behavior to be more directly attributed to the social attribute itself.

[0055] To ensure the purity of attribute expression, the generated text is strictly limited to containing only content related to social attributes, and must not introduce any medical facts, disease information, treatment outcomes, or prognostic implications, nor involve or imply other sensitive attributes. This design aims to avoid interference from non-target factors in the model's reasoning process, thereby ensuring that subsequently observed differences in diagnostic behavior can be clearly attributed to the injected social attributes themselves, rather than implicit medical cues or cross-attribute effects. This further supports the neutrality and impartiality of this section.

[0056] Furthermore, the attribute text focuses on social perception and contextual information, rather than medical reasoning itself. By introducing detailed and comprehensive sub-dimensions, it systematically characterizes social attributes to enhance the controllability, consistency, and semantic integrity of attribute expression—another key difference between this method and existing approaches. Specifically, educational attributes are described from multiple aspects, including education level, quality, content, functional skills, and continuity of education; economic attributes cover economic level, income stability, income sources, purchasing power, wealth status, and asset situation; and cultural attributes are developed from dimensions such as lifestyle, social norms, traditional festivals, community participation, values, interests, and social identity. This multi-sub-dimension design avoids the semantic impoverishment problem caused by summarizing complex social attributes with single labels or coarse-grained descriptions, making the generated text more realistic, coherent, and distinctive at the socio-semantic level. Compared to previous methods that only use coarse-grained or templated attribute descriptions, this invention can more fully simulate the comprehensive impact of social attributes on cognition and judgment in real-world scenarios without introducing medical clues. Meanwhile, the fine-grained sub-dimensions provide stable semantic anchors for the subsequent fusion of attribute text and standardized medical records, enabling attribute information to be embedded in the patient context naturally and gradually, rather than as abrupt external interventions. This not only improves the semantic consistency of the fused text but also helps distinguish the model's response to social attributes themselves from its dependence on surface cue patterns in subsequent evaluations, thereby enhancing the interpretability and robustness of the analysis results.

[0057] It is worth noting that in the attribute text generation process, this part provides the top-level field "patient behavior" from the neutral dataset as auxiliary context to the model, allowing the model to utilize non-sensitive information in the patient context to a limited extent without introducing any additional sensitive attributes, thereby further enhancing the realism and naturalness of the text. This design enables the generated text to maintain basic consistency with the patient context while avoiding templated output, thus ensuring the semantic personalization of attribute texts corresponding to different medical records.

[0058] Furthermore, it is important to emphasize that this section handles cross-attributes more rigorously, which is what distinguishes this method from previous cross-attribute constructions. Specifically, this section explicitly constrains the generated text to be based on the common values ​​and interdependent characteristics of both attributes, rather than describing a single attribute, nor listing them independently or sequentially: when both are positive categories, the text must reflect mutually reinforcing and mutually beneficial background features; when both are negative categories, it must reflect superimposed restrictive background features.

[0059] Through the above process, this section generates corresponding attribute text for each social attribute. Subsequently, the GPT-4.1 model assigns the attribute text to appropriate fields in the neutral medical record and integrates it with the original field content to construct a fused dataset. This method injects controllable, fine-grained social attribute effects while maintaining the integrity of the patient's original background information, ensuring the semantic coherence of the generated fused dataset and making the observation of attribute effects reliable and interpretable. Thus, this section obtains three key datasets: the attribute text set, the fused dataset, and the neutral dataset.

[0060] Based on the above data, this section further designs an analysis method based on cosine similarity, demonstrating its methodological effectiveness and applicability from the perspective of semantic shift. Specifically, this section first combines each medical record and its corresponding attribute text in the neutral dataset by direct concatenation, rather than semantic fusion, to construct a concatenated dataset. The purpose of this dataset is to introduce attribute-related information as a control setting while preserving the original semantic structure of the neutral medical records to the greatest extent possible. Subsequently, this section uses the advanced large-scale text vectorization model text-embedding-3-large to uniformly map the four datasets—the attribute text set, the fused dataset, the neutral dataset, and the concatenated dataset—to the same vector space, enabling a quantitative analysis of the comparability of semantic relevance between different texts. On this basis, this section selects vector pairs with clear analytical objectives, calculates their sample-level cosine similarity, and summarizes and averages the results as the overall similarity index of the corresponding text types in the semantic space. Among them: cosine similarity sim1 between the neutral dataset and the fused dataset, and cosine similarity sim2 between the neutral dataset and the concatenated dataset, are used to measure the degree of preservation of the original neutral semantics during the introduction of attribute information; cosine similarity injection1 between the attribute text set and the fused dataset, cosine similarity injection2 between the attribute text set and the neutral dataset, and cosine similarity injection3 between the attribute text set and the concatenated dataset are used to evaluate the degree to which the attribute text is semantically absorbed and internalized in the overall input.

[0061] Unlike previous methods that relied on evaluation models and their results, the analytical approach presented in this section is not constrained by downstream task settings, offering flexibility and strong generalization capabilities while effectively avoiding interference from individual values ​​or extreme cases. Furthermore, this section provides clear semantic-level analysis, demonstrating excellent interpretability. In summary, this section provides a quantitative and reproducible systematic basis for demonstrating the advantages of the multi-agent collaborative paradigm.

[0062] 5. Constraint diagnosis and assessment.

[0063] Figure 5 This section demonstrates the workflow. First, the large language model being evaluated performs a constrained diagnostic task on a neutral dataset and all fused datasets. This means the model can only output a unique, most probable primary diagnosis, without providing accompanying diagnoses, reasoning processes, or other irrelevant content. After diagnosis, a corrective comparison function is performed: for each diagnosis, it is directly compared to the corresponding standard answer. If they match perfectly, the diagnosis is considered correct; if they don't match, a stable GPT-4o-mini model is used to map the diagnosis and standard answer to names conforming to the 11th edition of the International Classification of Diseases (ICD-11) to eliminate differences caused by different representations. The comparison is then performed again; if they match perfectly, the diagnosis is correct; otherwise, it is incorrect. Based on these results, the model is evaluated from two dimensions: performance metrics and changes in reasoning style.

[0064] For performance metrics, this invention first calculates the accuracy (ACC). Specifically, a binary variable is defined: if the large language model is determined to be correctly diagnosed, then... ,otherwise ,in Indicates the sequence number of the medical record. For the first A binary variable for each medical record at this stage, among which and It is an integer. This represents the total number of medical records in the dataset containing that medical record. Based on this, the accuracy (ACC) calculation formula is: .

[0065] This invention further calculates the relative accuracy rate. Specifically: Let the accuracy of the large language model on the neutral dataset be... ; and for a certain social attribute The accuracy of the large language model on the corresponding fused dataset is Based on this, the relative accuracy of the large language model on this social attribute... The calculation formula is: .

[0066] in, The possible values ​​of all social attributes include: positive economic attributes, negative economic attributes, positive educational attributes, negative educational attributes, Eastern cultural attributes, Western cultural attributes, positive economic and positive educational attributes, and negative economic and negative educational attributes.

[0067] Indicators of reasoning style variation include consistency rate, KL divergence, multidimensional scaling (MDS), and t-distributed random neighborhood embedding (t-SNE). The consistency rate measures the consistency of diagnostic results among models under the same input case conditions. A higher consistency rate indicates stronger consistency in diagnostic decisions among models; a lower consistency rate indicates more significant differences in diagnostic decisions among models. The evaluation method involves: for each dataset, statistically analyzing whether the diagnostic outputs of major language models are identical across all samples in that dataset, and constructing a pairwise consistency rate matrix between the major language models. The consistency rate is defined as the proportion of two major language models that output completely identical diagnostic results in all comparable samples. This consistency rate matrix can quantitatively characterize the degree of consistency in diagnostic decisions among different models under the same input conditions, thus enabling the analysis of the similarity and differences in reasoning styles among major language models.

[0068] The KL divergence metric is used to quantify the differences in the distribution of diagnostic results among different large language models at the overall probability distribution level. A larger value indicates more significant differences in overall diagnostic preferences and reasoning styles among the large language models, while a smaller value indicates more subtle differences. The evaluation method is as follows: For each dataset, firstly, the diagnostic categories and their frequencies output by each major language model on that dataset are statistically analyzed, and a corresponding probability distribution of diagnostic results is constructed accordingly. Then, the KL divergence of the pairwise probability distributions of diagnostic results between models is calculated, forming a KL divergence matrix between models, thus characterizing the differences in reasoning styles among different models at the distribution level.

[0069] Both MDS and t-SNE methods are used to perform low-dimensional mapping of the distribution relationships of diagnostic results among large language models, enabling structured analysis of model inference differences. Specifically, firstly, the probability distributions of diagnostic results from various language models on the same dataset are statistically analyzed, and a measure of distributional difference between each pair of models is calculated, forming a distance matrix between models. Here, the distance is also measured using KL divergence. Then, the MDS method is used to perform two-dimensional embedding on this distance matrix to preserve the global distance relationships between models as much as possible. Simultaneously, the t-SNE method is used to directly perform nonlinear dimensionality reduction on the model diagnostic distributions to highlight the local similarities between models. Through this low-dimensional representation, the relative positions of different models in space can intuitively reflect the differences in their inference styles. The closer the spatial distance, the more similar the distributions of the models' inference results; the farther the spatial distance, the more significant the differences in inference between models.

[0070] It is worth emphasizing that although the above methods are all existing technologies, through their organic combination, this invention constructs a systematic evaluation method that combines quantitative assessment and interpretable analysis. Specifically, the combined use of the consistency rate index and the KL divergence index enables this invention to comprehensively analyze model inference differences at both the sample and distribution levels. Based on this, MDS and t-SNE methods are introduced to visualize these distributional differences in a low dimension, intuitively presenting the clustering structure and similarity relationships of the model inference results. Through this combination of methods, this section not only characterizes changes in model inference style but also provides reproducible visual evidence at the semantic and structural levels, thereby significantly improving the comprehensiveness and reliability of the analysis.

[0071] This invention systematically evaluates six different large language models, including: Qwen2.5-Max, Deepseek-V3, Deepseek-V3.2, Gemini2.5-flash, GPT5-mini, and o4-mini.

[0072] first, Figure 6 and Figure 7 The evaluation results of the cosine similarity-based analysis method mentioned in the data construction section are presented. First, sim1 outperforms sim2 on all attribute datasets, indicating that the semantic-based fusion method better preserves the original neutral medical semantics while introducing attribute shifts. In contrast, directly concatenating attribute text using traditional methods leads to more significant semantic loss. Second, the values ​​of injection1, injection2, and injection3 are significantly lower than those of sim1 and sim2, indicating that the model's semantic representation undergoes structural changes due to the injection of different attributes. Furthermore, the trend across all attribute datasets shows injection3 > injection1 > injection2, demonstrating that the fusion method effectively absorbs and internalizes attribute information, allowing it to influence the overall text in the semantic space. These analyses show that the method of this invention performs well in maintaining the integrity of the original neutral semantics and effectively introducing attribute semantics. Compared to direct concatenation, the fusion method is superior in balancing neutrality preservation and attribute absorption.

[0073] Figure 6 and Figure 7 Each subgraph corresponds to a social attribute. The horizontal axis of each subgraph, from left to right, represents the five indicators sim1, sim2, injection1, injection2, and injection3, while the vertical axis represents the numerical value of the indicator.

[0074] Next, the diagnostic accuracy of each language model was calculated on the neutral dataset and the complete fusion dataset. The results are as follows: Figure 8As shown, even the most advanced mainstream large language models currently available still exhibit low accuracy in diagnostic tasks. Even the best-performing o4-mini model falls below 60% accuracy, failing to meet the minimum requirement. This result indicates significant room for improvement in the diagnostic performance of existing large language models. Further comparison reveals that while the introduction of social attribute information does alter the diagnostic accuracy of large language models compared to the neutral dataset, the overall change is not significant. This is primarily due to the limited diagnostic capabilities of large language models, making it difficult to directly reflect the impact of social attributes in the final performance metrics. Therefore, this invention further conducts fine-grained analysis at the reasoning style level, a reasonable and necessary extension to address the aforementioned phenomena.

[0075] Figure 8 Each subgraph in the graph corresponds to a large language model being evaluated. The horizontal axis of each subgraph represents the accuracy value as a percentage, and the vertical axis represents the social attribute value, which corresponds one-to-one with the legend.

[0076] To further explore the impact of social attributes on the diagnostic performance of large language models, Figure 9The relative accuracy values ​​of various language models on different fusion datasets are presented. The results show that Qwen2.5-Max and Deepseek-V3 exhibit a highly consistent overall trend under different social attribute conditions: the former consistently shows performance improvement, while the latter consistently shows performance decline. In contrast, the performance of other models shows more significant heterogeneity. Overall, in most cases, positive social attributes are more likely to be accompanied by improved diagnostic performance, while negative attributes are more likely to lead to performance decline. However, this invention also observes a series of counterintuitive phenomena: positive attributes do not necessarily bring performance gains, and in some cases, they may even lead to performance degradation; similarly, negative attributes do not necessarily weaken diagnostic ability, and in some cases, they may even promote diagnostic results. This suggests that social attributes do not directly affect diagnostic results through simple positive or negative emotional polarity, but rather more likely through changing the way the model models the patient's prior risk and the organizational structure of the diagnostic reasoning path. Because different models employ significantly different internal strategies when integrating social attribute information, the same attribute may have opposite effects in different models. Furthermore, even within the same model, the positive or negative meaning of an attribute does not necessarily correspond to performance improvement or decline. This further illustrates that the impact of social attributes on model behavior exhibits significant model-dependent and context-dependent characteristics. Looking at specific attribute types, economic attributes typically show greater fluctuations, and their impact patterns are relatively clear. Specifically, positive economic attributes exhibit a stable and significant positive impact in all models except Deepseek-V3, and are often the largest contributor to performance enhancement compared to other attributes; while negative economic attributes show only a slight negative or near-zero impact in most models. This indicates that for current mainstream advanced large language models, positive economic attributes are a relatively common and reliable performance enhancement signal, while negative economic attributes, due to their lower information content and semantic clarity, are more prone to introducing noise and bias. In contrast, educational and cultural attributes show greater uncertainty in both the direction and magnitude of their impact, failing to exhibit the consistent and clear trends seen in economic attributes. This difference may stem from variations in the pre-training corpora, architectural design, and the understanding and encoding of relevant social concepts among different models. Furthermore, under the cross-attribute setting, the attribute synergy effect is clearly evident. For example, when positive economic and positive educational attributes are simultaneously combined, they typically result in the highest accuracy increase in this experiment, with Gemini2.5-flash achieving a performance improvement of 7.51%. Conversely, the combination of two negative attributes often leads to a more significant performance decline. This result indicates that the influence of social attributes is not a simple linear additive effect, but rather exhibits a clear synergistic enhancement or inhibition effect across specific dimensions.

[0077] Figure 9Each subgraph in the graph corresponds to a large language model being evaluated. The horizontal axis of each subgraph represents the relative accuracy value as a percentage, and the vertical axis represents the social attribute value, which corresponds one-to-one with the legend.

[0078] at last, Figures 10 to 15 The reasoning style of each model on each attribute is evaluated from four dimensions: consistency rate, KL divergence, MDS, and t-SME.

[0079] Figure 10 and Figure 11 The consistency rates of different models under various social attribute conditions are presented. Overall, the consistency rate between models within the same series is significantly higher than that between models across series. For example, Deepseek-V3 and Deepseek-V3.2, and GPT5-mini and o4-mini maintain consistency rates exceeding 0.6 across all attribute dimensions. In contrast, cross-model consistency is generally lower, with most models failing to reach 0.6. This indicates that even advanced large models exhibit significant differences in their inference paths and judgment results when faced with inputs from different social attributes. Further analysis from the attribute dimension reveals that the overall consistency of models is relatively better under positive attributes than under negative attributes, with Qwen2.5-Max showing a more pronounced difference compared to other models. Regarding cultural attributes, the consistency distribution of Western cultural attributes is more even than that of Eastern cultural attributes, and Qwen2.5-Max exhibits a more significant deviation. After introducing cross-attributes, the consistency rates between models did not shift significantly. These results provide direct insights for model selection in medical diagnosis or ensemble learning.

[0080] Figure 10 and Figure 11 Each subgraph corresponds to a social attribute, where the horizontal and vertical axes represent the names of the evaluated models, and the grayscale represents the numerical value.

[0081] Figure 12 and Figure 13The KL divergence results are presented, analyzing inference styles from a distributional perspective. It can be seen that models within the same series have similar distributions, while the distributions of different series differ significantly. Furthermore, the differences in patterns caused by the same social attribute among models exhibit heterogeneity. For example, under the negative economic attribute condition, the KL divergence between the Gemini2.5-flash and GPT5-mini series increases significantly; under the negative education attribute condition, the KL divergence between Deepseek-V3.2 and other models consistently maintains a moderately high level of difference; under the cross-attribute condition of positive education and positive economy, the KL divergence between Qwen2.5-Max and GPT5-mini reaches as high as 6.51, indicating that the superposition of dual positive attributes significantly amplifies the differences in model distributions; under the Eastern culture attribute condition, the overall KL divergence among models is low, suggesting that the models have a relatively consistent overall understanding of this cultural context. Further analysis using sample-level consistency reveals that KL divergence effectively reveals fundamental differences at the distribution level, particularly when the KL divergence value exceeds 5. Consistency, on the other hand, supplements the model's consistency in judging specific samples. The synergistic analysis of these two indicators is crucial in scenarios with overlapping attributes. For example, under negative economic and educational conditions, the KL divergence value increases further, indicating that the superposition of multiple negative attributes amplifies differences between models. This provides important insights for model selection in high-risk decision-making scenarios. Therefore, in practice, model combinations with moderate KL divergence values ​​can be prioritized for system integration to balance diversity and stability. High KL divergence model combinations are more suitable as cross-validation tools for key decisions, and risk control should be implemented in conjunction with consistency. Overall, this combined analysis method using both distribution-level and sample-level indicators achieves a systematic characterization from micro-level consistency to macro-level differences, providing a more comprehensive and operational analytical framework for model evaluation.

[0082] Figure 12 and Figure 13 Each subgraph in the graph corresponds to a social attribute, where the horizontal and vertical axes represent the names of the models being evaluated, and the grayscale represents the numerical value.

[0083] Figure 14 and Figure 15The results of model representation analyses based on MDS and t-SNE are presented respectively. MDS results reveal the impact of different attributes on the model embedding space at the overall structural level: under the economic attribute, the impact of positive and negative conditions on the overall relative position of the model is relatively limited; under the education attribute, the model exhibits a style reversal under positive and negative conditions, reflecting the model's high sensitivity to the polarity of education-related values; under the cultural attribute, Western cultural attributes are generally more loosely distributed, while Eastern cultural attributes are more likely to form stable model clusters, indicating that the cultural orientation of the training corpus shapes the semantic alignment of the model to some extent. Furthermore, under the intersection of negative economic and negative education attributes, the model space structure forms a new geometric layout distinct from that of a single attribute, indicating that the effect of composite attributes is not linearly superimposed but may induce new semantic topological structures. Correspondingly, t-SNE analysis provides supplementary evidence for the above phenomena from the perspective of local neighborhood relationships, and is consistent with the overall trend revealed by MDS. Compared to the overall smooth structure of MDS, t-SNE further amplifies the nonlinear separation phenomenon. For example, the model distribution is more dispersed under Western cultural attributes, while it exhibits a more stable clustering structure under Eastern cultural attributes; negative economic attributes make the distribution within and between classes more loose.

[0084] Figure 14 and Figure 15 Each subgraph corresponds to a social attribute, and each point corresponds to the name of the model being evaluated, with a one-to-one correspondence between the points and the legend.

[0085] All large language models used in this invention were obtained via API calls. The values ​​of the two key parameters, temperature and random seed, were set to 0.0 to ensure reproducibility of the results. All other parameters were set to their default values. All experiments in this invention were implemented using Python 3.8.

[0086] Before use, the API key needs to be set to the user's personally registered key. During use, simply follow the flowcharts for each module, set the name of the large language model to be evaluated, and the dataset path; this invention will automatically complete all steps.

[0087] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0088] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple steps or stages, which are not necessarily completed at the same time, but may be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but may be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0089] In one embodiment, a computer system is provided, which may be a server. The computer system includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data used in the methods described above. The network interface communicates with external terminals via a network connection. The computer program is executed by the processor to implement the methods described above.

[0090] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0091] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention, and no reference numerals in the claims should be construed as limiting the scope of the claims.

[0092] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A method for assessing the impact of social attributes on the diagnostic reasoning process of large models, characterized in that, include: The original medical record dataset was standardized by removing samples associated with demographic attributes and organizing it into a hierarchical standard format to obtain a standardized dataset. The standardized dataset is neutralized by replacing the population attribute information and verifying it through reverse calculation to obtain a neutral dataset. For multiple social attributes, attribute text is generated based on multiple sub-dimensions, and corresponding fields of medical records in the neutral dataset are embedded through semantic fusion to construct a fused dataset corresponding to each social attribute. A large language model is used to perform a constrained diagnostic task that outputs only a unique primary diagnosis on the neutral dataset and each fused dataset to obtain diagnostic results. Based on the diagnostic results, performance indicators and reasoning style change indicators are calculated to assess the impact of social attributes on the diagnostic reasoning process of large language models.

2. The method for assessing the impact of social attributes on the diagnostic reasoning process of a large model according to claim 1, characterized in that, The standardization process for the original medical record dataset, which involves removing samples associated with demographic attributes and organizing them into a hierarchical standard format to obtain a standardized dataset, specifically includes the following steps: S11, based on rules and regular expressions, remove medical records from the original medical record dataset that are irrelevant to the diagnostic task, and label the source of the remaining medical records; S12, treat medical records from each source as a group, use a text encoding model to convert the medical records from the current source into semantic vectors, calculate the cosine similarity between the semantic vectors of any two medical records, and determine that the two cases are semantic duplicate pairs when the cosine similarity between the two cases is higher than a set threshold; model the case deduplication problem as an undirected graph, identify duplicate case clusters by calculating connected components, and perform in-group deduplication. S13, using large language models to identify and filter medical records that are associated with demographic attributes; S14. Using a large language model, the filtered medical records are organized into a standard medical record format, all clinically significant information is extracted, and the information is classified and organized into a hierarchical form that includes patient behavior information, physical examination results, and auxiliary examination results. S15, Perform an integrity check on each medical record with a standard medical record format, and remove medical records with empty top-level fields or missing information in lower-level fields. S16 integrates medical records from different sources that have undergone integrity checks and deduplication, resulting in a standardized dataset.

3. The method for assessing the impact of social attributes on the diagnostic reasoning process of a large model according to claim 2, characterized in that, The process of modeling the deduplication problem of cases as an undirected graph, identifying clusters of duplicate cases by calculating connected components, and performing intra-group deduplication specifically includes: Each medical record is treated as a node in an undirected graph. If two medical records are semantically duplicated, there is an undirected edge between the corresponding nodes. Duplicate case clusters are identified by calculating the connected components of the undirected graph. Only one representative medical record is retained for each connected component, and the rest are removed.

4. The method for assessing the impact of social attributes on the diagnostic reasoning process of a large model according to claim 1, characterized in that, The process of neutralizing the standardized dataset, replacing population attribute information, and verifying it through reverse engineering to obtain a neutral dataset specifically includes: The explicit or implicit demographic attribute values ​​of medical records in the standardized dataset are replaced with neutral placeholders, pronoun elimination is performed, and the processed medical records are input into a large language model for back-inference verification. Medical records that can still be inferred from demographic attributes are removed, resulting in a neutral dataset.

5. The method for assessing the impact of social attributes on the diagnostic reasoning process of a large model according to claim 1, characterized in that, The social attributes include: positive economic attributes, negative economic attributes, positive educational attributes, negative educational attributes, Eastern cultural attributes, Western cultural attributes, the intersection of positive economic and positive educational attributes, and the intersection of negative economic and negative educational attributes.

6. The method for assessing the impact of social attributes on the diagnostic reasoning process of a large model according to claim 5, characterized in that, The method involves generating attribute text based on multiple sub-dimensions for various social attributes, and embedding corresponding fields from medical records in a neutral dataset through semantic fusion to construct a fused dataset for each social attribute. Specifically, this includes: A large language model is used as a neutral agent to generate attribute text for each social attribute. During generation, conditional constraints are applied based on the value of the social attribute: when the economic and educational attributes are positive, the corresponding attribute text reflects favorable social characteristics; when the economic and educational attributes are negative, the corresponding attribute text reflects unfavorable social characteristics. For cultural attributes, corresponding characteristics are presented according to regional differences. The attribute text contains only social attribute-related content and does not include medical facts, disease information, treatment results, or prognostic implications. The attribute text is systematically characterized based on multiple sub-dimensions, including: educational attributes such as education level, education quality, education content, functional skills, and education continuity; economic attributes such as economic level, income stability, income source, purchasing power, wealth status, and asset status; and cultural attributes such as lifestyle, social norms, traditional festivals, community participation, values, hobbies, and social identity. When generating attribute text, the patient behavior field of medical records in the neutral dataset is used as auxiliary context; When generating attribute text for cross attributes, it is generated based on the common values ​​of the cross attributes; The generated attribute text is assigned to the corresponding fields of medical records in the neutral dataset and integrated with the original content of the fields to form a fused dataset. The semantic preservation of the original content and the semantic absorption of the attribute text are analyzed based on cosine similarity during the integration process.

7. The method for assessing the impact of social attributes on the diagnostic reasoning process of a large model according to claim 6, characterized in that, The degree of semantic preservation of the original content and the degree of semantic absorption of the attribute text during the integration process based on cosine similarity analysis specifically include: Each medical record in the neutral dataset is directly concatenated with its corresponding attribute text to construct a concatenated dataset; By using a text vectorization model, the collection of attribute texts, the fused dataset, the neutral dataset, and the concatenated dataset are uniformly mapped to the same vector space; The cosine similarity sim1 between the neutral dataset and the fused dataset, and the cosine similarity sim2 between the neutral dataset and the spliced ​​dataset are calculated to measure the degree of semantic preservation of the original content in the process of introducing social attribute information. The cosine similarity injection1 between the attribute text set and the fused dataset, the cosine similarity injection2 between the attribute text set and the neutral dataset, and the cosine similarity injection3 between the attribute text set and the concatenated dataset are calculated to measure the degree to which the attribute text is semantically absorbed.

8. The method for assessing the impact of social attributes on the diagnostic reasoning process of a large model according to claim 1, characterized in that, It also includes: for each diagnosis, it is first directly compared with the corresponding standard answer. If they are completely consistent, the diagnosis is considered correct. If they are inconsistent, a large language model is used to map the diagnosis and the standard answer to names that conform to the International Classification of Diseases (ICD) and then compare them again. If they are completely consistent, the diagnosis is considered correct; otherwise, the diagnosis is considered incorrect.

9. The method for assessing the impact of social attributes on the diagnostic reasoning process of a large model according to claim 1, characterized in that, The performance metrics include accuracy and relative accuracy; the inference style variation metrics include consistency rate, KL divergence, multidimensional scaling analysis method, and t-distributed random neighborhood embedding method.

10. A computer system comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 9.

Images