Method and system for ethical examination during scale topic generation by large language model

Abstract

The invention relates to an ethical examination method and system for generating scale questions by a large language model. The method comprises the following steps: firstly, constructing a multi-dimensional dynamic support system, forming a structured labeling system by defining four types of core ethical conflict dimensions, constructing a dynamic training data set, and training an ethical risk classification model by adopting a multi-task learning framework; secondly, deploying a real-time linkage review module, embedding the review module in a large language model text generation and output link, and establishing and generating a single question triggering review mechanism to realize synchronization of generation and review; and finally, ethical conflict processing and compliance adaptation are executed, a traceability report is generated through an interpretability algorithm, a culture knowledge base is called to generate an alternative expression, and a compliance scale is finally output in combination with a cross-regulation adaptation engine and a compliance degree scoring system. The method solves the problems of low efficiency, weak generalization ability and insufficient compliance of traditional examination, and is suitable for ethical management and control of scale topic generation in the fields of psychological assessment and the like.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, in particular to an ethical review method and system for large language model generating scale questions. BACKGROUND

[0002] In the prior art, when a large language model is used to generate scale questions for psychological evaluation, medical research and other fields, its ethical review mainly relies on a static rule library and manual post-review. This method has three major inherent defects: first, the static rule library cannot cover emerging ethical scenarios such as AI algorithm discrimination and cross-border data privacy conflicts, resulting in weak risk identification generalization ability and incomplete coverage; second, the review process is severely lagging, usually requiring manual review after the scale is completely generated, resulting in low efficiency, with an average review period of 2 weeks for a 200-question scale, and high manual error rate; third, the existing method lacks explainability and adaptive compliance ability, neither can it clearly trace the semantic root of ethical conflicts to assist decision-making, nor can it dynamically adapt to the differentiated requirements of HIPAA, GDPR and other multi-system regulations. These problems seriously restrict the large-scale and high-fidelity application of AI-generated scales. SUMMARY

[0003] In view of the technical problems in the above background art, the present application proposes an ethical review method and system for large language model generating scale questions.

[0004] In a first aspect, the present application proposes an ethical review method for large language model generating scale questions, comprising the following steps: defining a structured ethical conflict labeling system, the structured ethical conflict labeling system including four ethical conflict dimensions of cultural discrimination, legal violation, privacy leakage and value bias; collecting multi-field ethical conflict cases, and labeling the multi-field ethical conflict cases using the structured ethical conflict labeling system to construct a training data set; training a model using the training data set to obtain an ethical risk classification model; In the text generation process of the large language model, the output text sequence is monitored in real time, and when a termination symbol representing the completion of single question generation is parsed in the text sequence, a trigger signal is generated immediately; In response to the trigger signal, a pre-trained ethical risk classification model is called to detect ethical conflicts in the currently generated single question; For questions that identify ethical conflicts, generate a traceability report through an explainability algorithm and call a cultural knowledge base to generate alternative representations, filter the alternative representations using a compliance score system and a cross-regulation adaptation engine, and finally output a compliant scale.

[0005] Preferably, the calling of the pre-trained ethical risk classification model to perform ethical conflict detection on the current generated single question includes: calling a pre-trained ethical risk classification model to obtain a probability distribution of the to-be-reviewed question in a plurality of predefined ethical conflict dimensions; selecting the highest value dimension probability from the probability distribution, comparing the highest value dimension probability with a preset conflict determination threshold, and determining that the question has an ethical conflict if the highest value dimension probability is greater than or equal to the threshold; When an ethical conflict is determined, a risk level is calculated based on the probability distribution and a preset risk weight matrix.

[0006] Preferably, the specific method for training the ethical risk classification model is to use a dual-task cooperative training mechanism to set dimension classification and risk level determination as cooperative training targets, and to construct a joint loss function: ; is the loss of the dimension classification task, and Specifically: ; wherein N represents the number of samples in the dynamic training data set; represents the true label of sample i in dimension ; represents the probability that the model predicts that sample i belongs to dimension ; is the loss of the risk level determination task, and Specifically: ; wherein represents the true label of sample i in risk level ; represents the probability that the model predicts that sample i belongs to risk level ; represents the weight of risk level ; and β are preset weight coefficients.

[0007] Preferably, when the termination symbol representing the completion of single question generation appears in the text sequence, a trigger signal is generated immediately. By detecting the termination symbol of the large language model generated text, a trigger signal is generated immediately when the cumulative character number meets the single question length requirement.

[0008] Preferably, the method further comprises the steps of: screening the input question text for compliance before calling the pre-trained ethical risk classification model; selecting whether to continue calling the pre-trained ethical risk classification model to perform ethical conflict detection on the currently generated single-question question based on the compliance determination result, and filtering the current question text, generating alternative representations, or incorporating the scale based on the compliance determination result and the ethical conflict determination result after the ethical conflict detection.

[0009] Preferably, the generation of the traceability report by the explainability algorithm comprises: adopting natural language processing technology to locate and extract the conflict core unit from the question text with ethical conflicts; associating the conflict core unit with the pre-built ethical rule knowledge graph, and calculating its association weight with a specific ethical dimension through a graph neural network; When the association weight is greater than or equal to a preset threshold, it is determined that there is a causal chain between the conflict core unit and the ethical conflict dimension, and a structured ethical conflict traceability report is generated.

[0010] Preferably, the calling of the cultural knowledge base to generate alternative representations comprises: calling the cultural knowledge base to query related cultural taboos and scenario-specific requirements based on the ethical conflict traceability report; Based on the original question and the requirements obtained from the cultural knowledge base, a sequence-to-sequence model is called to generate multiple candidate alternative representations; adopting a semantic vector model to calculate the semantic similarity between each candidate alternative representation and the original question; Based on the semantic similarity, the filtered alternative representations are output.

[0011] Preferably, the filtering of the alternative representations using the compliance degree scoring system and the cross-regulation adaptation engine and the final output of the compliance scale comprise: inputting the filtered alternative representations into the ethical compliance degree scoring system to calculate the comprehensive ethical compliance degree score, and filtering the alternative representations based on the comprehensive ethical compliance degree score; inputting the filtered alternative representations into the cross-regulation adaptation engine, and performing the following operations based on the cross-regulation adaptation engine: automatically matching a target regulation set based on the question scenario; calculating the semantic entailment relationship and the corresponding confidence between each alternative representation and the target regulation clause through a natural language reasoning model; Based on whether the confidence between each alternative representation and the target regulation clause reaches a preset threshold, the regulation compliance verification result of each alternative representation is output; The alternative representations verified by the cross-regulation adaptation engine are summarized and output as the final compliance scale.

[0012] Preferably, the input of the screened alternative representation into the ethical compliance degree scoring system, calculating the comprehensive ethical compliance degree score, and filtering the alternative representation based on the comprehensive ethical compliance degree score comprise: determining the weight of each ethical conflict dimension by analytic hierarchy process; scoring each ethical conflict dimension by using the ethical compliance degree scoring rule engine, and calculating the comprehensive score by weighting the weight obtained by analytic hierarchy process; dividing the alternative representation into low-score questions, medium-score questions and high-score questions based on the comprehensive score; deleting the low-score questions in the output process, calculating the review priority for the medium-score questions for review, and directly passing the high-score questions.

[0013] In the second aspect, the present application also proposes an ethical review system for large language model to generate scale questions, which comprises: An ethical conflict labeling system construction module configured to define a structured ethical conflict labeling system, wherein the structured ethical conflict labeling system comprises four ethical conflict dimensions of cultural discrimination, legal violation, privacy leakage and value bias; A training data set construction module configured to collect multi-field ethical conflict cases, and label the multi-field ethical conflict cases by using the structured ethical conflict labeling system to construct a training data set; An ethical risk classification model training module configured to train a model by using the training data set to obtain an ethical risk classification model; A trigger signal generation module configured to monitor the text sequence output by the large language model in real time during the text generation process of the large language model, and generate a trigger signal as soon as a termination symbol representing the completion of single question generation is parsed in the text sequence; An ethical conflict detection module configured to respond to the trigger signal, call the pre-trained ethical risk classification model, and perform ethical conflict detection on the single question question that has been generated; A compliance scale output module configured to generate a traceability report by using an explainability algorithm for the question in which the ethical conflict is identified, and call a cultural knowledge base to generate an alternative representation, filter the alternative representation by using a compliance degree scoring system and a cross-regulation adaptation engine, and finally output a compliance scale.

[0014] Advantages: 1. The present application can improve the risk identification capability: by adopting double task cooperative training, combining dimension classification and risk level determination, and matching dynamic updating data set, the traditional single task model performance single problem can be solved, and new type of ethical conflict can be identified.

[0015] 2. The application can improve the efficiency of review: embed the review module in the output layer of the large language model, trigger the generation and review synchronization through the terminal symbol, break the traditional lag mode, and adapt to the rapid development of large-scale scale.

[0016] 3. The application can solve the problems of explanation and compliance: use semantic atlas and GNN reasoning to trace the source of conflict, avoid algorithm black box; through the comparison of cultural double database and cross-regulation, adapt to cultural differences and multi-system regulations, and reduce compliance risks.

[0017] 4. The application can balance rigor and flexibility: whole-process closed-loop control, and flexible processing of medium-risk questions through dynamic scoring and manual review, while retaining the core objectives of the scale. BRIEF DESCRIPTION OF DRAWINGS

[0018] The accompanying drawings provide further understanding of the embodiments and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments and serve to explain the principles of the application. It will be readily appreciated that other embodiments and many of the intended advantages of the embodiments will become better understood because they become better understood by reference to the following detailed description. The elements of the drawings are not necessarily to scale. Like reference numbers refer to corresponding similar parts.

[0019] Figure 1 is a flowchart of an ethical review method for a large language model generating scale questions according to the application.

[0020] Figure 2 is a schematic diagram of one specific embodiment of an ethical review method for a large language model generating scale questions that can be applied to the application.

[0021] Figure 3 is a schematic diagram of the training process of an ethical risk classification model in one embodiment of the application.

[0022] Figure 4 is a schematic diagram of the weight of the ethical conflict dimension to the risk level in one embodiment of the application.

[0023] Figure 5 is a schematic diagram of deploying real-time linkage review in one embodiment of the application.

[0024] Figure 6 is a schematic diagram of the compliance output of the ethical conflict question in one embodiment of the application.

[0025] Figure 7 is a traceability report schematic diagram in one embodiment of the application.

[0026] Figure 8 is a schematic diagram of the module structure of the ethical review system for a large language model generating scale questions in one embodiment of the application. DETAILED DESCRIPTION

[0027] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0028] In the description of the present application, the terms "first", "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of "multiple" is two or more, unless otherwise explicitly and specifically limited.

[0029] Figure 1 A flowchart of an ethical review method for generating a scale question by a large language model is shown in the present application, referring to Figure 1 The method specifically includes the following steps: S1, defining a structured ethical conflict labeling system, the structured ethical conflict labeling system including four types of ethical conflict dimensions of cultural discrimination, legal violation, privacy leakage and value bias; S2, collecting multi-field ethical conflict cases, labeling the multi-field ethical conflict cases using the structured ethical conflict labeling system to construct a training data set; S3, using the training data set for model training to obtain an ethical risk classification model; S4, in the text generation process of a large language model, real-time monitoring of the output text sequence, when a termination symbol representing the completion of single question generation is parsed in the text sequence, a trigger signal is generated immediately; S5, in response to the trigger signal, calling a pre-trained ethical risk classification model to detect ethical conflicts of the single question topic that has been generated; S6, for the identified ethical conflict question, generate a traceability report through an explainable algorithm and generate a substitute representation by calling a cultural knowledge base, filter the substitute representation by a compliance scoring system and a cross-regulation adaptation engine, and finally output a compliance scale.

[0030] Figure 2 A schematic diagram of one specific embodiment of an ethical review method that can be applied to large language model generated scale questions of the present application is shown with reference to Figure 2 , comprising the following steps: Step 1: Build a multi-dimensional dynamic support system 1) Define a structured ethical conflict annotation system In building the ethical conflict annotation system, the present application defines four core ethical conflict dimensions and their sub-item examples in detail to ensure the comprehensiveness and accuracy of the annotation. Specifically, it includes cultural discrimination, such as expressions involving religious medical taboos, such as refusing Christian prayer treatment; legal violations, such as unauthorized reference to patient medical record numbers or forced collection of genetic data; privacy disclosure, such as requiring the filling of specific hospital names or exposing details of past mental illness history; and value bias, such as expressions that women are more likely to exaggerate depression symptoms or that depression in the elderly does not require treatment. In addition, each case also needs to be additionally annotated with cross-border attributes, to determine whether it applies to China, the United States, or both, and to annotate the medical data sensitivity level, which is divided into high, medium, and low levels, to further refine the annotation content and assess potential risks.

[0031] 2) Build a dynamic training dataset In the process of building a dynamic training dataset, first, 120,000 medical scale questions containing ethical conflicts were collected from the Lancet Psychiatry journal (2018-2023), the American Psychiatric Association (APA) depression scale library, and ethical review cases from domestic top three hospitals, and were annotated in detail according to the above annotation system. For example, the question "Please provide your medical record number to verify the diagnosis of depression" is annotated as a legal violation, with the sub-item "forced provision of medical record number", the violation basis "HIPAA Article 164.514", the cross-border attribute "dual applicable", and the sensitivity level "high"; while the question "Female patients need to fill in additional fertility history to assess depression" is annotated as a value bias, with the sub-item "gender differentiation requirement", the violation basis "Article 11 of the Women's Rights Protection Law", the cross-border attribute "dual applicable", and the sensitivity level "medium". In order to maintain the timeliness and adaptability of the dataset, 3000 new conflict cases (such as requiring the upload of facial photos for depression expression analysis) were added through crawler technology from the FDA's public medical AI ethical controversy cases in 2024 and the mental health scale compliance guidelines released by the Chinese Health Commission, and were included in the dataset for dynamic updating.

[0032] 3) Train an ethical risk classification model Model architecture adjustment: the encoder adopts the BioBERT-base pre-training model in the medical field (adapted to the text semantics), and the output layer retains the dimension classification and risk level determination double tasks. Training parameter execution: The dynamic training dataset is divided into training set, validation set and test set by 8:1:1. The AdamW optimizer is adopted, the initial learning rate is 1e-4, the weight decay is 1e-5, the batch size is 32, and the validation set loss converges after 100 iterations. Model performance verification: the output result on the test set meets: 4) Dimension detection accuracy: the average value of legal violations, privacy leaks, value bias, and cultural discrimination is greater than or equal to 98%; The generalization recognition rate of unmarked new ethical conflicts is greater than or equal to 85%; Inference delay: the average inference time of a single question is less than or equal to 50ms.

[0033] Combination Figure 3 The training method of the ethical risk classification model is described: Adopting a double-task collaborative training mechanism, dimension classification and risk level determination are set as collaborative training targets, and dynamic training dataset is input. Through the AdamW optimizer, iterative optimization is performed to make the model meet the performance indicators of 4 core dimension detection accuracy greater than or equal to 98%, unmarked new ethical conflict generalization recognition rate greater than or equal to 85%, and inference delay less than or equal to 50ms, solving the problem of weak generalization ability and single performance of traditional single-task models. The specific content is as follows: Calculate the dimension classification task loss Adopting the cross-entropy loss function, the classification error of the model on the 4 core ethical conflict dimensions (cultural discrimination , legal violations , privacy leaks , value bias ) is calculated: N represents the number of samples in the dynamic training dataset; represents the true label of sample i in dimension ; represents the probability that the model predicts that sample i belongs to dimension ; Calculate the risk level determination task loss Adopting the weighted cross-entropy loss function, for high-risk , medium-risk , and low-risk , higher weights are given to high-risk samples ), improve the accuracy of high-risk conflict identification: denotes the true label of sample i in risk level ; denotes the probability that the model predicts sample i belongs to risk level ; denotes the weight of risk level ; Create a joint optimization objective function: By introducing task weight coefficients (w1, w2) ), balance the training priority of two tasks (dimension classification as the basis, risk judgment as the supplement), and construct a joint loss function: During training, minimize When the model simultaneously meets the following three performance indicators: Core dimension detection accuracy : denotes the indicator function; denotes the core dimension that the model predicts sample i belongs to; New ethical conflict generalization recognition rate: M represents the number of unlabeled new ethical conflict samples; , are the predicted probability and true label of new sample j, respectively.

[0034] Inference delay: K is the number of test samples, denotes the time when the tth sample is input into the model; denotes the time when the tth sample outputs the prediction result.

[0035] Step 2: Deploy real-time linkage review module 1) Module embedding and data direct connection Embed the RESTful API interface in the 32nd layer of the Transformer decoder of the large language model (e.g. GPT-4) (the last step before the output layer), and use HTTPS protocol encryption transmission for the interface; Establish a streaming transmission channel, configure the transmission rate to be 1200 characters / second, and test the transmission time of a single text (average 60 characters) to be 0.05 seconds, with no data backlog.

[0036] 2) Terminator trigger mechanism execution When GPT-4 generates the question text "Have you ever visited a hospital for depression symptoms? Please specify the hospital name." The terminator "?" is detected, and the cumulative character count since the last trigger is 45, which is greater than or equal to 5, triggering the review request (Trigger = 1); If the generated text is "Have you ever been depressed?" (only 6 characters), although it contains "?", the cumulative character count does not meet the threshold, and the review is not triggered (Trigger = 0).

[0037] The module detects the terminator of the large language model generated text to determine whether to trigger a single ethical review; only when the generated text appears a terminator and the cumulative character count meets the single question length requirement, the review is triggered to ensure single question review, avoiding multiple question combined review or short text false triggering.

[0038] 3) Ethical risk detection execution Dimension probability output: The review module calls the trained ethical risk classification model to output the dimension probability of the question: such as (cultural discrimination) = 0.02, (law violation) = 0.15, (privacy leakage) = 0.93, (value bias) = 0.01; Then the conflict determination max( ) = 0.93 greater than or equal to 0.85, then Conflict = 1 (there is a privacy leakage conflict); Risk level calculation: Substitute the weight matrix (privacy leakage has a high risk weight w_{3,1}=0.9), calculate =0.9×0.93 / (0.3×0.02+0.9×0.15+0.9×0.93+0.2×0.01)=0.837 / 1.004≈0.83, =0.17, =0; Set different ranges for different risk levels.

[0039] In a specific embodiment, after the review module receives the question text, it first outputs two key results through the pre-trained ethical risk classification model, providing a basis for subsequent review: for the four core ethical conflict dimensions of cultural discrimination, legal violation, privacy leakage, and value bias, it outputs the probability value of the question belonging to each dimension; then, based on the output of each dimension, the conflict determination result is calculated according to the ethical conflict detection formula, which is as follows: 1 means there is a conflict, 0 means there is no conflict, represents the probability value that the topic output by the model belongs to the class dimension; represents the conflict determination threshold, which is 0.85; If it is determined that there is a conflict, the module further calculates the risk level corresponding to the topic, and the formula is: represents the comprehensive probability that the topic belongs to the risk level; represents the weight of the class dimension on the risk level, Figure 4 is an example.

[0040] 4) Preliminary screening of compliance Call Jieba word segmentation on the topic text to get the vocabulary set Search the PostgreSQL regulation taboo word library (including medical special taboo words), and the hospital name belongs to the HIPAA and the “Personal Information Protection Law” jointly prohibited mandatory collection of sensitive medical identification information expressions, determine Compliance=0 (serious violation).

[0041] In a specific embodiment, after determining the risk level, the module combines the preset regulation taboo word library to perform preliminary screening of compliance, so as to avoid topics that obviously violate HIPAA, GDPR and other regulations from entering the subsequent process, and the formula is: 0 means serious violation, 1 means compliance, is the vocabulary set after word segmentation of the topic text, and B represents the regulation taboo word library. According to the values of and Conflict, different operations are performed: If (serious violation): regardless of the value of Conflict, output filtering, and directly exclude the topic; If and (compliant and conflict): output a generated alternative expression, triggering subsequent cultural knowledge base calling; If and (Compliant and Conflict-free): The output passes, and the question enters the final table.

[0042] Figure 5 An example of deploying real-time linkage review.

[0043] Step 3: Perform ethical conflict processing and compliance adaptation Refer to Figure 6 , and take another question generated by GPT-4 "Are female patients more prone to sleep disorder-related depressive symptoms than men?" as an example (Compliance=1, Conflict=1), and perform the following processing: 1) Explainable algorithm generates a traceability report The first stage uses the StanfordCoreNLP natural language processing tool to build a subject-predicate-object core semantic graph, accurately positioning the conflict core unit C_i; The second stage is based on a pre-trained knowledge graph containing a large number of ethical rules and regulatory clause mapping relationships, and calculates the association weight between the core unit and the ethical dimension through GNN, with the formula: W_(i,j)=(Count(C_i∩R_j)) / (Count(C_i)+Count(R_j)-Count(C_i∩R_j)); C_i represents the i-th conflict core unit. The conflict core unit here is accurately extracted from the key semantic units in the question text that exist in ethical conflicts through the semantic graph positioning in the first stage. R_j represents the rule set corresponding to the j-th ethical dimension. Ethical dimensions include cultural discrimination, legal violations, privacy leaks, and value biases, which are four core dimensions, each corresponding to a series of ethical rules or regulatory clauses.

[0044] For example, for the question "Women are more prone to emotional", after building the subject-predicate-object core semantic graph through the StanfordCoreNLP tool, the "more prone to emotional" is a conflict core unit C_i.

[0045] For example, the legal violation ethical dimension j corresponds to the set of all ethical rules and regulatory clauses related to legal violations.

[0046] When W_(i,j) is greater than or equal to 0.8, the conflict causal chain is determined, and a structured traceability report containing conflict dimensions, violated clauses, etc. is finally generated, solving the defect that traditional algorithms cannot explain the source of risk.

[0047] Figure 7 An example of an XML traceability report.

[0048] 2) Cultural knowledge base call generates alternative representation Dual-knowledgebase matching: In one specific embodiment, first, the system receives and parses the structured ethical conflict provenance report generated by the explainability algorithm. Key information in the report, especially the conflict dimension and the conflict core unit, are extracted as instructions for querying the knowledgebase. For example, for a question that has value bias or gender bias, the system extracts its conflict core unit that women are more likely to have depressive symptoms than men. Subsequently, the system initiates the dual-knowledgebase matching mechanism: in the basic cultural norm knowledgebase, it queries the general cultural taboos about gender equality and avoiding gender comparison based on the target region code; at the same time, in the scenario-specific requirement knowledgebase, it queries the specific norms such as the medical psychiatric scale requiring the use of gender-neutral expressions based on the domain code. The purpose of this step is to transform abstract ethical conflicts into specific and executable cultural and scenario writing constraints.

[0049] After obtaining clear cultural and scenario constraints, the system inputs the original question and these constraint conditions into the pre-trained sequence-to-sequence model. In this process, the constraint conditions serve as key prompts for the generation model, guiding it to perform targeted rewriting. The task of the model is to avoid the identified ethical conflict points while remaining faithful to the core measurement intent of the original question. For example, the model is required to rewrite the gender-biased comparison structure into a gender-neutral expression. The sequence-to-sequence model learns based on a large amount of text data and can understand and perform this complex rewriting task, ultimately outputting 3 to 5 candidate replacement expressions that differ in wording but all meet the constraint conditions.

[0050] To ensure that the replacement expressions do not deviate from the measurement goal of the original question, the system conducts strict semantic equivalence verification on the generated candidate expressions. This verification is completed by a semantic vector model: the model converts the original question and each candidate replacement expression into semantic vectors in a high-dimensional space. Then, the system calculates the cosine similarity between the original question vector and each candidate expression vector as a quantitative semantic similarity indicator. Usually, the system sets a high threshold, and only keeps the candidate expressions with a similarity higher than this threshold.

[0051] After semantic similarity screening, the system outputs those replacement expressions that meet the cultural knowledgebase constraints and are highly consistent with the core semantics of the original question. These expressions successfully eliminate the identified ethical conflicts while maximizing the validity of the original question, preparing for subsequent cross-regulation adaptation and final inclusion in the compliant scale.

[0052] In one specific embodiment, the cultural knowledgebase is called to generate replacement expressions, for example: The first part builds a foundation library (MySQL stores 200+ regional cultural taboos, indexed by ISO3166-1) and a scenario library (MongoDB stores 8 field-specific requirements, classified by field code). This realizes the scenario matching of cultural constraints. The second part generates 3-5 alternative expressions based on the Seq2Seq model, and then calculates the semantic similarity using Sentence-BERT. The formula is: Retain expressions that ensure that alternative expressions comply with cultural norms and do not deviate from the core measurement objectives of the original title.

[0053] For example, the Seq2Seq model generates 3 alternative expressions: 1. Have you ever experienced depressive symptoms associated with sleep disorders? 2. Have sleep disorders ever accompanied your depressive symptoms? 3. Have you experienced sleep disorders during depressive episodes? Calculate the semantic similarity using Sentence-BERT: The similarity to the original title is 0.89, 0.91, and 0.87, respectively, all greater than or equal to 0.85. Retain the 3 expressions.

[0054] 3) Ethical compliance score and screening In the first stage, the analytic hierarchy process is used to build a judgment matrix (consistency test CR < 0.1) to determine the dimension weights of legal violations (40%), privacy leaks (30%), cultural discrimination (20%), and value bias (10%); In the second stage, the rule engine with more than 1000 preset rules is used to complete the dimension scoring, and then the comprehensive score is calculated according to the formula: In the third stage, dynamic screening is performed: Automatic deletion, According to the following formula, the review priority is calculated to achieve efficient management and control of machine screening and manual precise review. The formula is as follows: Where C is the core degree of the title and R is the risk level. For example, the rule engine scores 3 alternative expressions: Legal violations: No violation expressions (100 points); Privacy leaks: No privacy collection requirements (100 points); Cultural discrimination: No gender / region discrimination (100 points); Value bias: No biased expressions (100 points); Comprehensive score calculation: (≥ 90 points, no need for manual review).

[0055] 4) Cross-regulation adaptation verification Build an NLI semantic comparison system based on structured storage of regulations. The first part uses a PostgreSQL database to store structured data of 12 regulations (including HIPAA 18 requirements and GDPR 99 clauses). Each clause is labeled with applicable scenarios, consequences of violations, and judgment basis. The second part automatically matches the target regulation set based on the scenario regulation mapping table. The NLI model calculates the entailment relationship between the question and the clause (confidence greater than or equal to 0.9), outputs the compliance / violation / pending verification result, and generates a PDF comparison report, solving the problem that traditional methods cannot adapt to multiple regulations across scenarios.

[0056] In one specific embodiment: Scenario regulation matching: automatically match the regulation set: China's Personal Information Protection Law + US HIPAA; NLI semantic comparison: use RoBERTa-NLI model to calculate the entailment relationship between the alternative representation and the regulation clause, for example: With Article 13 of the Personal Information Protection Law (collection of personal information requires necessity): entailment relationship (compliance, confidence 0.96); With HIPAA Article 164.508 (minimum necessity of medical information): entailment relationship (compliance, confidence 0.95); Generate PDF report: the report contains question ID (MD-017-01), matching regulation name, comparison result and confidence, which can be used by the cross-border medical institution ethics review committee for archiving.

[0057] Finally, the alternative representations with confidence exceeding the preset threshold are summarized, and the final compliance scale is output.

[0058] For further reference Figure 8 , as an implementation of the above-mentioned method, the present application provides an embodiment of an ethics review system when a large language model generates a scale question. The system embodiment corresponds to the method embodiment shown in Figure 1 , and the system can be specifically applied to various electronic devices.

[0059] For reference Figure 8 , an ethics review system when a large language model generates a scale question, comprising: An ethical conflict labeling system construction module 101 configured to define a structured ethical conflict labeling system, the structured ethical conflict labeling system including four ethical conflict dimensions of cultural discrimination, legal violation, privacy leakage, and value bias; The training data set construction module 102 is configured to collect multi-field ethical conflict cases, label the multi-field ethical conflict cases by using the structured ethical conflict labeling system, and construct a training data set; The ethical risk classification model training module 103 is configured to train a model by using the training data set to obtain an ethical risk classification model; The trigger signal generation module 104 is configured to monitor the output text sequence of the large language model in real time during the text generation process of the large language model, and generate a trigger signal as soon as a termination symbol representing the completion of single-topic generation is parsed in the text sequence; The ethical conflict detection module 105 is configured to respond to the trigger signal, call the pre-trained ethical risk classification model, and perform ethical conflict detection on the single-topic question that has been generated; The compliance scale output module 106 is configured to generate a traceability report by using an explainability algorithm for the question in which the ethical conflict is identified, call a cultural knowledge base to generate a replacement representation, filter the replacement representation by using a compliance degree scoring system and a cross-regulation adaptation engine, and finally output a compliance scale.

[0060] As another aspect, the application also provides a computer readable storage medium, which can be included in the electronic device described in the above embodiments, or can exist independently without being assembled into the electronic device. The above computer readable storage medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the method as shown in Figure 1

[0061] ​Note that the computer readable storage medium described in the present application can be a computer readable signal medium or a computer readable storage medium or any combination thereof. The computer readable storage medium may, for example, be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination thereof. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present application, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus or device. In the present application, the computer readable signal medium can include a data signal that propagates in a baseband or as part of a carrier wave, in which the computer readable program code is carried. Such a propagated data signal can take any of a variety of forms, including, but not limited to, an electromagnetic signal, an optical signal, or any suitable combination thereof. The computer readable signal medium can also be any computer readable storage medium that can send, propagate or transfer the program for use by or in connection with the instruction execution system, apparatus or device. The program code contained on the computer readable storage medium can be transmitted or propagated using any suitable medium, including, but not limited to, wireless, wireline, optical fiber, RF, etc., or any suitable combination thereof.

[0062] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0063] The computer program product of the present application can be a computer program embodied on a non-transitory computer readable medium. The body of computer readable program code can be any suitable set of instructions stored in a storage medium, which can be run on a system using an operating system. The system can be a mainframe computer, a desktop computer, a laptop computer, or other programmable computer components that can store program code and execute it. The system can also be a mobile device, such as a smart phone, a tablet, or other mobile device that can store program code and execute it. The system can also be a server, a cloud computing system, or other system that can store program code and execute it. The system can also be a combination of the above systems.

[0064] The specific implementations of the present application described above are not intended to limit the scope of the present application, and any skilled person in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered by the scope of the present application. Therefore, the scope of the present application should be subject to the scope of protection of the claims.

[0065] In the description of the present application, it should be understood that the terms "upper", "lower", "inner", "outer", and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore should not be understood as limiting the present application. The word 'comprising' does not exclude the existence of elements or steps not listed in the claims. The word 'a' or 'an' in front of an element does not exclude the existence of multiple such elements. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that the combination of these measures cannot be used to improve. Any reference signs in the claims should not be interpreted as limiting the scope.

Claims

1. An ethical review method for generating scale items using a large language model, characterized in that, The method includes the following steps: A structured ethical conflict labeling system is defined, which includes four dimensions of ethical conflict: cultural discrimination, legal violations, privacy breaches, and value bias. Collect ethical conflict cases from multiple domains, and use the structured ethical conflict annotation system to annotate the ethical conflict cases from multiple domains in order to construct a training dataset; The model is trained using the training dataset to obtain an ethical risk classification model. During the text generation process of the large language model, the output text sequence is monitored in real time. When a terminator representing the completion of a single question generation is found in the text sequence, a trigger signal is generated immediately. In response to the trigger signal, a pre-trained ethical risk classification model is invoked to perform ethical conflict detection on the currently generated single question. For questions identified as having ethical conflicts, an interpretability algorithm is used to generate a source tracing report and a cultural knowledge base is called to generate alternative statements. The alternative statements are then filtered using a compliance scoring system and a cross-regulatory adaptation engine, and finally a compliance scale is output.

2. The ethical review method for generating scale items using a large language model according to claim 1, characterized in that, The step of calling the pre-trained ethical risk classification model to perform ethical conflict detection on the currently generated single-question questions includes: By calling a pre-trained ethical risk classification model, the probability distribution of the topic to be reviewed on multiple predefined ethical conflict dimensions is obtained; The highest-valued dimension probability is selected from the probability distribution. The highest-valued dimension probability is compared with a preset conflict judgment threshold. If the highest-valued dimension probability is greater than or equal to the threshold, the question is judged to have an ethical conflict. When an ethical conflict is determined to exist, the risk level is calculated based on the probability distribution and the preset risk weight matrix.

3. The ethical review method for generating scale items using a large language model according to claim 2, characterized in that: The specific method for training the ethical risk classification model is as follows: A dual-task collaborative training mechanism is adopted, setting dimensional classification and risk level determination as collaborative training objectives, and constructing a joint loss function: ； For the dimension classification task loss, and Specifically: ； Where N represents the number of samples in the dynamic training dataset; Indicates that sample i is in dimension The real labels on it; This indicates that the model predicts that sample i belongs to dimension i. The probability of; To determine the risk level of the task loss, and Specifically: ； in, This indicates that sample i is at risk level The real labels on it; This indicates that the model predicts sample i belongs to the risk level. The probability of; Indicates risk level The weights; α and β are preset weighting coefficients.

4. The ethical review method for generating scale items using a large language model according to claim 1, characterized in that, The step of generating a trigger signal immediately when a terminator indicating the completion of single-question generation is detected in the text sequence includes: By detecting the terminator of the text generated by the large language model, a trigger signal is generated immediately when the cumulative number of characters meets the single-question length requirement.

5. The ethical review method for generating scale items using a large language model according to claim 2, characterized in that, It also includes the following steps: Before calling the pre-trained ethical risk classification model, the input question text is screened for compliance. Based on the compliance assessment results, decide whether to continue using the pre-trained ethical risk classification model to perform ethical conflict detection on the currently generated single-question questions. After the ethical conflict detection, based on the compliance assessment results and the ethical conflict assessment results, filter the current question text, generate alternative expressions, or include it in the scale.

6. The ethical review method for generating scale items using a large language model according to claim 5, characterized in that, The generation of the source tracing report through the interpretability algorithm includes: Natural language processing technology is used to locate and extract the core units of conflict from texts containing ethical conflicts; The core conflict unit is associated with a pre-built ethical rule knowledge graph, and its association weight with a specific ethical dimension is calculated using a graph neural network. When the correlation weight is greater than or equal to a preset threshold, it is determined that there is a causal chain between the core conflict unit and the ethical conflict dimension, and a structured ethical conflict tracing report is generated.

7. The ethical review method for generating scale items using a large language model according to claim 6, characterized in that, The method of generating alternative expressions by invoking the cultural knowledge base includes: The cultural knowledge base is invoked to query relevant cultural taboos and specific contextual requirements based on the aforementioned ethical conflict tracing report. Based on the original question and the requirements obtained from the cultural knowledge base, the sequence to sequence model is invoked to generate multiple candidate alternative expressions; The semantic vector model is used to calculate the semantic similarity between each candidate alternative statement and the original question; The filtered alternative expressions are output based on the semantic similarity.

8. The ethical review method for generating scale items using a large language model according to claim 7, characterized in that, The process of using a compliance scoring system and a cross-regulatory adaptation engine to filter alternative expressions results in the following compliance scale: The selected alternative statements are input into the ethical compliance scoring system to calculate the comprehensive ethical compliance score, and the alternative statements are filtered based on the comprehensive ethical compliance score. The filtered alternative statements are input into the cross-regulatory adaptation engine, which then performs the following operations: Automatically match the target set of regulations based on the question scenario; Using a natural language reasoning model, the semantic implication relationship and corresponding confidence level between each alternative expression and the target legal clause are calculated. Based on whether the confidence level between each alternative statement and the target regulatory clause reaches a preset threshold, the regulatory compliance verification result of each alternative statement is output. The alternative statements verified through the cross-regulatory adaptation engine will be compiled and output as the final compliance scale.

9. The ethical review method for generating scale items using a large language model according to claim 8, characterized in that, The process of inputting the selected alternative statements into the ethical compliance scoring system, calculating the comprehensive ethical compliance score, and filtering the alternative statements based on the comprehensive ethical compliance score includes: The weights of each dimension of ethical conflict were determined using the analytic hierarchy process (AHP). The ethical compliance scoring rule engine is used to score each dimension of ethical conflict, and the weighted average score is calculated using the weights obtained by the analytic hierarchy process. Based on the overall score, the alternative statements are divided into low-scoring, medium-scoring, and high-scoring questions; During the output process, low-scoring questions are deleted, medium-scoring questions are assigned a review priority and reviewed, while high-scoring questions are passed directly.

10. An ethical review system for generating scale items using a large language model, characterized in that, The system includes: The ethical conflict labeling system construction module is configured to define a structured ethical conflict labeling system, which includes four dimensions of ethical conflict: cultural discrimination, legal violations, privacy breaches, and value bias. The training dataset construction module is configured to collect ethical conflict cases from multiple domains and to annotate these cases using the structured ethical conflict annotation system in order to construct the training dataset. An ethical risk classification model training module is configured to train the model using the training dataset to obtain an ethical risk classification model. The trigger signal generation module is configured to monitor the output text sequence of the large language model in real time during the text generation process. When a terminator representing the completion of single-question generation is found in the text sequence, a trigger signal is generated immediately. The ethical conflict detection module is configured to, in response to the trigger signal, invoke a pre-trained ethical risk classification model to perform ethical conflict detection on the currently generated single question. The compliance scale output module is configured to generate a source tracing report for questions that are identified as having ethical conflicts, and to generate alternative statements by calling a cultural knowledge base. The alternative statements are then filtered using a compliance scoring system and a cross-regulatory adaptation engine, and finally the compliance scale is output.

Images