Method, device, equipment, storage medium and program product for text quality evaluation
By assessing the completeness, logical relationships, and accuracy of hazard identification texts, the text quality issues caused by differences in the capabilities of safety inspectors were resolved, thereby improving the accuracy and reliability of risk analysis.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- TSINGHUA UNIVERSITY
- Filing Date
- 2024-04-30
- Publication Date
- 2026-07-24
AI Technical Summary
Due to differences in the capabilities of different safety inspectors, hazard identification texts may contain missing hazard elements, logical errors, and reasoning mistakes, affecting the accuracy of risk analysis.
By acquiring hazard elements, logical connectors, and hazard consequences from hazard identification texts, their completeness, logical relationships, and accuracy are assessed. A quality assessment result is generated using grammatical analysis, logical relationship identification, and classification models.
It enables multi-dimensional quality assessment of hazard identification texts, improves the accuracy and reliability of risk analysis, and provides quality assessment interpretation texts to facilitate improvement.
Smart Images
Figure CN118569243B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, specifically to a method, apparatus, device, storage medium, and program product for text quality assessment. Background Technology
[0002] In industrial environments, safety inspectors typically need to observe the site, identify and deduce potential hazards, and record information about these hazards and their consequences in a hazard identification report. Then, by analyzing, compiling, and issuing warnings based on the hazard identification reports, risk management can be implemented for the specific site environment.
[0003] Due to differences in the capabilities of different safety inspectors, the hazard identification texts recorded may contain issues such as missing hazard elements, logical errors, and reasoning mistakes, leading to inaccurate subsequent risk analyses.
[0004] Therefore, how to assess the text quality of hazard identification texts is a problem that needs to be solved. Summary of the Invention
[0005] The purpose of this application is to provide a method, apparatus, device, storage medium, and program product for text quality assessment, in order to solve the problem of how to assess the text quality of hazard identification text.
[0006] On one hand, this application provides a method for text quality assessment, the method comprising: acquiring hazard identification text to be assessed; obtaining a completeness score, a logical relationship score, and an accuracy score of the hazard identification text based on the hazard elements, logical connectives, and hazard consequences recorded in the hazard identification text; wherein, the completeness score indicates whether the element categories of the hazard elements in the hazard identification text are complete, the logical relationship score indicates whether the logical relationship of the hazard identification text is correct, and the accuracy score indicates whether the hazard consequences of the hazard identification text are accurate; and obtaining a quality assessment result of the hazard identification text based on the completeness score, the logical relationship score, and the accuracy score.
[0007] In one embodiment, based on the hazard elements, logical connectors, and hazard consequences recorded in the hazard identification text, a completeness score, a logical relationship score, and an accuracy score are obtained for the hazard identification text. This includes: assessing the completeness of the hazard identification text based on the hazard elements in the hazard identification text to obtain a completeness score; assessing the logical relationships of the hazard identification text based on the logical connectors and hazard elements in the hazard identification text to obtain a logical relationship score; and assessing the accuracy of the hazard identification text based on the hazard consequences in the hazard identification text to obtain an accuracy score.
[0008] In one implementation, the hazard identification text is assessed for completeness based on the hazard elements in the hazard identification text to obtain a completeness score. This includes: parsing each grammatical component in the hazard identification text to obtain a syntax analysis tree; extracting hazard elements from the hazard identification text based on the element extraction rules corresponding to each element category and the syntax analysis tree; the element extraction rules are set according to the grammatical characteristics of the hazard elements corresponding to the element categories; and obtaining a completeness score based on the element scores corresponding to the element categories of the hazard elements.
[0009] In one embodiment, the logical relationships in the hazard identification text are evaluated based on logical connectives and hazard elements to obtain a logical relationship score. This includes: extracting logical connectives from the hazard identification text; determining the text positions of the logical connectives in the hazard identification text; determining the logical relationships between different hazard elements based on the logical connectives and text positions; obtaining the set logical relationships between the element categories corresponding to different hazard elements; and obtaining a logical relationship score based on the matching degree between the logical relationships and the set logical relationships.
[0010] In one implementation, the accuracy of the hazard identification text is evaluated based on the potential consequences in the hazard identification text to obtain an accuracy score, including: inputting the hazard identification text into a classification model to obtain predicted consequences; extracting the potential consequences contained in the hazard identification text; and obtaining an accuracy score based on the matching degree between the predicted consequences and the potential consequences.
[0011] In one embodiment, the consequences of a potential hazard include a consequence type and a severity level; the predicted consequences include a predicted consequence type and a predicted severity level; an accuracy score is obtained based on the degree of matching between the predicted consequences and the consequences of the potential hazard, including: determining a first degree of matching between the consequence type and the predicted consequence type; determining a second degree of matching between the severity level and the predicted severity level; and weighted summing the first degree of matching and the second degree of matching to obtain the accuracy score.
[0012] In one embodiment, the quality assessment result includes a quality score and a quality assessment explanation text; obtaining the quality assessment result of the hazard identification text based on the integrity score, logical relationship score, and accuracy score includes: weighting and summing the integrity score, logical relationship score, and accuracy score to obtain the quality score; and generating the quality assessment explanation text of the hazard identification text.
[0013] In one implementation, the method further includes displaying a quality score and a quality assessment explanation text.
[0014] In one implementation, the quality assessment interpretation text includes at least one of the following: missing element categories in the hazard identification text, mismatched logical relationships in the hazard identification text, and mismatched hazard consequences in the hazard identification text.
[0015] In one embodiment, the method further includes: obtaining quality assessment results of multiple hazard identification texts and their corresponding actual scores; and obtaining quality assessment index information based on the quality assessment results of each hazard identification text and their corresponding actual scores.
[0016] On one hand, this application provides a text quality assessment apparatus, comprising: an acquisition unit for acquiring hazard identification text to be assessed; an assessment unit for obtaining a completeness score, a logical relationship score, and an accuracy score of the hazard identification text based on the hazard elements, logical connectives, and hazard consequences recorded in the hazard identification text; wherein, the completeness score indicates whether the element categories of the hazard elements in the hazard identification text are complete, the logical relationship score indicates whether the logical relationship of the hazard identification text is correct, and the accuracy score indicates whether the hazard consequences of the hazard identification text are accurate; and an acquisition unit for obtaining a quality assessment result of the hazard identification text based on the completeness score, the logical relationship score, and the accuracy score.
[0017] In one embodiment, the evaluation unit is configured to: evaluate the completeness of the hazard identification text based on the hazard elements in the hazard identification text, and obtain a completeness score; evaluate the logical relationship of the hazard identification text based on the logical connectors and hazard elements in the hazard identification text, and obtain a logical relationship score; and evaluate the accuracy of the hazard identification text based on the hazard consequences in the hazard identification text, and obtain an accuracy score.
[0018] In one implementation, the evaluation unit is used to: parse each grammatical component in the hazard identification text to obtain a syntax analysis tree; extract potential hazards from the hazard identification text according to the feature extraction rules corresponding to each feature category and the syntax analysis tree; the feature extraction rules are set according to the grammatical characteristics of the potential hazards corresponding to the feature categories; and obtain an integrity score according to the feature scores corresponding to the feature categories of the potential hazards.
[0019] In one implementation, the evaluation unit is used to: extract logical connectives contained in the hazard identification text; determine the text position of the logical connectives in the hazard identification text; determine the logical relationship between different hazard elements based on the logical connectives and text positions; obtain the set logical relationship between the element categories corresponding to different hazard elements; and obtain a logical relationship score based on the matching degree between the logical relationship and the set logical relationship.
[0020] In one implementation, the evaluation unit is used to: input hazard identification text into a classification model to obtain predicted consequences; extract potential consequences contained in the hazard identification text; and obtain an accuracy score based on the matching degree between the predicted consequences and the potential consequences.
[0021] In one embodiment, the potential consequences include a consequence type and a severity level; the predicted consequences include a predicted consequence type and a predicted severity level; the assessment unit is used to: determine a first degree of matching between the consequence type and the predicted consequence type; determine a second degree of matching between the severity level and the predicted severity level; and perform a weighted summation of the first degree of matching and the second degree of matching to obtain an accuracy score.
[0022] In one embodiment, the quality assessment result includes a quality score and a quality assessment explanation text; the obtaining unit is used to: perform a weighted summation of the integrity score, logical relationship score, and accuracy score to obtain the quality score; and generate the quality assessment explanation text of the hazard identification text.
[0023] In one embodiment, the obtaining unit is further configured to: display the quality score and the quality assessment explanation text.
[0024] In one implementation, the quality assessment interpretation text includes at least one of the following: missing element categories in the hazard identification text, mismatched logical relationships in the hazard identification text, and mismatched hazard consequences in the hazard identification text.
[0025] In one embodiment, the obtaining unit is further configured to: acquire the quality assessment results of multiple hazard identification texts and their corresponding actual scores; and acquire quality assessment index information based on the quality assessment results of each hazard identification text and their corresponding actual scores.
[0026] On one hand, embodiments of this application provide an electronic device, including: a processor; and a memory storing computer instructions for causing the processor to perform steps of the methods provided in various alternative implementations of any of the above-described text quality assessment methods.
[0027] On the one hand, embodiments of this application provide a computer-readable storage medium storing computer instructions for causing a computer to perform the steps of the methods provided in various alternative implementations of any of the above-described text quality assessment methods.
[0028] On one hand, this application provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code is run in the processor of an electronic device, the processor in the electronic device performs the steps of the method provided in any of the above-described alternative implementations of social interaction.
[0029] The text quality assessment method in this application includes obtaining the hazard identification text to be assessed; obtaining a completeness score, a logical relationship score, and an accuracy score for the hazard identification text based on the hazard elements, logical connectors, and hazard consequences recorded in the hazard identification text; wherein, the completeness score indicates whether the element categories of the hazard elements in the hazard identification text are complete, the logical relationship score indicates whether the logical relationship of the hazard identification text is correct, and the accuracy score indicates whether the hazard consequences of the hazard identification text are accurate; and obtaining the quality assessment result of the hazard identification text based on the completeness score, logical relationship score, and accuracy score. In this way, multiple dimensions can be combined to assess the text quality of the hazard identification text, obtaining a more accurate quality assessment result. Attached Figure Description
[0030] Figure 1 This is a flowchart of a text quality assessment method according to an embodiment of this application.
[0031] Figure 2 This is an example diagram of a logical relationship concept in an embodiment of this application.
[0032] Figure 3 This is an example diagram of a logical relationship in an embodiment of this application.
[0033] Figure 4 This is a schematic diagram of a text quality assessment process in an embodiment of this application.
[0034] Figure 5 This is an example of a quality assessment result display interface in the embodiments of this application. Figure 1 .
[0035] Figure 6 This is an example of a quality assessment result display interface in the embodiments of this application. Figure 2 .
[0036] Figure 7 This is an example diagram of a ROC curve in an embodiment of this application.
[0037] Figure 8 This is a flowchart of a hazard reporting and assessment method in an embodiment of this application.
[0038] Figure 9 This is a structural block diagram of a text quality assessment device according to an embodiment of this application.
[0039] Figure 10 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation
[0040] In industrial environments, safety inspectors typically observe the site, identify and deduce potential hazards, and record hazard information and consequences in hazard identification reports. By analyzing, compiling, and issuing warnings based on these reports, risk management can be implemented for the specific environment. However, due to varying skill levels among safety inspectors, the recorded hazard identification reports may contain missing hazard elements, logical errors, and reasoning mistakes, leading to inaccurate subsequent risk analyses. Therefore, assessing the quality of hazard identification reports is a problem that needs to be addressed.
[0041] Based on the deficiencies of the aforementioned related technologies, this application provides a method, apparatus, device, storage medium, and program product for text quality assessment, aiming to solve the problem of how to assess the text quality of hazard identification text.
[0042] This application provides a method for text quality assessment, which can be applied to electronic devices. This application does not limit the type of electronic device, which can be any suitable type of device, such as terminal devices and servers, etc. This application will not elaborate further.
[0043] See Figure 1 The diagram shown is a flowchart of a text quality assessment method according to an embodiment of this application. Figure 1 As shown, the method may include steps 101 to 103.
[0044] Step 101: Obtain the hazard identification text to be assessed.
[0045] In one implementation, safety inspectors observe the site of the environment to be inspected and infer potential hazards, then fill in and submit the observation information and inference results to obtain a hazard identification text.
[0046] In industrial environments, the management of potential safety hazards is crucial for the sustainable development of enterprises. Hazard identification in enterprises typically includes various forms such as daily inspections, periodic inspections, and special inspections to achieve timely detection, assessment, and rectification of hazards, thereby achieving risk control objectives. Hazard identification requires safety inspectors to possess certain capabilities in hazard identification and consequence assessment. Specifically, after identifying hazard information in the environment through observation and logically deducing potential consequences, safety inspectors can compile information on hazard details and their potential consequences into a hazard identification report (i.e., a hazard identification document) for subsequent risk analysis and rectification.
[0047] Due to the varying skill levels of different safety inspectors, hazard identification reports may contain missing hazard information, logical errors, and reasoning mistakes. Therefore, in subsequent steps, it is necessary to assess the quality of the hazard identification reports completed by the safety inspectors.
[0048] Step 102: Based on the hazard elements, logical connectors, and hazard consequences recorded in the hazard identification text, obtain the completeness score, logical relationship score, and accuracy score of the hazard identification text.
[0049] Among them, the completeness score indicates whether the element categories of the hazard identification text are complete, the logical relationship score indicates whether the logical relationship of the hazard identification text is correct, and the accuracy score indicates whether the consequences of the hazards in the hazard identification text are accurate.
[0050] In one embodiment, step 102 may specifically include the following steps: S1021: Based on the hazard elements in the hazard identification text, perform a completeness assessment of the hazard identification text to obtain a completeness score. S1022: Based on the logical connectors and hazard elements in the hazard identification text, assess the logical relationships of the hazard identification text to obtain a logical relationship score. S1023: Based on the hazard consequences in the hazard identification text, assess the accuracy of the hazard identification text to obtain an accuracy score.
[0051] In one implementation, executing S1021 may specifically include the following steps: S1021-1, S1021-2, and S1021-3.
[0052] In S1021-1: The syntactic components in the hazard identification text are parsed to obtain a syntax analysis tree.
[0053] In one implementation, a vector transformation-based analysis tool is used to parse the grammatical components in the hazard identification text to obtain a syntax analysis tree.
[0054] The grammatical components can include: subject, predicate, object, attributive, adverbial, and complement.
[0055] Specifically, it can identify each grammatical component in the hazard identification text, generate each node of the syntax analysis tree based on the grammatical component identification results, and connect each node to obtain the syntax analysis tree.
[0056] S1021-2: Extract potential hazard elements from the hazard identification text based on the element extraction rules corresponding to each element category and the syntax analysis tree.
[0057] The element categories may include: hazard factors, victims, triggering events and consequences, and may also include emergency measures.
[0058] In one implementation, element extraction rules are set in advance based on the grammatical characteristics of the potential hazards.
[0059] The element extraction rules are set based on the grammatical characteristics of the potential hazards corresponding to the element category. These grammatical characteristics can include keywords contained in the potential hazard elements, as well as the grammatical components or textual positions of the potential hazard elements.
[0060] In practical applications, the element categories and element extraction rules can be set according to the actual application scenario, and there are no restrictions here.
[0061] In one implementation, each potential hazard element can be divided into entity type elements and event type elements based on the grammatical characteristics of each hazard element.
[0062] The grammatical components of event type elements are verb phrases, such as hazard factors, emergency measures, and consequences. Each event type element corresponds to a main clause or subordinate clause and belongs to a subtree of the parse tree, with its root node being the core word of the main clause or subordinate clause. Specifically, by traversing the parse tree, the subject, object, and subordinate clause can be identified, and these subordinate clauses can be split into independent subtrees. These independent subtrees can then be used as candidate hazard elements to further determine the element type.
[0063] In this context, the grammatical components of entity type elements are either subjects or objects, such as "victim." For victim-type entity class elements, the subject can be a subtree of consequences determined based on the meaning of the consequence.
[0064] For example, hazard identification texts, as datasets describing hazards generated in industrial production, can be short texts with a length of around a hundred words. As an example, a hazard identification text might be: "The distribution box has no protective door, the wires inside the distribution box have aged due to wind and rain, and the electrician did not wear insulated gloves and shoes while performing the work, resulting in electric shock."
[0065] The following examples, in conjunction with Table 1, illustrate the potential hazards in the aforementioned hazard identification text.
[0066] Table 1: Example Table of Hazard Element Description
[0067]
[0068] Table 1 lists the element categories as hazard factors, triggering events, emergency measures, and consequences. Taking hazard factors as an example, the grammatical characteristics of hazard factors (i.e., element categories) are: they are abnormal attributes of the causal event or subject, and typically include at least one of the following keywords: "none," "absent," "too large," or "too few." Specifically, in the above hazard identification text, the hazard factors corresponding to the hidden danger elements are: the distribution box has no protective door, the wires are aging after being exposed to wind and rain, and no insulated gloves or shoes are worn.
[0069] In one implementation, the following steps are performed for each element category:
[0070] Method 1: Extract the hazard elements that match the keywords corresponding to the element categories from the parsing tree. If a node matching the keyword exists in the parsing tree, that node can be identified as a hazard element. For example, in Table 1, the keywords corresponding to the hazard factors include at least one of the following keywords: "none," "absent," "too large," or "too few."
[0071] Optionally, keywords can be extracted using automatic and / or manual extraction methods.
[0072] Method 2: Extract potential elements that conform to grammatical features from the parsing tree based on the grammatical components corresponding to the element categories. Keywords and grammatical features are determined based on grammatical characteristics. Grammatical features can include grammatical components and text positions, etc.
[0073] Method 3: Extract potential elements from the parsing tree based on the keywords and grammatical features corresponding to the element categories.
[0074] This allows for the extraction of hazard elements even with limited labeled data. Furthermore, the research domain and domain data sources can be determined first, followed by identifying the types of entities (i.e., hazard elements) (i.e., element categories), establishing hazard element specifications, and ultimately determining the element extraction rules. Hazard element extraction can also employ any of the following methods: statistical machine learning-based methods, deep learning-based methods, and deep learning methods based on multi-party fusion.
[0075] S1021-3: Obtain an integrity score based on the element score corresponding to the element category of the potential hazard element.
[0076] In one implementation, the element score corresponding to the element category of each extracted hidden danger element is obtained, and the element scores corresponding to each element category are weighted and summed to obtain an integrity score.
[0077] There can be one or more potential hazard elements. The weight of each element category can be set according to the actual application scenario, and there are no restrictions here.
[0078] In one implementation, the following formula can be used to determine the integrity score:
[0079]
[0080] Among them, S integrity For integrity scoring, Dev represents the hazard factor, Cons represents the consequences, Victim represents the victim, and otherpoints represents emergency measures. The bool() function represents 1 if it exists, and 0 otherwise.
[0081] In one implementation, S1022 may include S1022-1 to S1022-5.
[0082] S1022-1: Extract logical connectors from the hazard identification text.
[0083] In one implementation, a set of related words is pre-set, and logical related words that match the related words in the set are extracted from the hazard identification text.
[0084] As an example, if a word in the hazard identification text is the same as a related word in the related word set, then the two are determined to be a match, and the word is determined to be a logical related word.
[0085] The conjunctions in the conjunction set are words used to determine logical relationships, which may include at least one of the following: causality, sequence, contrast, and condition. The conjunctions in the conjunction set can be derived from common logical relationships and expressions specific to hazard identification texts. The conjunction set can be set according to the actual application scenario and is not limited here.
[0086] S1022-2: Determine the textual position of logical connectives in the hazard identification text.
[0087] In one implementation, the textual position of logical connectives in each potential hazard element is determined.
[0088] S1022-3: Determine the logical relationships between different potential hazard elements based on logical connectors and text positions.
[0089] In one implementation, the causal scope is determined by analyzing the text position syntax tree of logical connectives, thereby dividing each hazard element into at least one pair of hazard elements and determining the logical relationship between each pair. Furthermore, a logical relationship diagram of the hazard identification text can be generated based on the logical relationship between each pair of hazard elements. As an example, the logical relationship can be a causal relationship.
[0090] The following is combined Figure 2 Explain the concept map of logical relationships. Figure 2 This is an example diagram illustrating a logical relationship concept. Figure 2 In the logical relationship diagram, there are three types of nodes: environmental conditions, triggering conditions, and potential consequences. Environmental conditions are the hazards, and triggering conditions are the events that trigger them. Nodes representing environmental conditions are labeled G, such as location, material, and deviation. Nodes representing triggering conditions are labeled W, such as process. Nodes representing potential consequences are labeled T, such as possible events and consequences.
[0091] The following is combined Figure 3 Provide an example of a logic diagram. Figure 3 This is an example diagram of a logical relationship. Figure 3 As can be seen from the various types of nodes, the lack of a protective door in the distribution box led to personnel being triggered and killed.
[0092] In this way, logical relationships can be determined based on the semantic features of logical relationships and the rules for judging logical relationships. High-quality labeled data is not required to determine logical relationships. Furthermore, by pointing arrows from cause to effect, the evolution process of potential incidents can be modeled as a logical relationship diagram.
[0093] It should be noted that logical relationships include explicit and implicit logical relationships. Considering the simple format and concise logic of hazard identification text descriptions, determining logical relationships can also be achieved through at least one of the following: machine learning-based methods and deep learning-based methods. This can reduce the workload of manual rule formulation.
[0094] S1022-4: Obtain the setting logic relationship between the element categories corresponding to different hidden danger elements.
[0095] This involves pre-configuring logical relationships between different element categories. For example, hazard factors and triggering events are the "causes" in a causal relationship, while consequences are the "results."
[0096] S1022-5: Obtain a logical relationship score based on the logical relationship and the matching degree between the set logical relationships.
[0097] As an example, if the causal relationship and the established logical relationship corresponding to a pair of safety hazard factors are the same, then the two are determined to match; otherwise, the two are determined to not match. For example, the logical relationship may have the problem of reversed causality of safety hazard factors, such as "hazard factor" as the "result" in "causal relationship" and "consequence factor" as the "cause" in "causal relationship".
[0098] In one implementation, if a mismatched logical relationship exists, the logical relationship score is 0; otherwise, it is 1.
[0099] In one implementation, S1023 may include S1023-1 to S1023-3.
[0100] S1023-1: Input the hazard identification text into the classification model to obtain the predicted consequences.
[0101] The predicted consequences include the predicted consequence type and the predicted severity level. The predicted consequence type indicates the type of consequence the model predicts for the hazard, while the predicted severity level indicates the predicted consequence and severity level of the hazard.
[0102] Furthermore, before executing S1023-1, a pre-training method can be used to obtain a pre-trained model, and the sample data can be labeled to obtain labeled data. Then, a supervised fine-tuning method can be used to fine-tune the pre-trained model using the labeled data to obtain a trained classification model. This classification model is used to classify hazard identification texts and obtain the predicted consequence type and predicted severity level of the hazard identification text.
[0103] The pre-trained model is based on fine-tuning of the Transformer model. It models the dependencies between words in the input text sequence through a self-attention mechanism, capturing long-distance relationships between words, which helps in the semantic understanding of the text content during text scoring. The pre-trained model allows downstream tasks in natural language processing to be completed by fine-tuning the parameters of the pre-trained model.
[0104] In this embodiment, the classification model uses text classification technology. Text classification technology mainly classifies text at the semantic level, such as text content, topic, and sentiment. It is commonly used in scenarios such as news classification, spam classification, and public opinion analysis. Traditional machine learning methods and deep learning methods can also be used for classification.
[0105] As an example, the experience of experts (e.g., domain security experts) can be used to label the sample data with the corresponding consequence types and severity levels to determine whether the consequence types and severity levels entered by the user are consistent with the predicted consequence types and predicted severity levels output by the model.
[0106] Furthermore, the classification model can be tested using test datasets (such as a large number of hazard identification texts generated in industrial production environments) to obtain a classification model that meets the testing requirements.
[0107] Furthermore, during the pre-training phase, the corresponding weights in the loss function can be set according to the proportion of data for each consequence type and / or severity level to obtain an unbalanced loss function.
[0108] For example, the 4,760 data points annotated by experts cover four consequence types, with the most numerous consequence type accounting for over 80% of the data. Weights (i.e., attention) are added to the loss function for the smaller category (i.e., annotated data of a consequence type with a smaller data proportion).
[0109] In this embodiment of the application, to address issues such as the cost of expert annotation data, imbalance in the types of annotated data, and limited coverage of scenarios, the number of subcategories is increased when sampling data to be annotated to improve the balance of the annotated data. Furthermore, training can be conducted on imbalanced data during the training process, including strategies such as oversampling, undersampling, and using imbalanced loss functions to optimize the training results. This reduces the proportion of imbalanced consequence types and / or severity levels during the data preprocessing stage, and uses a weighted loss function to increase attention to the smaller proportion of consequence types and / or severity levels.
[0110] The following example, using Table 2, illustrates the prediction accuracy for different consequence types.
[0111] Table 2: Examples of Consequence Type Prediction
[0112] Prediction accuracy 0.9845 0.7308 0.2500 0.6667
[0113] The following example, using Table 3, illustrates the accuracy of severity level prediction.
[0114] Table 3: Examples of Severity Level Prediction
[0115]
[0116] As can be seen from Tables 2 and 3, the classification model has high accuracy in predicting "personnel injury" (i.e., consequence type) and "S3" (i.e., severity level).
[0117] S1023-2: Extract the potential consequences contained in the hazard identification text.
[0118] Among them, the consequences of potential hazards include the type of consequences and / or the severity level.
[0119] S1023-3: Obtain an accuracy score based on the degree of matching between predicted consequences and potential consequences.
[0120] In one implementation, when performing S1023-3, the following steps may be adopted:
[0121] S1023-31: Determine the first degree of matching between the type of consequence and the predicted type of consequence.
[0122] S1023-32: Determine the second degree of matching between the severity level and the predicted severity level.
[0123] S1023-33: The first matching degree and the second matching degree are weighted and summed to obtain the accuracy score.
[0124] As an example, if the consequence type is the same as the predicted consequence type and the severity level is the same as the predicted severity level, then the accuracy score is 1.
[0125] In this embodiment of the application, the predicted consequences are taken as correct information. If the consequences of the hidden danger are consistent with the predicted consequences, the consequences of the hidden danger are determined to be accurate; otherwise, the consequences of the hidden danger are determined to be inaccurate.
[0126] Furthermore, the accuracy score can be determined by combining the probability of hazard. The probability of hazard refers to whether the overall frequency of occurrence from the initial hazard to the final consequence conforms to experience.
[0127] Step 103: Obtain the quality assessment results of the hazard identification text based on the completeness score, logical relationship score, and accuracy score.
[0128] The quality assessment results include quality scores and explanatory texts of the quality assessment.
[0129] In one implementation, the integrity score, logical relationship score, and accuracy score are weighted and summed to obtain a quality score; a quality assessment explanation text for the hazard identification text is then generated.
[0130] As an example, the following formula can be used to determine the quality score:
[0131] Score = S integrity +S relation +S accuracy ;
[0132] Where Score is the quality rating, S integrity For integrity scoring, S relation Score the logical relationship. accuracy A score is given for accuracy.
[0133] In this way, the quality of hazard identification text can be evaluated from three aspects: completeness, correctness (i.e., logical correctness), and accuracy.
[0134] Furthermore, quality assessments can be conducted by combining readability and / or authenticity.
[0135] Readability includes at least one of the following: correct spelling, complete syntactic structure, and concise sentences. Correct spelling refers to checking for typos in the hazard identification text. Complete syntactic structure refers to checking for the use of consecutive nouns or verbs and the integrity of sentence structure. Concise sentences refer to checking whether a single word in a sentence has more than three modifiers and whether the predicate of a sentence has more than two subjects. Authenticity refers to contextual relevance, checking whether the hazard identification text is relevant to the context of the submitted image content, thereby determining whether it corresponds to the actual production scenario.
[0136] Readability and completeness are general text quality assessment dimensions, while accuracy, correctness, and authenticity are requirements specific to industrial scenarios. Accuracy and correctness ensure the self-consistency of hazard identification text content, while authenticity, based on the needs of hazard investigation and rectification, ensures that the hazard identification text expresses hazards in real-world scenarios. Entity recognition technology is used to assess completeness, logical relationship recognition technology is used to assess logical correctness, and text classification technology is used to assess accuracy.
[0137] Furthermore, the quality assessment explanation text shall at least include the reasons for the quality assessment deduction; the quality assessment explanation text shall include at least one of the following: the missing element categories in the hazard identification text, the mismatched logical relationships in the hazard identification text, and the mismatched hazard consequences in the hazard identification text. Furthermore, the quality assessment explanation text may also include at least one of the following: hazard elements, logical relationships, predicted consequences, and grammatical components.
[0138] Furthermore, the quality assessment results and explanatory text of the quality assessment can also be displayed.
[0139] In one implementation, determining the reasons for deducting points in a quality assessment may include at least one of the following:
[0140] Method 1: If the completeness score does not reach the first score threshold, then determine the category of missing elements in the hazard identification text.
[0141] Method 2: If the logical relationship score does not reach the second scoring threshold, then the mismatched logical relationship in the hazard identification text is determined.
[0142] Method 3: If the accuracy score does not reach the third scoring threshold, then identify the potential consequences of mismatch in the hazard identification text.
[0143] In practical applications, the first, second, and third scoring thresholds can all be set according to the actual application scenario, and no restrictions are imposed here.
[0144] The following is combined with Figure 4 The framework of the text quality assessment process is illustrated. Figure 4This is a schematic diagram of a text quality assessment process. Specifically, it involves identifying potential hazards in the hazard identification text and determining the completeness of the text based on the hazard hazard element categories, thus obtaining a completeness score. Furthermore, it analyzes the logical relationships within the hazard identification text to obtain a well-structured and logically evolving logic diagram, and then assesses the logical correctness of the hazard identification text based on this diagram, obtaining a logic relationship score. Finally, it determines the accuracy of the hazard identification text based on the predicted consequences of the classification model and the potential consequences within the hazard identification text, obtaining an accuracy score. Ultimately, the hazard identification text is assessed for quality across three dimensions: completeness, logical correctness, and accuracy, yielding the final quality assessment result.
[0145] The following is combined with Figure 5 and Figure 6 An example is provided to illustrate the interface for displaying quality assessment results. Figure 5 Example of a quality assessment result display interface Figure 1 . Figure 6 Example of a quality assessment result display interface Figure 2 . Figure 5 The input hazard identification text includes: hazard description, potential consequences, consequence category (i.e. accident category), and severity (i.e. severity level). The output quality assessment result is 5, and the assessment recommendation, i.e., the predicted consequences, is also output. Figure 6 The document displays various potential risks, logical relationships, and grammatical components.
[0146] Furthermore, the capability of the quality assessment results can be evaluated. In one implementation, the following steps can be taken:
[0147] Obtain the quality assessment results and actual scores of multiple hazard identification texts; based on the quality assessment results and actual scores of each hazard identification text, obtain quality assessment indicator information.
[0148] The quality assessment metrics include a quadratic weighted kappa (QWK) and an evaluation score. The evaluation score represents the accuracy of the quality score. Quality assessment metrics may also include precision and recall.
[0149] In one implementation, the consistency between the quality score and the expert score is evaluated to obtain a consistency score.
[0150] In a test scenario for quality assessment indicators, the quality score is divided into five quality levels. Multiple hazard identification texts are evaluated by experts to obtain actual scores, which in turn determine the corresponding quality levels, ultimately resulting in 1800 expert score data points. Precision and recall are determined based on the quality levels corresponding to the actual scores and quality scores, and the resulting evaluation score is 0.3934. An example is provided below with reference to Table 4.
[0151] Table 4: Examples of Quality Assessment Indicator Information
[0152] 1 0.0769 0.1053 0.0889 19 2 0.3165 0.1852 0.2336 135 3 0.1466 0.1830 0.1628 153 4 0.4901 0.6289 0.5509 908 5 0.3521 0.2198 0.2707 655 Weighted average 0.3969 0.4118 0.3934 1870
[0153] The meaning of the QWK indicator will be explained below with reference to Table 5.
[0154] Table 5: Examples of the meaning of the QWK indicator
[0155] Consistency strength Poor generally medium Strong powerful
[0156] Furthermore, the quality assessment results can be divided into two quality levels: acceptable and unacceptable. The calculated QWK value is 0.235, showing general agreement with the expert assessment. The receiver operating characteristic (ROC) curve is shown below. Figure 7 As shown. Figure 7 This is an example of a ROC curve. Figure 7 In this study, the area under the ROC curve (AUC) was 0.64. When a predicted severity level of 4 or above was considered "qualified," the false positive rate was 0.61% and the true positive rate was 0.86%.
[0157] The following is combined with Figure 8 The method for submitting hazard identification text is explained. Figure 8 The flowchart illustrates a method for hazard reporting and assessment. The specific steps of this method are as follows:
[0158] Step 801: Display the employee login system interface according to the employee login instruction.
[0159] The employees are safety inspectors.
[0160] Step 802: Display the hazard report browsing interface according to the employee's browsing instructions.
[0161] Among them, the hazard report is the hazard identification text.
[0162] Step 803: Determine whether to report the potential hazard. If yes, proceed to step 804; otherwise, proceed to step 810.
[0163] Step 804: Fill out the hazard report according to the employee's input instructions.
[0164] Step 805: Determine whether to assess the quality of the report. If yes, proceed to step 806; otherwise, proceed to step 808.
[0165] Step 806: Display the text quality assessment results.
[0166] Step 807: Determine whether to modify the text quality assessment result. If yes, proceed to step 804; otherwise, proceed to step 808.
[0167] Step 808: Determine whether to save the text quality assessment results. If yes, proceed to step 809; otherwise, proceed to step 810.
[0168] Step 809: Save the text quality assessment results.
[0169] For example, an employee clicks the "Add" button to save the text quality assessment results.
[0170] Step 810: Process ends.
[0171] In this way, users can check whether the text has been filled in correctly based on the text quality assessment results, and can also modify the hazard identification text based on the text quality assessment results, thereby improving the professionalism of employees' work.
[0172] In this embodiment, potential hazards in hazard identification text can be identified, and the completeness of the hazard identification text can be determined based on the hazard hazard element category to obtain a completeness score. Furthermore, the logical relationships in the hazard identification text can be analyzed to obtain a clear and logically evolving logical relationship diagram, and the correctness of the logic in the hazard identification text can be determined based on the logical relationship diagram to obtain a logical relationship score. Additionally, the accuracy of the hazard identification text can be determined based on the predicted consequences of the classification model and the potential consequences in the hazard identification text to obtain an accuracy score. Furthermore, the quality of hazard identification text can be assessed from three dimensions: completeness, logical correctness, and accuracy, solving the problem of difficulty in assessing the quality of hazard identification text and effectively alleviating the pressure of manual review. Moreover, the assessment criteria for completeness, logical correctness, and accuracy can be determined, and the reasons for deductions in the quality assessment results can be identified, providing interpretable analytical text to help users with subsequent improvements. Finally, the effectiveness of the above quality assessment can be verified through expert quality assessment results to validate its effectiveness, and optimization can be carried out based on the verification results.
[0173] Based on the same inventive concept, this application also provides a text quality assessment apparatus. Since the principle of the above apparatus and device in solving the problem is similar to that of a text quality assessment method, the implementation of the above apparatus can refer to the implementation of the method, and repeated details will not be elaborated further. This apparatus can be applied to electronic devices. This application does not limit the type of electronic device; it can be any suitable type of device, such as terminal devices and servers, etc., which will not be elaborated further in this application.
[0174] See Figure 9 The diagram shown is a structural block diagram of a text quality assessment apparatus according to an embodiment of this application. In some embodiments, the text quality assessment apparatus exemplified in this application includes:
[0175] Acquisition unit 901 is used to acquire the hazard identification text to be evaluated;
[0176] The assessment unit 902 is used to obtain a completeness score, a logical relationship score, and an accuracy score for the hazard identification text based on the hazard elements, logical connectors, and hazard consequences recorded in the hazard identification text. The completeness score indicates whether the hazard elements in the hazard identification text are complete, the logical relationship score indicates whether the logical relationship in the hazard identification text is correct, and the accuracy score indicates whether the hazard consequences in the hazard identification text are accurate.
[0177] Unit 903 is used to obtain the quality assessment results of the hazard identification text based on the integrity score, logical relationship score, and accuracy score.
[0178] In one embodiment, the evaluation unit 902 is used to: evaluate the integrity of the hazard identification text based on the hazard elements in the hazard identification text, and obtain an integrity score; evaluate the logical relationship of the hazard identification text based on the logical connectors and hazard elements in the hazard identification text, and obtain a logical relationship score; and evaluate the accuracy of the hazard identification text based on the hazard consequences in the hazard identification text, and obtain an accuracy score.
[0179] In one embodiment, the evaluation unit 902 is used to:
[0180] The syntactic components in the hazard identification text are analyzed to obtain a syntax tree;
[0181] Based on the element extraction rules corresponding to each element category and the syntax analysis tree, the hazard elements in the hazard identification text are extracted; the element extraction rules are set according to the grammatical characteristics of the hazard elements corresponding to the element categories.
[0182] An integrity score is obtained based on the element score corresponding to the element category of the potential hazard element.
[0183] In one embodiment, the evaluation unit 902 is used to:
[0184] Extract logical connectors from the hazard identification text;
[0185] Identify logical connectors and their textual locations within the hazard identification text;
[0186] Based on logical connectors and text location, determine the logical relationships between different potential hazard elements;
[0187] Obtain the logical relationships between the element categories corresponding to different potential hazards;
[0188] A logical relationship score is obtained based on the logical relationships and the degree of matching between them.
[0189] In one embodiment, the evaluation unit 902 is used to:
[0190] Input the hazard identification text into the classification model to obtain the predicted consequences;
[0191] Extract the potential consequences contained in the hazard identification text;
[0192] An accuracy score is obtained based on the degree of matching between the predicted consequences and the potential consequences.
[0193] In one implementation, the consequences of a potential hazard include the type of consequence and the severity level; the predicted consequences include the type of predicted consequences and the predicted severity level.
[0194] Evaluation unit 902 is used for:
[0195] Determine the first degree of match between the type of consequence and the predicted type of consequence;
[0196] Determine the second degree of match between the severity level and the predicted severity level;
[0197] The accuracy score is obtained by weighted summation of the first and second matching scores.
[0198] In one implementation, the quality assessment results include a quality score and a quality assessment explanation text;
[0199] Unit 903 is used for:
[0200] The quality score is obtained by weighting and summing the completeness score, logical relationship score, and accuracy score.
[0201] Generate a quality assessment explanation text for the hazard identification text.
[0202] In one embodiment, the obtaining unit 903 is further configured to:
[0203] Display the quality score and the explanatory text of the quality assessment.
[0204] In one implementation, the quality assessment interpretation text includes at least one of the following:
[0205] The missing element categories in the hazard identification text, the mismatched logical relationships in the hazard identification text, and the potential consequences of mismatches in the hazard identification text.
[0206] In one embodiment, the obtaining unit 903 is further configured to:
[0207] Obtain the quality assessment results of multiple hazard identification texts and their corresponding actual scores;
[0208] Based on the quality assessment results of each hazard identification text and its corresponding actual score, quality assessment indicator information is obtained.
[0209] The text quality assessment method in this application includes obtaining the hazard identification text to be assessed; obtaining a completeness score, a logical relationship score, and an accuracy score for the hazard identification text based on the hazard elements, logical connectors, and hazard consequences recorded in the hazard identification text; wherein, the completeness score indicates whether the element categories of the hazard elements in the hazard identification text are complete, the logical relationship score indicates whether the logical relationship of the hazard identification text is correct, and the accuracy score indicates whether the hazard consequences of the hazard identification text are accurate; and obtaining the quality assessment result of the hazard identification text based on the completeness score, logical relationship score, and accuracy score. In this way, multiple dimensions can be combined to assess the text quality of the hazard identification text, obtaining a more accurate quality assessment result.
[0210] In this embodiment of the application, an electronic device is provided, including:
[0211] Processor; and
[0212] The memory stores computer instructions that cause the processor to execute the methods of any of the above-described embodiments.
[0213] In this application embodiment, a computer-readable storage medium is provided, storing computer instructions for causing a computer to perform the methods of any of the above embodiments.
[0214] This application also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in the processor of an electronic device, the processor in the electronic device performs the method of any of the above-described embodiments.
[0215] Figure 10 A schematic diagram of an electronic device is shown. (See also...) Figure 10As shown, the electronic device 10000 includes a processor 10010 and a memory 10020, and optionally may also include a power supply 10030, a display unit 10040, and an input unit 10050.
[0216] The processor 10010 is the control center of the electronic device 10000. It connects various components through various interfaces and lines, and performs various functions of the electronic device 10000 by running or executing software programs and / or data stored in the memory 10020, thereby performing overall monitoring of the electronic device 10000.
[0217] In this embodiment, when the processor 10010 calls the computer program stored in the memory 10020, it executes the steps in the above embodiments.
[0218] Optionally, the processor 10010 may include one or more processing units; preferably, the processor 10010 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 10010. In some embodiments, the processor and memory may be implemented on a single chip; in some embodiments, they may also be implemented separately on independent chips.
[0219] The memory 10020 may primarily include a program storage area and a data storage area. The program storage area may store the operating system, various applications, etc.; the data storage area may store data created based on the use of the electronic device 10000, etc. In addition, the memory 10020 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device, etc.
[0220] Electronic device 10000 also includes a power supply 10030 (such as a battery) that supplies power to various components. The power supply can be logically connected to processor 10010 through a power management system, thereby enabling the power management system to manage functions such as charging, discharging, and power consumption.
[0221] The display unit 10040 can be used to display information input by the user or information provided to the user, as well as various menus of the electronic device 10000. In this embodiment, it is mainly used to display the display interfaces of various applications in the electronic device 10000, and the text, pictures, and other objects displayed on the display interfaces. The display unit 10040 may include a display panel 10041. The display panel 10041 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.
[0222] The input unit 10050 can be used to receive information such as numbers or characters input by the user. The input unit 10050 may include a touch panel 10051 and other input devices 10052. The touch panel 10051, also known as a touch screen, can collect touch operations on or near the touch panel 10051 (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel 10051).
[0223] Specifically, the touch panel 10051 can detect user touch operations and the signals generated by these operations, convert them into touch point coordinates, send them to the processor 10010, and receive and execute commands from the processor 10010. Furthermore, the touch panel 10051 can be implemented using various types of sensors, including resistive, capacitive, infrared, and surface acoustic wave sensors. Other input devices 10052 can include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.
[0224] Of course, the touch panel 10051 can cover the display panel 10041. When the touch panel 10051 detects a touch operation on or near it, it transmits the information to the processor 10010 to determine the type of touch event. Subsequently, the processor 10010 provides corresponding visual output on the display panel 10041 according to the type of touch event. Although in Figure 10 In this embodiment, the touch panel 10051 and the display panel 10041 are two separate components to realize the input and output functions of the electronic device 10000. However, in some embodiments, the touch panel 10051 and the display panel 10041 can be integrated to realize the input and output functions of the electronic device 10000.
[0225] The electronic device 10000 may also include one or more sensors, such as a pressure sensor, a gravity acceleration sensor, a proximity sensor, etc. Of course, depending on the specific application, the electronic device 10000 may also include other components such as a camera. Since these components are not the focus of this application embodiment, therefore... Figure 10 It is not shown in the text and will not be described in detail here.
[0226] Those skilled in the art will understand that Figure 10 This is merely an example of an electronic device and does not constitute a limitation on the electronic device. It may include more or fewer components than shown, or a combination of certain components, or different components.
[0227] For ease of description, the above sections are divided into modules (or units) according to their functions and described separately. Of course, in implementing this application, the functions of each module (or unit) can be implemented in one or more software or hardware components.
Claims
1. A method for text quality assessment, characterized in that, The method includes: Obtain the hazard identification text to be evaluated; the hazard identification text includes observation information and reasoning results obtained from observing the environment to be tested and reasoning about potential hazards; Based on the hazard elements, logical connectors, and hazard consequences recorded in the hazard identification text, a completeness score, a logical relationship score, and an accuracy score are obtained for the hazard identification text. The completeness score indicates whether the hazard elements in the hazard identification text have complete element categories; the logical relationship score indicates whether the logical relationships in the hazard identification text are correct; and the accuracy score indicates whether the hazard consequences in the hazard identification text are accurate. The logical relationship refers to the logical relationship between each pair of hazard elements, and the logical relationship score is determined based on the matching degree between the logical relationship and a predefined logical relationship. Based on the completeness score, the logical relationship score, and the accuracy score, the quality assessment result of the hazard identification text is obtained; the quality assessment result includes a quality score and a quality assessment explanation text. The accuracy score is obtained by the following steps: inputting the hazard identification text into a classification model to obtain the predicted consequences; extracting the potential consequences contained in the hazard identification text; and obtaining the accuracy score based on the matching degree between the predicted consequences and the potential consequences. The potential consequences include the type of consequences and the severity level; the predicted consequences include the type of predicted consequences and the predicted severity level. The accuracy score is obtained based on the degree of matching between the predicted consequences and the potential consequences, including: Determine a first degree of matching between the said consequence type and the predicted consequence type; Determine a second degree of matching between the severity level and the predicted severity level; The accuracy score is obtained by weighted summation of the first matching degree and the second matching degree.
2. The method according to claim 1, characterized in that, The process of obtaining a completeness score, a logical relationship score, and an accuracy score for the hazard identification text based on the hazard elements, logical connectors, and hazard consequences recorded in the hazard identification text includes: Based on the hazard elements in the hazard identification text, an integrity assessment is performed on the hazard identification text to obtain the integrity score; Based on the logical connectors and potential hazard elements in the hazard identification text, the logical relationship of the hazard identification text is evaluated to obtain a logical relationship score; The accuracy of the hazard identification text is evaluated based on the potential consequences of the hazards, and an accuracy score is obtained.
3. The method according to claim 2, characterized in that, The step of conducting an integrity assessment of the hazard identification text based on the hazard elements in the hazard identification text to obtain the integrity score includes: The syntactic components in the hazard identification text are parsed to obtain a syntax analysis tree; Based on the element extraction rules corresponding to each element category and the syntax analysis tree, the hidden danger elements in the hazard identification text are extracted; the element extraction rules are set according to the grammatical characteristics of the hidden danger elements corresponding to the element categories. The integrity score is obtained based on the element score corresponding to the element category of the potential hazard element.
4. The method according to claim 2, characterized in that, The step of evaluating the logical relationships of the hazard identification text based on logical connectors and potential hazard elements to obtain a logical relationship score includes: Extract logical connectors from the hazard identification text; Determine the text position of the logically related words in the hazard identification text; Based on the logical connectives and the text positions, determine the logical relationships between different potential hazard elements; Obtain the logical relationships between the element categories corresponding to different potential hazards; The logical relationship score is obtained based on the matching degree between the logical relationship and the set logical relationship.
5. The method according to any one of claims 1-4, characterized in that, The quality assessment results include a quality score and a quality assessment explanation text; The process of obtaining the quality assessment result of the hazard identification text based on the completeness score, the logical relationship score, and the accuracy score includes: The quality score is obtained by weighted summation of the completeness score, the logical relationship score, and the accuracy score. Generate a quality assessment explanation text for the hazard identification text.
6. The method according to claim 5, characterized in that, The method further includes: Display the quality score and the explanatory text of the quality assessment.
7. The method according to claim 5, characterized in that, The quality assessment interpretation text includes at least one of the following: The missing element categories in the hazard identification text, the mismatched logical relationships in the hazard identification text, and the potential consequences of mismatches in the hazard identification text.
8. The method according to any one of claims 1-4, characterized in that, The method further includes: Obtain the quality assessment results of multiple hazard identification texts and their corresponding actual scores; Based on the quality assessment results of each hazard identification text and its corresponding actual score, quality assessment indicator information is obtained.
9. A device for text quality assessment, characterized in that, The device includes: The acquisition unit is used to acquire the hazard identification text to be evaluated; the hazard identification text includes observation information and reasoning results obtained from observing the environment to be tested and reasoning about potential hazards; The evaluation unit is used to obtain a completeness score, a logical relationship score, and an accuracy score for the hazard identification text based on the hazard elements, logical connectors, and hazard consequences recorded in the hazard identification text. The completeness score indicates whether the element categories of the hazard elements in the hazard identification text are complete; the logical relationship score indicates whether the logical relationships in the hazard identification text are correct; and the accuracy score indicates whether the hazard consequences in the hazard identification text are accurate. The logical relationship refers to the logical relationship between each pair of hazard elements, and the logical relationship score is determined based on the matching degree between the logical relationship and a predefined logical relationship. The obtaining unit is configured to obtain a quality assessment result of the hazard identification text based on the completeness score, the logical relationship score, and the accuracy score; the quality assessment result includes a quality score and a quality assessment explanation text. The accuracy score is obtained by the following steps: inputting the hazard identification text into a classification model to obtain the predicted consequences; extracting the potential consequences contained in the hazard identification text; and obtaining the accuracy score based on the matching degree between the predicted consequences and the potential consequences. Potential consequences include the type of consequences and the severity level; predicted consequences include the type of predicted consequences and the predicted severity level. The assessment unit is used to: determine the first degree of match between the consequence type and the predicted consequence type; determine the second degree of match between the severity level and the predicted severity level; and perform a weighted summation of the first and second degree of match to obtain an accuracy score.
10. An electronic device, characterized in that, include: processor; as well as A memory storing computer instructions for causing the processor to perform the method according to any one of claims 1 to 8.
11. A computer-readable storage medium, characterized in that, The computer contains computer instructions for causing the computer to perform the method according to any one of claims 1 to 8.
12. A computer program product, characterized in that, Includes computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is executed in a processor of an electronic device, the processor in the electronic device is the method according to any one of claims 1 to 8.