Method and system for technical text translation quality evaluation

Abstract

The application relates to the technical field of natural language processing, and provides a technical text translation quality evaluation method, a technical text corpus oriented to source language technical texts, corresponding target language translation texts and associated annotation information, and an associated annotation information combined with a quantitative evaluation system. The translation quality evaluation dimensions and weights of the technical text are oriented to technical accuracy, term consistency, language quality and regional habit requirements; independent scores of the evaluation dimensions are respectively calculated, weighted calculation is carried out by dynamically adjusting the weight values to automatically generate a comprehensive quality score of the translation text, and based on a preset score threshold interval, the translation quality grade of the translation text is determined according to the comprehensive quality score. The application also provides a system for executing the above method, and the automatic determination of the translation quality grade is realized through quantitative index calculation and a preset score threshold interval, so that the time cost of manual correction and evaluation is greatly reduced, and the efficiency of large-scale technical document translation processing is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and particularly to a method and system for evaluating the translation quality of technical texts. Background Art

[0002] In technology-intensive fields such as high-end equipment manufacturing, aerospace, and new energy engineering, technical texts, as the core carriers of cross-language knowledge transfer, technical collaboration, and achievement transformation, their translation quality directly affects the implementation efficiency of technical solutions, the success rate of patent authorization, and the accuracy of international cooperation. Taking frontier research directions such as lithium-ion battery health management, satellite optical network communication, and smart city sewage treatment as examples, technical texts usually contain precise definitions of professional terms, complex logical derivation processes, high-precision experimental data presentations, and strict technical specification constraints. The translation requirements of such texts have long exceeded the basic level of semantic equivalence. With the increasing surge in cross-language technical collaboration needs, the demand for high-quality translations of technical texts is becoming more urgent.

[0003] CN121279326A discloses a method, device, computing device, storage medium, and program product for evaluating text translation quality, which obtains a translation pair composed of a source text and a translated text to be evaluated; performs a first-level evaluation, including: evaluating the translation pair based on rule detection to obtain an evaluation result; performs a second-level evaluation, including: evaluating the translation pair based on the chain-of-thought reasoning mechanism of a large language model to obtain an evaluation result; performs a fusion process, including: fusing the obtained evaluation results to obtain a target evaluation result representing the evaluation quality of the translation pair.

[0004] CN121543608A discloses an intelligent translation method applied to an intelligent translation system. The intelligent translation system includes an acquisition module, a term dynamic management module, an intelligent translation module, and a translation quality evaluation module. Among them, the method includes: the acquisition module acquires a question-and-answer dataset to be translated and its metadata information, and the metadata information at least includes a source language identifier, a target language identifier, and a target professional field; the term dynamic management module uses a feature matching rule library and a hierarchical term library to translate each vocabulary in the text to be translated corresponding to the question-and-answer dataset to be translated into the target language to obtain a pre-translation result; the intelligent translation module uses a semantic calibration system enhanced by RAG to perform semantic enhancement processing on the pre-translation result, combines a pre-trained local translation model and the semantic enhancement processing result to translate the text to be translated, and obtains a translated text output by the pre-trained local translation model; the translation quality evaluation module evaluates the translation quality of the translated text from multiple dimensions, generates a translation quality evaluation report, and completes the translation of the text to be translated and the update of the hierarchical term library based on the translation quality evaluation report.

[0005] Current technical text translation relies heavily on general translation tools or human literal translation, lacking a standardized evaluation system for the core attributes of technical texts. It fails to focus on the specific needs of technical texts, such as the uniformity of translation of professional terminology and the accuracy of conveying complex logical semantics. Instead, it only focuses on general evaluation dimensions such as grammatical correctness and vocabulary matching, making it impossible to accurately determine whether the translation results meet the needs of actual scenarios such as engineering applications. This can easily lead to technical problems such as technical cognitive biases and misleading implementation of technical solutions due to inconsistent terminology and distorted logical semantics. Summary of the Invention

[0006] Long-term practice has shown that the translation of existing technical texts mainly relies on general translation tools or human literal translation, and their evaluation systems are often limited to general dimensions such as grammatical correctness and vocabulary matching. However, this approach lacks standardized consideration of the core attributes of technical texts and fails to effectively focus on key technical requirements such as the consistency of technical terminology and the accuracy of complex logical semantics. Therefore, existing solutions cannot accurately determine whether the translation results meet the stringent requirements of practical scenarios such as engineering applications. They are highly susceptible to technical cognitive biases due to inconsistent terminology or distorted logical semantics, and may even lead to misleading technical solutions during implementation, resulting in serious technical problems.

[0007] In view of this, the present invention provides a method for evaluating the quality of technical text translation, including, Step S1: Construct a technical text corpus. The corpus includes source language technical text, corresponding target language translated text, and associated annotation information between the source language technical text and the translated text. The associated annotation information is set in conjunction with the MQM quantitative evaluation system and includes at least technical field annotations, terminology sets, logical relationship annotations, technical effect description annotations, error type annotations, and error severity annotations. Step S2: Determine the dimensions for evaluating the quality of technical text translation, including at least technical accuracy, terminology consistency, language quality, and regional customary requirements; and set quantitative evaluation indicators and weights for each evaluation dimension. Step S3: Based on the associated annotation information, calculate the quantitative evaluation index for each evaluation dimension to obtain the independent score for each evaluation dimension; perform a weighted calculation by weighting the independent scores of each evaluation dimension with the corresponding dynamically adjusted weight values ​​to automatically generate the comprehensive quality score of the translated text; and determine the translation quality level of the translated text based on the comprehensive quality score according to a preset scoring threshold range.

[0008] Preferably, the associated annotation information also includes penalty score annotations, which correspond one-to-one with the error severity annotations. Different severity levels are preset with penalty score ranges to quantify the negative impact of the error.

[0009] Preferably, the quality of technical text translation is evaluated from the perspective of technical accuracy. in, To score for accuracy, The corpus matching score is used to measure the number of source language characters that are completely identical to the corpus. The value is [0, 1], calculated by the Meteor algorithm, and is used to measure overall semantic similarity and fluency.

[0010] Preferably, the terminology consistency score is: in, The percentage of words in successfully matched terms. The terminology check score is the accuracy obtained by performing contextual semantic checks and consistency checks using a large language model.

[0011] Preferably, a large language model is used to perform error checking on the translated text, and points are deducted according to minor or serious problems. The language quality score is as follows: in, Score for grammatical errors. Points are awarded for spelling errors. Points are awarded for punctuation errors. Points are awarded for incorrect matching; , , , They are respectively , , , The weighting coefficients, , , , The sum is 1.

[0012] Preferably, a large language model is used to check the translated text for errors in four aspects: regional habits, writing style, audience adaptability, and design and tagging. The regional habits score is required to be... in, Score based on regional habits. Score for writing style. For audience adaptability score, Scoring based on design and labeling. , , , They are respectively , , , The weighting coefficients, , , , The sum is 1.

[0013] Preferably, the method for assessing the quality of technical text translation further includes, Step S4: Using the source text and the translated text as input and the comprehensive quality score as output, a machine learning-based fusion model is trained and optimized using sample data from the corpus.

[0014] This invention also discloses a system for evaluating the quality of technical text translation as described above, the system comprising: The corpus construction unit is used to construct a technical text corpus, which includes source language technical text, corresponding target language translated text, and associated annotation information between the source language technical text and the translated text. The associated annotation information is set in conjunction with the MQM quantitative evaluation system and includes at least technical field annotations, terminology sets, logical relationship annotations, technical effect description annotations, error type annotations, and error severity annotations. An evaluation dimension configuration unit is used to determine the evaluation dimensions for the quality of technical text translation. The evaluation dimensions include at least technical accuracy, terminology consistency, language quality, and regional customary requirements. Quantitative evaluation indicators and weights are set for each evaluation dimension. The quality calculation and judgment unit is used to calculate the quantitative evaluation index of each evaluation dimension according to the associated annotation information to obtain the independent score of each evaluation dimension; to perform weighted calculation of the independent score of each evaluation dimension with the corresponding dynamically adjusted weight value to automatically generate the comprehensive quality score of the translated text; and to determine the translation quality level of the translated text based on the comprehensive quality score according to a preset scoring threshold range.

[0015] The present invention also discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the above-described method for evaluating the quality of technical text translation when executing the computer program.

[0016] The present invention also discloses a machine-readable storage medium storing instructions, characterized in that, when executed by a processor, the instructions cause the processor to be configured to perform the above-described method for evaluating the quality of technical text translation.

[0017] This invention provides a method for evaluating the translation quality of technical texts. It constructs a corpus of technical texts containing source language technical texts, corresponding target language translated texts, and their associated annotation information. This associated annotation information, set using the MQM quantitative evaluation system, includes at least technical field annotations, terminology sets, logical relationship annotations, technical effect description annotations, error type annotations, and error severity annotations. The method identifies at least four dimensions for evaluating the translation quality of technical texts: technical accuracy, terminology consistency, language quality, and regional customary requirements. Quantitative evaluation indicators and weights are assigned to each dimension. Independent scores for each evaluation dimension are calculated based on the associated annotation information. These independent scores are then weighted with their corresponding dynamically adjusted weights to automatically generate a comprehensive quality score for the translated text. Finally, based on a preset scoring threshold range, the comprehensive quality score determines the translation quality level of the translated text. This invention also provides a system for the aforementioned method for evaluating the translation quality of technical texts. This method and system, by introducing the MQM quantitative evaluation system and combining it with in-depth annotation information such as technical field and logical relationships, overcome the limitations of traditional general translation evaluation methods that only focus on grammatical and lexical matching. This enables precise identification and standardized, refined evaluation of the consistency of technical terminology and the accuracy of complex logical semantics in technical texts. Meanwhile, a multi-dimensional evaluation system encompassing technical accuracy and terminology consistency is constructed, employing a dynamically adjusted weighting calculation method. This allows for flexible adjustments to the evaluation focus based on the specific needs of different technical texts, ensuring that the overall quality score more accurately and objectively reflects the actual quality level of the translated text within a specific technical field. Furthermore, the system automatically determines the translation quality level through quantitative indicator calculations and preset scoring threshold ranges, significantly reducing the time cost of manual proofreading and evaluation, improving the efficiency of large-scale technical document translation processing, and providing quantifiable data support for continuous improvement of translation quality through a standardized scoring mechanism. Attached Figure Description

[0018] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is a schematic diagram of a method for evaluating the quality of technical text translation according to one embodiment of the present invention. Detailed Implementation

[0019] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0020] Currently, technical text translation largely relies on general-purpose tools, and its evaluation standards are limited to grammar and vocabulary, severely lacking a standardized system tailored to the specific characteristics of technical texts. Because core requirements such as terminology consistency and logical accuracy are neglected, existing methods often fail to meet the practical requirements of engineering applications, leading to critical problems such as translation distortion causing biases in technical understanding and misleading implementation of solutions. This invention proposes a method for evaluating the quality of technical text translation, such as... Figure 1 As shown, the method for evaluating the quality of technical text translation includes: Step S1: Construct a technical text corpus. The corpus includes source language technical text, corresponding target language translated text, and associated annotation information between the source language technical text and the translated text. The associated annotation information is set in conjunction with the MQM (Multidimensional Quality Metrics) quantitative evaluation system. The associated annotation information includes at least technical field annotations, terminology sets, logical relationship annotations, technical effect description annotations, error type annotations, and error severity annotations. The MQM quantitative evaluation system is a framework for analytical translation quality assessment, applied to the quality evaluation of human translation, machine translation, and AI-generated translation. Its core includes a standardized error classification and scoring system.

[0021] Step S2: Determine the dimensions for evaluating the quality of technical text translation, including at least technical accuracy, terminology consistency, language quality, and regional customary requirements; and set quantitative evaluation indicators and weights for each evaluation dimension. Step S3: Based on the associated annotation information, calculate the quantitative evaluation index for each evaluation dimension to obtain the independent score for each evaluation dimension; perform a weighted calculation by weighting the independent scores of each evaluation dimension with the corresponding dynamically adjusted weight values ​​to automatically generate the comprehensive quality score of the translated text; and determine the translation quality level of the translated text based on the comprehensive quality score according to a preset scoring threshold range.

[0022] This invention provides a method for evaluating the translation quality of technical texts. It constructs a corpus of technical texts containing source language technical texts, corresponding target language translated texts, and their associated annotation information. This associated annotation information is set using the MQM quantitative evaluation system and includes at least technical field annotations, terminology sets, logical relationship annotations, technical effect description annotations, error type annotations, and error severity annotations. The method identifies at least four dimensions for evaluating the translation quality of technical texts: technical accuracy, terminology consistency, language quality, and regional customary requirements. Quantitative evaluation indicators and weights are set for each evaluation dimension. Independent scores for each evaluation dimension are calculated based on the associated annotation information. These independent scores are then weighted with their corresponding dynamically adjusted weights to automatically generate a comprehensive quality score for the translated text. Finally, based on a preset scoring threshold range, the translation quality level of the translated text is determined according to the comprehensive quality score. This invention also provides a system for the aforementioned method of evaluating the translation quality of technical texts. This method, by introducing the MQM quantitative evaluation system and combining it with deep-related annotation information such as technical field annotations and logical relationship annotations, effectively overcomes the limitations of traditional general translation evaluations that only focus on grammatical correctness and lexical matching. It achieves precise identification of the consistency of professional terminology translations and the accuracy of conveying complex logical semantics in technical texts, thus achieving a standardized and refined evaluation of the translation quality of technical texts. Simultaneously, this method constructs a comprehensive evaluation system covering multiple dimensions such as technical accuracy, terminology consistency, language quality, and regional customary requirements. It adopts a dynamic weight adjustment calculation mode, which can flexibly adjust the emphasis and weight of each evaluation dimension according to the application scenarios and specific needs of different technical texts. This allows the generated comprehensive quality score to more realistically and objectively reflect the actual adaptability and quality level of the translated text in a specific technical field. Furthermore, by using a system to calculate quantitative evaluation indicators and preset scoring threshold ranges, this method can automatically determine the translation quality level, significantly reducing the time cost of manual proofreading and evaluation, and effectively improving the efficiency of large-scale technical document translation processing. At the same time, the standardized scoring mechanism provides quantifiable and traceable data support for the continuous optimization and iterative improvement of translation quality, further ensuring the professionalism and reliability of technical text translation.

[0023] In order to build a standardized, traceable, and quantifiable translation quality evaluation system to address the industry pain points that it is difficult to accurately measure the translation quality of technical texts and the impact of errors cannot be quantified. On the one hand, the clear classification of error severity and the corresponding penalty score ranges provide a unified evaluation standard for the negative impact of translation errors, avoiding subjectivity and arbitrariness in manual evaluation. On the other hand, binding the penalty scores deeply to the evaluation dimensions enables refined control of translation quality, ensuring that the evaluation results can truly reflect the translation level of technical texts, providing a clear direction for optimizing the translation quality of technical texts, while enhancing the standardization and professionalism of technical document processing to adapt to the usage requirements of various technical scenarios. In a more preferred case of the present invention, the associated annotation information further includes a penalty score annotation, and the penalty score annotation corresponds one-to-one with the error severity annotation. Different penalty score ranges corresponding to different severity levels are preset for quantitatively calculating the negative impact brought by errors. Presetting the error severity levels and the corresponding penalty score ranges, combined with the error classification criteria of the MQM quantitative evaluation system, divides the error severity into four core levels, namely serious errors (10 points), general errors (5 points), minor errors (1 point), and no errors (0 points). Each level corresponds to a preset and clear penalty score range to ensure the standardization of penalty score quantification. Serious errors (10 points) mainly include incorrect translation of professional terms, distortion of core logical semantics, etc., which may lead to errors in technical cognition deviation and misguidance of technical solution implementation. General errors (5 points) mainly include errors such as unannotated term abbreviations and incoherent non-core logical expressions, which do not affect the transmission of core technical information but affect the reading fluency. Minor errors (1 point) mainly include minor grammar flaws, improper use of punctuation marks, etc., which do not affect the transmission of technical information and reading comprehension. During the process of associated annotation of the technical text corpus, for each identified translation error, first annotate its corresponding error severity level, and then determine the specific penalty score value according to the penalty score range corresponding to this level, combined with the specific impact degree of the error, to complete the penalty score annotation, ensuring the accurate matching of the severity of each error and the penalty score value, and avoiding penalty score deviation. For example, if it is identified that "Gramian Angular Field" is mis-translated as "格拉米安角域" and the correct translation is "格拉姆角场", this error belongs to the incorrect translation of professional terms, is annotated as a serious error, and considering its impact on technical cognition, the specific penalty score is determined to be 10 points, and the corresponding penalty score annotation is completed. During the calculation process of the quantitative indicators of each evaluation dimension, the penalty score value of each error is incorporated into the deduction calculation of the corresponding evaluation dimension. Among them, errors such as incorrect translation of professional terms and term inconsistency are incorporated into the deduction of the term consistency dimension, errors such as distortion of logical semantics are incorporated into the deduction of the technical accuracy dimension, and errors such as grammar and punctuation are incorporated into the deduction of the language quality dimension; summarize the total penalty scores of each evaluation dimension, and perform corresponding deductions in the comprehensive quality score in combination with the weights of each dimension to achieve a quantitative manifestation of the negative impact of errors on translation quality.The penalty score range for each error severity level can be dynamically adjusted based on the application scenario of different technical texts, such as academic research, patent applications, and engineering applications. For example, for patent application technical texts, the penalty score range for serious errors is adjusted to 12-15 points per error, strengthening the control over the accuracy of core technical information. For engineering application texts, the penalty score range for general errors, such as incoherent logical expressions, is adjusted to 6-9 points per error, ensuring the practicality and feasibility of the translated text.

[0024] To ensure consistency between core terminology and key expressions in technical texts and corpus standards, and to uphold the fundamental principle of technical accuracy, the Meteor algorithm is introduced to calculate semantic similarity and fluency scores. This approach balances the semantic integrity and linguistic fluency of the translated text, avoiding misjudgments of semantic accuracy due to differences in literal expression, and achieving a scientific and reasonable assessment of technical accuracy. In a more preferred embodiment of this invention, the quality of the translated technical text is assessed from the perspective of technical accuracy as follows: in, To score for accuracy, The corpus matching score is used to measure the number of source language characters that are completely identical to the corpus. The value is [0, 1], calculated by the Meteor algorithm, and is used to measure overall semantic similarity and fluency. In the method provided by this invention, a dedicated corpus of technical texts is built in advance, clearly defining core terms, standard expressions, sentence structure norms, etc., within the corpus, which serve as the corpus matching score. This serves as the benchmark for computation. Simultaneously, the Meteor algorithm tool was deployed, and its parameters were debugged to ensure accurate calculation of semantic similarity and fluency of text across different technical fields. To adapt to the specialized characteristics of technical texts, the corpus matching score is calculated by comparing the translated technical text to a pre-defined corpus, word by word and sentence by sentence. The number of source language words that are completely identical to those in the corpus is counted, and the proportion of these words to the total number of words in the translated text is calculated. This proportion is the specific value of Sy, for example, ranging from [0, 1]. A higher proportion indicates a higher degree of matching between the translated text and the standard corpus. Furthermore, semantic similarity and fluency scores are calculated. Input the translated text to be evaluated into the debugged Meteor algorithm tool. The algorithm will analyze it from three dimensions: word matching, sentence coherence, and semantic completeness, and automatically output the results. The score, ranging from [0, 1], indicates that the translated text conveys semantics more accurately, expresses language more fluently, and has no semantic bias or expression gaps. The final score Sz for technical accuracy is calculated by combining the Sy and Smeteor values ​​obtained in the previous steps, with a value ranging from [0, 1]. Specifically, when the translated text is completely consistent with the corpus, Sy=1 and Sz=1, meaning the accuracy score is full. When the translated text has no matching content with the corpus, Sy=0 and Sz=Smeteor, using only semantic similarity as the accuracy criterion, while also considering the flexibility of translation. The calculated Sz score is incorporated into the comprehensive evaluation system of technical text translation quality, combined with scores from other evaluation dimensions such as terminology consistency and language fluency, to form a complete translation quality evaluation result. Simultaneously, based on the Sz score, weaknesses in the technical accuracy of the translated text are identified; for example, a low Sy score indicates a focus on optimizing terminology standardization, and a low Smeteor score indicates a focus on optimizing semantic delivery.

[0025] In a more preferred embodiment of the present invention, the terminology consistency score is: in, The percentage of words in successfully matched terms. The accuracy rate for terminology checking is obtained through contextual semantic checks and consistency checks using a large language model. A professional terminology database corresponding to the technical text's domain is constructed, clarifying the standard translations, common collocations, and usage scenarios of various terms in the database, serving as a benchmark for terminology matching. Simultaneously, a large language model adapted to the technical text's domain is deployed, and model parameters are debugged to ensure it can accurately identify the usage scenarios and contextual semantics of terms in the text, possessing efficient terminology consistency checking capabilities. The percentage of successfully matched terms (Ssp) is calculated. Term extraction is performed on the translation results of the technical text to be evaluated, filtering out all professional terms in the text. The extracted terms are compared one by one with the standard translations in the preset professional terminology database. The total number of successfully matched terms is counted, and their proportion to the total number of professional terms in the text is calculated. This proportion is the specific value of Ssp [0, 1]. The higher the proportion, the stronger the standardization of the terminology translation. The terminology check score Ssc is calculated by extracting the technical text translations of all professional terms and inputting them into a debugged large language model. The model checks the terminology from two core dimensions: first, contextual semantics, which determines whether the translation matches the semantics of the text context and is consistent; second, terminology consistency, which determines whether the translation of the same term is consistent across different paragraphs and contexts and is not contradictory. The model outputs an accuracy rate based on the check results, which is the specific value of Ssc [0, 1]. The higher the accuracy rate, the better the consistency and semantic fit of the terminology. The final terminology consistency score Ss is calculated, with a value range of [0, 1]. The values ​​of Ssp and Ssc obtained in the above steps are used to derive the final score Ss for the terminology consistency dimension. When all terms completely match the terminology database, Ssp=1 and Ss=1, meaning the terminology consistency score is full. When no term matches the terminology database, Ssp=0 and Ss=Ssc. Only the terminology consistency and semantic fit checked by the large language model are used as the evaluation criteria, taking into account both the flexibility and rationality of terminology use. The calculated Ss score is incorporated into a comprehensive evaluation system for the quality of technical text translation, and combined with scores from other evaluation dimensions such as technical accuracy and language quality to form a complete translation quality evaluation result.

[0026] To develop targeted optimization strategies based on the severity of errors and improve the language accuracy and fluency of the translated text, in a more preferred embodiment of the invention, a large language model is used to perform error checking on the translated text, deducting points according to minor or serious problems. The language quality score is... in, Score for grammatical errors. Points are awarded for spelling errors. Points are awarded for punctuation errors. Points are awarded for incorrect matching; , , , They are respectively , , , The weighting coefficients, , , , The sum of these factors equals 1. Deploy a large language model adapted to the technical text domain, complete model parameter debugging, and clarify the model's identification criteria for grammatical errors, spelling errors, punctuation errors, and collocation errors to ensure accurate differentiation of various language errors and identification of their severity. Based on the application scenario of the technical text, preset four weight coefficients (α, β, γ, δ) to ensure the sum of these coefficients equals 1. The weight allocation is tailored to the scenario requirements. For example, patent texts emphasize punctuation and spelling accuracy, so β=0.3, γ=0.3, α=0.2, δ=0.2; academic texts emphasize grammatical and collocational rigor, so α=0.3, δ=0.3, β=0.2, γ=0.2. For error checking and severity determination in the large language model, input the translation results of the technical text to be evaluated into the debugged large language model. The model scans the text sentence by sentence, automatically identifying the four types of problems: grammatical errors, spelling errors, punctuation errors, and collocation errors, and marking the severity of each type of error. Serious problems refer to errors that affect text comprehension, cause semantic ambiguity, or create confusion, such as grammatical inconsistencies, misspellings of core vocabulary, missing or misused key punctuation, and incorrect core collocations. Minor problems refer to errors that do not affect text comprehension but only reading fluency, such as non-core grammatical flaws, minor spelling errors, non-standard punctuation usage that does not affect meaning, and inappropriate non-core collocations.

[0027] To improve the regional adaptability, stylistic consistency, audience relevance, and design tagging rationality of translated texts, in a more preferred embodiment of this invention, a large language model is used to perform error checks on the translated text in four aspects: regional habits, writing style, audience adaptability, and design and tagging. The regional habits score is required to be... in, Score based on regional habits. Score for writing style. For audience adaptability score, Scoring based on design and labeling. , , , They are respectively , , , The weighting coefficients, , , , The sum of these factors is 1. Deploy a large language model adapted to the technical text domain and target region, complete model parameter debugging, and clarify the model's identification criteria for errors in four aspects: regional habits, writing style, audience adaptability, and design and marking. This ensures that the model can accurately distinguish the severity of various problems, while adapting to the language habit norms of different regions and the writing style requirements of different technical texts. Combining the application scenario, target region, and audience of the technical text, preset four weight coefficients: λ, μ, ν, and ξ, ensuring that the sum of the four coefficients is 1. The weight allocation fits the actual needs. For example, for patent texts in a specific region, focusing on regional habits and design markings, λ=0.3, ξ=0.3, μ=0.2, and ν=0.2 can be set. For general academic texts, focusing on writing style and audience adaptability, μ=0.3, ν=0.3, λ=0.2, and ξ=0.2 can be set. The large language model employs multi-dimensional error checking and severity assessment. The translated technical text to be evaluated is input into the pre-tuned large language model, which performs a comprehensive scan of the text sentence by sentence and paragraph by paragraph. The model conducts specific checks on four aspects: regional customs, writing style, audience adaptability, and design and tagging. It automatically identifies various problems and labels the severity of each type. Severe problems refer to errors that affect text dissemination, lead to poor regional adaptability, cause difficulty in audience comprehension, or result in chaotic design tagging. Examples include violations of core language customs in the target region, writing style significantly incompatible with the technical text type, complete incompatibility with the audience's cognitive level, and missing / incorrect core design tagging. Minor problems refer to those that do not affect core dissemination and comprehension and require only minor adjustments and optimizations. Examples include slight deviations in regional customs, minor inconsistencies in writing style, minor deficiencies in audience adaptability, and minor non-standard design tagging formats.

[0028] The scores for the four dimensions Sqy, Sxz, Ssz, and Ssj are all converted to a percentage scale and range from [0, 1]. The calculation logic is consistent and strictly combines the severity of the errors, as follows: The regional habit score Sqy is the total number of regional habit-related errors in the text, distinguishing between serious and minor issues. Serious regional habit errors, such as violations of target region terminology, grammar, and expression habits, are deducted 0.2 points each. Minor regional habit errors, such as deviations in regional expression details or non-core habit inconsistencies, are deducted 0.05 points each. A maximum score of 1 is used. After deducting the scores corresponding to all regional habit errors, Sqy is obtained. If the score after deduction is negative, it is set to 0. For example, if there is 1 serious regional habit error and 3 minor regional habit errors in the text, the deduction score is 0.2 + 0.05 × 3 = 0.35, then Sqy = 1 - 0.35 = 0.65.

[0029] The writing style score Sxz represents the total number of writing style-related errors in the text, distinguishing between serious and minor issues. Serious writing style errors, such as overly colloquial technical patent texts, overly casual academic texts, or overly obscure engineering documents, deduct 0.2 points for each error. Minor writing style errors, such as slightly inconsistent style or expressions not fitting the text type, deduct 0.04 points for each error. The maximum score is 1. Sxz is calculated by deducting the points for all writing style errors. If the score after deductions is negative, it is set to 0.

[0030] The audience adaptability score Ssz is the total number of audience adaptability-related errors in the text, distinguishing between serious and minor issues. Serious audience adaptability errors, such as technical texts being too complex and beyond the audience's cognitive level, or expressions being too simplistic and failing to meet the audience's needs, are deducted 0.2 points each. Minor audience adaptability errors, such as individual expressions not being sufficiently relevant and requiring minor adjustments to suit the audience, are deducted 0.04 points each. With 1 point as the maximum score, Ssz is obtained by deducting the scores corresponding to all audience adaptability errors. If the score is negative after deduction, it is set to 0.

[0031] The design and labeling score, Ssj, is the total number of design and labeling-related errors in the text, distinguishing between serious and minor issues. Serious errors, such as inconsistent heading hierarchy, incorrect chart labeling, missing formula labels, and incorrect key content labeling, deduct 0.25 points for each error. Minor errors, such as non-standard labeling formatting and deviations from non-key labeling, deduct 0.05 points for each. A maximum score of 1 is used. Ssj is calculated by deducting the points corresponding to all design and labeling errors. If the score is negative after deduction, it is set to 0. The preset weighting coefficients λ, μ, ν, and ξ are checked to ensure their sum is 1. If the target area, audience, or application scenario of the technical text changes, the weights can be dynamically adjusted, but the sum of the four factors must still be 1. This approach improves the flexibility of the solution and avoids modifying the scoring formula itself.

[0032] To address the fragmented nature of existing assessment technologies, where scores across different dimensions are independent and fail to intuitively reflect the overall quality of the text, and to achieve a comprehensive and quantitative evaluation of translation quality, making the assessment results more intuitive and valuable for reference, the overall text translation quality score is proposed. , , , , For weights. Through... , , , The flexible setting of the four weighting coefficients allows for adjustments to the influence weights of each dimension based on the evaluation focus of different technical fields and application scenarios, without requiring modifications to the calculation logic of each dimension or the core system structure. Compared to fixed-weight evaluation methods, this scheme is more adaptable, meeting the comprehensive evaluation needs of various technical documents and improving the relevance and practicality of the evaluation. The weight allocation is consistent with the setting of the evaluation dimension configuration units and the training logic of the fusion model.

[0033]

[0034] To significantly improve evaluation efficiency, adapt to large-scale processing, achieve intelligent evaluation processes, reduce labor costs, and enhance overall reliability, in a more preferred embodiment of the present invention, the method for evaluating the quality of technical text translation further includes, Step S4: Using the source text and translated text as input and the comprehensive quality score as output, a machine learning-based fusion model is trained and optimized using sample data from the corpus. Sample data preparation involves selecting technical text samples from the corpus covering different technical fields (e.g., machinery, electronics, new energy, chemical engineering); different application scenarios (e.g., patents, academic papers, engineering documents); and different translation quality levels. Each sample includes the complete source text, the corresponding translated text, and a standardized comprehensive quality score calculated based on the aforementioned evaluation dimensions of technical accuracy, terminology consistency, language quality, and regional custom requirements, with values ​​ranging from [0, 1]. Training, validation, and test sample sets are constructed, with a ratio of 7:2:1 to ensure sample diversity and representativeness and avoid model overfitting. Model selection and initialization involve selecting a machine learning foundation model suitable for the text processing scenario, such as a CNN-LSTM fusion model or a Transformer model as the basic framework. This framework must possess strong text feature extraction capabilities and simultaneously capture the semantic relationships, grammatical features, terminology features, and regional custom features of the source and translated texts. The model is initialized by determining the structural parameters of the input, hidden, and output layers. The input layer corresponds to the feature vectors of the source and translated texts, and the output layer corresponds to the overall quality score. The number of hidden layers and neurons is dynamically adjusted according to the sample size. Feature extraction and input processing are performed on the source and translated texts in the training sample set. Core features are extracted and transformed into feature vectors that the model can recognize. These include semantic similarity features, terminology consistency features, grammatical norm features, regional habit adaptation features, and the intermediate scores Sz, Ss, Sq, and Sbd features of each evaluation dimension mentioned above. All features are normalized and mapped to the [0, 1] interval to avoid the impact of differences in the dimensions of different features on the model training effect, thus forming the model input data. Model training and parameter optimization involve inputting the preprocessed training samples into the initialized machine learning model. The overall quality score is used as the output target, and the mean squared error (MSE) is used as the loss function to measure the deviation between the model's predicted score and the actual overall quality score of the sample. The model parameters are iteratively updated using the gradient descent algorithm, and key parameters such as hidden layer weights and learning rate are adjusted. After each iteration, the model performance is verified using a validation sample set. If the model prediction deviation exceeds a preset threshold, iterative optimization continues until the model's prediction accuracy on the validation sample set reaches a preset standard and the prediction error is ≤5%. At this point, training is stopped, and a preliminary fusion model is obtained.The test sample set is input into the trained fusion model to obtain the model's overall predicted quality score. This score is compared with the actual overall quality score of the samples to calculate evaluation metrics such as prediction accuracy and average error. If the metrics do not meet the preset requirements, for cases with large prediction deviations in the samples, sample data from the corresponding domain is added, and local iterations are performed again to fine-tune the model, optimizing its adaptability to technical texts in specific scenarios and domains. The final result is an optimized machine learning-based fusion model. This optimized fusion model is then applied to technical text translation quality assessment scenarios. The source and translation texts to be assessed can be directly input into the model to quickly output an overall quality score, eliminating the need for manual calculation of scores for each evaluation dimension. Simultaneously, new technical text translation samples and their corresponding actual evaluation scores are regularly collected and added to the corpus for periodic iterative training of the model, continuously optimizing its performance and adapting to constantly updated technical terminology, regional conventions, and text styles.

[0035] This invention also discloses a system for evaluating the quality of technical text translation as described above, the system comprising: The corpus construction unit is used to construct a technical text corpus, which includes source language technical text, corresponding target language translated text, and associated annotation information between the source language technical text and the translated text. The associated annotation information is set in conjunction with the MQM quantitative evaluation system and includes at least technical field annotations, terminology sets, logical relationship annotations, technical effect description annotations, error type annotations, and error severity annotations. An evaluation dimension configuration unit is used to determine the evaluation dimensions for the quality of technical text translation. The evaluation dimensions include at least technical accuracy, terminology consistency, language quality, and regional customary requirements. Quantitative evaluation indicators and weights are set for each evaluation dimension. The quality calculation and judgment unit is used to calculate the quantitative evaluation index of each evaluation dimension according to the associated annotation information to obtain the independent score of each evaluation dimension; to perform weighted calculation of the independent score of each evaluation dimension with the corresponding dynamically adjusted weight value to automatically generate the comprehensive quality score of the translated text; and to determine the translation quality level of the translated text based on the comprehensive quality score according to a preset scoring threshold range.

[0036] The technical text translation quality assessment system disclosed in this invention is deeply adapted to the assessment methods described above. Through the collaborative operation of the corpus construction unit, the assessment dimension configuration unit, and the quality calculation and judgment unit, it achieves the standardization, systematization, and automation of technical text translation quality assessment. The assessment dimensions are more systematic, the weight configuration is more flexible, and it adapts to the needs of multiple scenarios.

[0037] The present invention also discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the above-described method for evaluating the quality of technical text translation when executing the computer program.

[0038] The present invention also discloses a machine-readable storage medium storing instructions, characterized in that, when executed by a processor, the instructions cause the processor to be configured to perform the above-described method for evaluating the quality of technical text translation.

[0039] In the above embodiments, the descriptions of each embodiment have different focuses. Parts not described in detail in a certain embodiment can be referred to in the relevant descriptions of other embodiments. The above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for evaluating the quality of technical text translation, characterized in that, The method for assessing the quality of technical text translation includes: Step S1: Construct a technical text corpus. The corpus includes source language technical text, corresponding target language translated text, and associated annotation information between the source language technical text and the translated text. The associated annotation information is set in conjunction with the MQM quantitative evaluation system and includes at least technical field annotations, terminology sets, logical relationship annotations, technical effect description annotations, error type annotations, and error severity annotations. Step S2: Determine the dimensions for evaluating the quality of technical text translation, including at least technical accuracy, terminology consistency, language quality, and regional customary requirements; and set quantitative evaluation indicators and weights for each evaluation dimension. Step S3: Based on the associated annotation information, calculate the quantitative evaluation index for each evaluation dimension to obtain the independent score for each evaluation dimension; perform a weighted calculation by weighting the independent scores of each evaluation dimension with the corresponding dynamically adjusted weight values ​​to automatically generate the comprehensive quality score of the translated text; and determine the translation quality level of the translated text based on the comprehensive quality score according to a preset scoring threshold range.

2. The method for evaluating the quality of technical text translation according to claim 1, characterized in that, The associated annotation information also includes penalty score annotations, which correspond one-to-one with the error severity annotations. Different severity levels are preset with penalty score ranges to quantify the negative impact of the error.

3. The method for evaluating the quality of technical text translation according to claim 1, characterized in that, The quality of technical text translation is assessed from the perspective of technical accuracy. in, To score for accuracy, The corpus matching score is used to measure the number of source language characters that are completely identical to the corpus. The value is [0, 1], calculated by the Meteor algorithm, and is used to measure overall semantic similarity and fluency.

4. The method for evaluating the quality of technical text translation according to claim 1, characterized in that, The terminology consistency score is: in, The percentage of words in successfully matched terms. The terminology check score is the accuracy obtained by performing contextual semantic checks and consistency checks using a large language model.

5. The method for evaluating the quality of technical text translation according to claim 1, characterized in that, Error checking of the translated text is performed using a large language model, with points deducted for minor or serious issues. The resulting language quality score is... in, Score for grammatical errors. Points are awarded for spelling errors. Points are awarded for punctuation errors. Points are awarded for incorrect matching; , , , They are respectively , , , The weighting coefficients, , , , The sum is 1.

6. The method for evaluating the quality of technical text translation according to claim 1, characterized in that, A large language model was used to check the translated text for errors in four areas: regional habits, writing style, audience adaptability, and design and tagging. The regional habits score was required to be... in, Score based on regional habits. Score for writing style. For audience adaptability score, Scoring based on design and labeling. , , , They are respectively , , , The weighting coefficients, , , , The sum is 1.

7. The method for evaluating the quality of technical text translation according to any one of claims 1-6, characterized in that, The method for evaluating the quality of technical text translation also includes... Step S4: Using the source text and the translated text as input and the comprehensive quality score as output, a machine learning-based fusion model is trained and optimized using sample data from the corpus.

8. A system for evaluating the quality of technical text translation as described in any one of claims 1-7, characterized in that, The system includes, The corpus construction unit is used to construct a technical text corpus, which includes source language technical text, corresponding target language translated text, and associated annotation information between the source language technical text and the translated text. The associated annotation information is set in conjunction with the MQM quantitative evaluation system and includes at least technical field annotations, terminology sets, logical relationship annotations, technical effect description annotations, error type annotations, and error severity annotations. An evaluation dimension configuration unit is used to determine the evaluation dimensions for the quality of technical text translation. The evaluation dimensions include at least technical accuracy, terminology consistency, language quality, and regional customary requirements. Quantitative evaluation indicators and weights are set for each evaluation dimension. The quality calculation and judgment unit is used to calculate the quantitative evaluation index of each evaluation dimension according to the associated annotation information to obtain the independent score of each evaluation dimension; to perform weighted calculation of the independent score of each evaluation dimension with the corresponding dynamically adjusted weight value to automatically generate the comprehensive quality score of the translated text; and to determine the translation quality level of the translated text based on the comprehensive quality score according to a preset scoring threshold range.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the technology-oriented text translation quality assessment method according to any one of claims 1-7.

10. A machine-readable storage medium storing instructions thereon, characterized in that, When executed by a processor, this instruction causes the processor to be configured to perform the technology-oriented text translation quality assessment method according to any one of claims 1-7.

Images