A method for evaluating the quality of foreign language geographical name translation software

Abstract

The application discloses a kind of foreign language place name translation software quality evaluation method, comprising the following steps: step one, constructing place name database;Step two, the component annotation of foreign language place name in the data set to be measured;Step three, foreign language place name is input foreign language place name translation software one by one, if corresponding standard Chinese translation name can be directly inquired from reference data set, then quality evaluation score is the highest score;Otherwise, enter step four;Step four, translation and calculate quality evaluation score.Step five, traverse data set to be measured, obtain the quality evaluation score of each foreign language place name.Step six, the score of foreign language place name translation software is calculated.The method of the application is simple, reasonable in design, aimed at Chinese translation name acquisition place name component, for different place name component reasonable distribution confidence, output accurate quality evaluation score, and place name is divided into superior data and data that need to be approved, clear the focus of manual review, improve the efficiency of evaluation method, improve the adaptability of evaluation method.

Description

Technical Field

[0001] This invention relates to the field of translation evaluation technology, and more specifically, to a method for evaluating the quality of foreign language place name translation software. Background Technology

[0002] Currently, in the field of computer place name translation, the mainstream method for evaluating the quality of foreign language place name translation software is still mainly manual review, supplemented by simple rule matching models.

[0003] Manual review typically involves professionals verifying the translation results one by one based on official reference books such as the "Guidelines for the Translation of Foreign Place Names into Chinese Characters". The manual review model requires a large amount of professional human resources, is inefficient, and is difficult to adapt to the verification needs of large-scale place name data. Furthermore, the review results are easily affected by the subjective judgment of the personnel, and consistency and objectivity are difficult to guarantee.

[0004] The rule-matching model strictly adheres to official specifications, but its verification dimensions are limited. It can only perform simple automatic checks for obvious spelling, formatting errors, and deviations in the translation of generic terms. It cannot effectively evaluate core translation quality dimensions such as proper noun transliteration, component weight matching, and correspondence with land cover types. Therefore, there is an urgent need to develop a method that can systematically evaluate the output quality of foreign language place name translation software. Summary of the Invention

[0005] The technical problem to be solved by this application is to provide a quality evaluation method for foreign language place name translation software, which addresses the shortcomings of the existing technology. The method is simple and reasonably designed. It obtains place name components from Chinese translations, reasonably assigns reliability to different place name components, outputs accurate quality evaluation scores, and divides place names into excellent data and data requiring verification. It clarifies the focus of manual review, improves the efficiency of the evaluation method, and enhances the adaptability of the evaluation method.

[0006] To solve the above-mentioned technical problems, the technical solution adopted in this application is: This invention provides a method for quality evaluation of foreign language place name translation software, characterized by the following steps: Step 1: Construct a place name database, which includes a place name result library and a component matching library. The place name result library includes a benchmark dataset and a dataset to be tested. Step 2: Label the foreign place names in the dataset to be tested. Step 3: Input the foreign place names in the dataset to be tested into the foreign place name translation software one by one. If the corresponding standard Chinese translation can be found directly from the benchmark dataset, the quality evaluation score F is the highest score, and proceed to step 5; otherwise, proceed to step 4. Step 4: Translate and calculate the quality evaluation score: Step 401: The foreign language place name translation software outputs the Chinese translation of the current foreign language place name and obtains the place name components corresponding to the Chinese translation; Step 402: Obtain the weights of the regional name components in the Chinese translations based on the weight mapping table; Step 403: Assign reliability scores to the regional name components of the Chinese translations; Step 404: Calculate the quality evaluation score F based on the weight and confidence level of the place name components; Step 405: If the quality evaluation score F is greater than the quality evaluation threshold F set Data that is deemed excellent is classified as such; otherwise, it is classified as data requiring approval. Step 5: Traverse the dataset to be tested and obtain the quality evaluation score for each foreign place name.

[0007] Step 6: Obtain the score for the foreign language place name translation software based on the quality evaluation score of each foreign language place name.

[0008] The aforementioned method for evaluating the quality of foreign language place name translation software is characterized by the following specific method for assigning reliability scores to each place name component: Step 4031: If the place name component can be directly queried from the benchmark dataset, the confidence level is 100%; otherwise, proceed to step 4032. Step 4032: For the generic term portion of the place name component, determine the confidence level based on the element coding class code matching-confidence level mapping table; generic terms often have standard translations, while proper names rely more on transliteration rules. Step 4033: For the proper noun portion of the place name component, compare the syllable segmentation results of the rule model and the machine learning model. If the syllable segmentation results are consistent, the confidence level A = 100%; if the segmentation positions and the number of syllables are inconsistent, the confidence level... Where q represents the difference in the number of letters in the syllable segmentation results output by the rule model and the machine learning model, λ represents the penalty coefficient, λ∈[0.5,1], n1 represents the number of syllable segments output by the rule model, n2 represents the number of syllable segments output by the machine learning model, and L avg This indicates the average number of letters per syllable. Q represents the total number of letters in the proper noun; syllable segmentation results show inconsistent segmentation positions but consistent number of syllable segments, with a confidence level of [missing information]. .

[0009] The aforementioned method for evaluating the quality of foreign language place name translation software is characterized by the following specific method for segmenting the proper noun portion of the Chinese translation using a rule model: Standardize the proper noun portion of the Chinese translation; The official transliteration table was split to construct a transliteration dictionary with key-value pairs of Chinese transliteration units and foreign language original syllables; Candidate syllable boundaries are detected based on phonetic features. Using candidate syllable boundaries as constraints, a transliteration dictionary is retrieved using a forward maximum matching algorithm to perform syllable segmentation.

[0010] The aforementioned method for evaluating the quality of foreign language place name translation software is characterized by the following specific method for segmenting the proper noun portion of the Chinese translation using a machine learning model: Collect corpora, each corpus including the proper noun portion of the Chinese translation and ground truth boundary annotations; convert the proper noun portion of the Chinese translation into a sequence of subwords; construct the dataset; Configure the basic parameters of the first Transformer model; Given the input dataset, train the first Transformer model with the optimization objective being to minimize the cross-entropy loss function between the predicted boundary probabilities and the true boundary labels. The proper noun portion of the Chinese translation to be processed is input into the first Transformer model. The first Transformer model outputs a predicted boundary probability p for every two adjacent letters in the proper noun portion. If the predicted boundary probability p is greater than the segmentation threshold, the position is determined as the syllable boundary, and the syllable segmentation result of the machine learning model is obtained.

[0011] The aforementioned method for evaluating the quality of foreign language place name translation software is characterized by the following specific method for syllable segmentation using a machine learning model: Collect bilingual parallel corpora. Each corpus includes a foreign proper name, the proper name portion of the corresponding Chinese translation, and the ground truth boundary label corresponding to the proper name portion of the Chinese translation. Convert both the foreign proper name and the proper name portion of the Chinese translation into subword sequences. Construct a dataset. The machine learning model includes a first Transformer model and a second Transformer model, and the basic parameters of the first Transformer model and the second Transformer model are configured. The first Transformer model takes as input a sequence of sub-words from the proper noun portion of the Chinese translation, and its optimization objective is to minimize the cross-entropy loss function between the predicted boundary probabilities and the true boundary labels. It outputs a sequence of predicted boundary probabilities for the syllable boundaries of the proper noun portion of the Chinese translation, thus obtaining the syllable segmentation result. The second Transformer model takes as input the syllable segmentation result output by the first Transformer model, and its optimization objective is to maximize the cross-entropy loss function of the character similarity between the reconstructed Chinese translation and the true Chinese translation. It outputs the probability distribution of Chinese characters at each position in the proper noun portion of the Chinese translation, thus obtaining the character sequence of the reconstructed Chinese translation. The first and second Transformer models are trained jointly. The proper noun portion of the Chinese translation to be processed is input into the first Transformer model. The first Transformer model outputs the predicted boundary probability p for every two adjacent letters in the foreign proper noun, representing the confidence that the position is a syllable boundary. If the confidence is greater than the segmentation confidence threshold, the position is determined as a syllable boundary, and the syllable segmentation result of the machine learning model is obtained.

[0012] The quality evaluation method for the aforementioned foreign language place name translation software is characterized in that: step 404, the specific method for calculating the quality evaluation score F based on the weight and confidence level of the place name components, is as follows: quality evaluation score F = (A×B+C×D+E×G)×100%+σ; where A represents the confidence level of the proper noun part, B represents the weight of the proper noun, C represents the confidence level of the generic noun part, D represents the weight of the generic noun, E represents the confidence level of the adjective part, G represents the weight of the adjective, and σ represents the correction parameter.

[0013] The aforementioned method for evaluating the quality of foreign language place name translation software is characterized by: a quality evaluation threshold F. set For dynamic thresholds, ,in Indicates the basic threshold. Indicates the language adjustment factor. This represents the country adjustment factor. This represents the data distribution adjustment coefficient.

[0014] The aforementioned method for quality evaluation of foreign language place name translation software is characterized by the following: In step two, after labeling the foreign place names in the dataset to be tested, the foreign place names are divided into W major categories, including at least a set of proper names consisting only of proper names. Then, the dataset to be tested is preprocessed. The specific method for preprocessing the dataset to be tested is as follows: Step 201: Segment the foreign place names in the dataset to be tested according to words, filter out high-frequency words, determine the composition of high-frequency words based on the land cover type, and add high-frequency words to the place name database; Step 202: For the set of proper names, create a nested dictionary: {Foreign place names: {Chinese place names 1: f1; ...; Chinese place names n: f n The system iterates through the proper noun set to obtain multiple Chinese place names that contradict the same foreign place name and their frequency of occurrence. It then determines the standard Chinese place name corresponding to the foreign place name and adds it to the place name database.

[0015] The quality evaluation method for foreign language place name translation software described above is characterized in that: the weight mapping table in step 402 is a mapping table of place name components-place name types-weights.

[0016] The quality evaluation method for the aforementioned foreign language place name translation software is characterized by: labeling the place names in the test dataset with their components, and classifying the place names into five categories: consisting only of proper nouns; consisting of proper nouns and generic nouns; consisting of adjectives or locative words and proper nouns; consisting of adjectives or locative words and proper nouns and generic nouns; and consisting of adjectives or locative words and generic nouns.

[0017] This application has the following advantages compared with the prior art: 1. The process of this application is simple, the design is reasonable, and it is convenient to implement and use.

[0018] 2. This application addresses the acquisition of place name components from Chinese translations, reasonably assigns reliability to different place name components, outputs accurate quality evaluation scores, and classifies place names into superior data and data requiring verification, clarifying the focus of manual review and improving the efficiency of the evaluation method.

[0019] 3. This application improves the place name database by mining high-frequency words and sorting out contradictory translations, thereby enhancing the coverage and accuracy of the place name database for the dataset to be tested and improving the adaptability of the evaluation method.

[0020] 4. This application assigns confidence scores to the proper noun portion of place names, using the syllable segmentation results of two independent models—a rule-based model and a machine learning model—as the objective basis for judgment. It also divides the confidence score calculation into three methods based on whether the segmentation position and number are consistent, thereby improving the targeting of the review.

[0021] 5. The machine learning model in this application provides two methods. When the benchmark database has sufficient data and the labeled data used for training is high-quality manually labeled data, only the first Transformer model is used. However, when the sample size is insufficient, the first Transformer model and the second Transformer model are used in combination, which yields better results.

[0022] In summary, this application addresses the acquisition of place name components from Chinese translations, rationally assigns reliability to different place name components, outputs accurate quality evaluation scores, and classifies place names into superior data and data requiring verification, clarifying the focus of manual review, thereby improving the efficiency and adaptability of the evaluation method.

[0023] The technical solution of this application will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0024] Figure 1 This is a flowchart of the method of the present invention.

[0025] Figure 2 This is a flowchart of the method for calculating quality evaluation scores according to the present invention.

[0026] Figure 3 This is a flowchart illustrating the method for assigning confidence scores to the proper noun portion of place names according to the present invention. Detailed Implementation

[0027] The present application will now be described in further detail with reference to the accompanying drawings and embodiments thereof.

[0028] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0029] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0030] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0031] For ease of description, spatial relative terms such as "above," "on top of," "on the upper surface of," "above," etc., are used herein to describe the spatial positional relationship of a device or feature as shown in the figures to other devices or features. It should be understood that spatial relative terms are intended to encompass different orientations in use or operation beyond the orientation of the device as described in the figures. For example, if the device in the figures were inverted, a device described as "above" or "on top of" other devices or structures would subsequently be positioned as "below" or "under" other devices or structures. Thus, the exemplary term "above" can include both "above" and "below." The device may also be positioned in other different ways (rotated 90 degrees or in other orientations), and the spatial relative descriptions used herein will be interpreted accordingly.

[0032] Example 1 like Figure 1 As shown, the quality evaluation method for foreign language place name translation software of the present invention is characterized by comprising the following steps: Step 1: Construct a place name database, which includes a place name result library and a component matching library. The place name result library includes a benchmark dataset and a dataset to be tested.

[0033] The benchmark dataset covers currently publicly available authoritative place name data, such as the "World Standard Place Names Atlas" published by China Map Press and the "21st Century World Place Names Directory" published by Modern Press. The benchmark dataset is a mapping set of (authoritative place names, standard Chinese translations).

[0034] The dataset to be tested typically comes from open-source data (such as OSM, GeoNames, Google Maps, etc.), commercial data (such as Here, TomTom, etc.), and data from national ministries. The dataset to be tested is a mapping set of (unpublished official place names, with different translations).

[0035] Step 2: Perform component labeling on the foreign language place names in the dataset to be tested. In one possible embodiment, the component matching library includes a generic name library, a personal name library, and other libraries. Component labeling of the place names in the dataset to be tested is performed based on rule-based matching, matching the place name text in the dataset to the sub-libraries in the component matching library and then labeling the matches.

[0036] It should be noted that the data in the generic name database primarily comes from the *Guidelines for the Translation of Foreign Place Names into Chinese Characters* and the *Handbook for the Translation of Commonly Used Words in World Place Names* for the corresponding languages. The data in the personal name database primarily comes from the *Guidelines for the Translation of Foreign Place Names into Chinese Characters* and the *World Dictionary of Personal Names* for the corresponding languages. Other databases contain other words in place names that may require translation, including numerals, dates, adjectives, directional words, and even ethnic and religious names in the corresponding languages. The data in these other databases also comes from the *Guidelines for the Translation of Foreign Place Names into Chinese Characters* for the corresponding languages. For any incomplete coverage, supplementation based on expert knowledge is necessary.

[0037] The specific method is as follows: First, the place name text in the dataset to be tested is split into matching text fragments. If the text fragment matches the generic name database, it is marked as a generic name. If the place name fragment matches the personal name database, it is marked as a personal name. If the place name fragment matches the administrative division database, it is marked as an administrative division. If none of the fragments match any sub-database in the component matching database, the fragment is marked as a proper name.

[0038] The place names in the dataset to be tested are labeled with their components and divided into five categories: those consisting only of proper names; those consisting of proper names and generic names; those consisting of adjectives or locative words and proper names; those consisting of adjectives or locative words and proper names and generic names; and those consisting of adjectives or locative words and generic names.

[0039] The datasets to be tested generally come from open source data (such as OSM, GeoNames, Google Maps, etc.), commercial data (such as Here, TomTom, etc.), and data from national ministries. The Chinese translations of such place names vary greatly, and they need to be preprocessed such as cleaning and authentication before they can be stored in the database.

[0040] The specific method for processing the dataset to be tested is as follows: Step 201: Segment the foreign place names in the dataset to be tested according to words, filter out high-frequency words, determine the composition of high-frequency words based on the land cover type of the high-frequency words, and add the high-frequency words to the place name database.

[0041] In one possible implementation, a place name consists of 2 to 5 words. The place name is segmented into words, and the number of words and the frequency of each word are counted. If the number of words in the dataset is less than 10,000, words that appear more than 10 times are considered high-frequency words; if the number of words in the dataset is more than 10,000, words that appear more than one-thousandth of the total number of words are considered high-frequency words.

[0042] High-frequency words are extracted, their corresponding land cover types are obtained, their components are determined, and they are added to the corresponding sub-database of the component matching library in the place name database.

[0043] For example, statistical analysis reveals that the word "weir" appears frequently in the dataset to be tested. Among the place names containing "weir", more than 50% of the type descriptions are related to dams or levees. It is determined that the word "weir" is a common name, and "weir" is added to the common name library in the component matching library.

[0044] Step 202: Create a nested dictionary for the proper name set, {foreign place name: {Chinese place name 1: f1;...; Chinese place name n: f n}}, Traverse the proper name set to obtain multiple Chinese place names with contradictions for the same foreign place name and their occurrence frequencies, determine the standard Chinese place name corresponding to the foreign place name, and add it to the benchmark dataset in the place name database.

[0045] In a possible embodiment, all place names are split into combinations of "foreign language + corresponding Chinese translation". Traverse the proper name set. Each time a combination of "foreign language + corresponding Chinese translation" is encountered, update the frequency in the nested dictionary. After traversing all place names, finally obtain the results of which Chinese translations correspond to the same foreign language and how many times each translation appears.

[0046] In principle, the Chinese translations of foreign place names are unified. However, due to the uneven capabilities of previous translators, the phenomenon of the same name with different translations often occurs. The book "Manual for Mutual Search of Chinese Translations of Foreign Language Place Names" published by Tourism Education Press lists many such place names. For example, given {'Thornbill Road': {'索恩比尔路': 2, '桑比尔路': 10}}, it can be seen that the nested dictionary can quickly analyze the contradictory results of place name translations in the original data. Although "Thornbill Road" has been translated as "桑比尔路" more times, the widely used or frequently occurring one is not necessarily in line with the translation rules, or not necessarily the best. Therefore, it is necessary for humans to determine the standard Chinese place name corresponding to the foreign place name and then add it to the benchmark dataset in the place name database.

[0047] It should be noted that through preprocessing, the place name database can be gradually iteratively improved, enhancing the coverage rate and accuracy of the place name database for the dataset to be tested, unifying the standard translations, and avoiding evaluation biases caused by data缺失.

[0048] Step 3: Input the foreign place names in the dataset to be tested into the foreign language place name translation software one by one. If the corresponding standard Chinese translation can be directly queried from the benchmark dataset, the quality score F is the highest score, and proceed to Step 5; otherwise, proceed to Step 4.

[0049] In a possible embodiment, the highest score is 100 points.

[0050] As Figure 2 shown, Step 4: Translate and calculate the quality evaluation score: Step 401: The foreign language place name translation software outputs the Chinese translation of the current foreign language place name and obtains the place name components corresponding to the Chinese translation. The quality of the place name translation results is mainly determined by the composition of the place name and the source of the translation. Therefore, it is necessary to further break down the Chinese translation of the place name and obtain the place name components corresponding to the Chinese translation.

[0051] Step 402: Obtain the weights of the place name components in the Chinese translations based on the weight mapping table; as shown in Table 1, the weight mapping table is a weight mapping table of place name components-place name types-weights.

[0052] Table 1 Weight Mapping Table

[0053] As shown in Table 1, when a place name consists only of a proper name, the weight of the proper name is 1, and the weight of the other components is 0. When a place name consists of a proper name and a generic name, the weight of each component needs to be determined according to the type of land cover. For example, the proper name has a heavier weight for large natural geographical entities and key cities, while the generic name has a heavier weight for points of interest. If a place name consists of a proper name, a generic name, and a directional term, the weight of each part will also be assigned separately according to the type of land cover.

[0054] Step 403: Assign reliability scores to the regional name components of the Chinese translations; The specific method for assigning reliability scores to local component scores is as follows: Step 4031: If the place name component can be directly queried from the benchmark dataset, the confidence level is 100%; otherwise, proceed to step 4032. Step 4032: For the generic part of the place name component, determine the confidence level according to the mapping table of element code class code matching-confidence level; For the generic part of a place name, first look up the first standard element code corresponding to the generic part of the original foreign word.

[0055] For the generic part of the Chinese translation, the corresponding land cover type is obtained based on national or industry standards. For example, GB / T 13923-2022, the national basic geographic information element classification and code, maps the land cover type to the second standard element code. In GB / T 13923-2022, the element code adopts a 4-digit or 6-digit hierarchical structure. The first two digits represent the major category code, the third digit represents the intermediate category code, and the fourth digit represents the minor category code.

[0056] If the first standard element code and the second standard element code are basically matched, that is, the major category code and the intermediate category code are exactly the same, then the confidence level of the generic name part is C=100%; if the first standard element code and the second standard element code are basically similar, that is, only the major category code is the same, then the confidence level of the generic name part is C=75%; if the first standard element code and the second standard element code do not match, that is, the major category codes are different, then the confidence level of the generic name part is set to C=50%.

[0057] like Figure 3 As shown, in step 4033, for the proper noun portion of the place name component, compare the syllable segmentation results of the rule model and the machine learning model. If the syllable segmentation results are consistent, the confidence level A = 100%; if the syllable segmentation results are inconsistent, there are two cases: one is that the segmentation positions in the syllable segmentation results are inconsistent and the number of syllables is inconsistent, and the confidence level A = 100%. Where q represents the difference in the number of letters in the syllable segmentation results output by the rule model and the machine learning model, λ represents the penalty coefficient, λ∈[0.5,1], n1 represents the number of syllable segments output by the rule model, n2 represents the number of syllable segments output by the machine learning model, and L avg This indicates the average number of letters per syllable. Q represents the total number of letters in the proper noun; another type is where the segmentation positions are inconsistent but the number of syllables is consistent, with a confidence level of... .

[0058] The Chinese translation of proper nouns is highly dependent on the accuracy of syllable segmentation. Therefore, a rule-based model and a machine learning model are set up to perform syllable segmentation respectively, and the confidence level is determined based on the comparison results of syllable segmentation.

[0059] The rule-based model and the machine learning model operate independently. The rule-based model strictly adheres to official specifications, ensuring highly controllable segmentation results, transparent segmentation logic, and interpretability. The machine learning model leverages the sequence modeling capabilities of the Transformer to accurately capture the phonetic relationships between characters. It automatically learns complex pronunciation rules and transliteration habits, overcoming the limitations of the rule-based model in adapting to special spellings and rare languages, thus improving segmentation coverage. Consistent segmentation results between the two syllables indicate high reliability, therefore the confidence level A for the proper noun portion is 100%. Inconsistent segmentation results indicate discrepancies, requiring quantification of the degree of disagreement using a formula.

[0060] When the number of syllables and the specific segmentation location are different, the confidence level is determined by the formula. The confidence level A of the proper name part is calculated from the formula. It can be seen from the formula that the more letters are misplaced, the lower the confidence level A of the proper name part is; the greater the difference in the number of syllables, the lower the confidence level A of the proper name part is; the smaller the divergence in the syllable segmentation results, the higher the confidence level A of the proper name part is; the lowest confidence level A of the proper name part is 0.

[0061] The syllable segmentation results are inconsistent, but the number of syllables segmented is consistent. The confidence level is determined using the formula. The confidence level A of the proper name portion is calculated. As can be seen from the formula, the more misaligned the letters, the lower the confidence level A of the proper name portion, and the closer it is to the lower limit of 0.5.

[0062] It should be noted that the specific method for syllable segmentation of the proper noun portion of Chinese translations using a rule-based model is as follows: the proper noun portion of the Chinese translation is standardized according to the detailed rules of the "Guidelines for the Translation of Foreign Place Names into Chinese Characters" for the corresponding language; the official transliteration table is split to construct a transliteration dictionary with key-value pairs of Chinese transliteration units and foreign language original syllables; candidate syllable boundaries are detected based on phonetic features to accurately locate potential segmentation positions that conform to pronunciation rules, avoiding arbitrary segmentation without basis and improving the rationality of segmentation; with candidate syllable boundaries as constraints, the transliteration dictionary is retrieved through a forward maximum matching algorithm to perform syllable segmentation, making the final segmentation result closer to the phonetic rhythm of the source language.

[0063] It should be noted that the specific method for segmenting the proper noun portion of Chinese translations using machine learning models is as follows: Collect corpora, each including the proper noun portion of the Chinese translation and ground truth boundary annotations; convert the proper noun portion of the Chinese translation into a sequence of subwords. Construct a dataset. Divide the dataset into training, validation, and test sets in an 8:1:1 ratio to ensure a balanced distribution across the domains.

[0064] Configure the basic parameters of the first Transformer model; adopt an encoder-decoder structure. The encoder can fully extract the semantic and phonetic features of the Chinese translation, and the decoder can accurately predict the boundary probability between adjacent characters, making it suitable for syllable segmentation tasks and better able to capture complex pronunciation rules than traditional rule-based methods. Configure the basic parameters: d_model=512 / 768, number of layers=6, attention head=8; set dropout=0.1-0.3, feedforward network dimension=2048 to effectively prevent overfitting and improve the generalization ability of the first Transformer model.

[0065] The sequence of sub-words from the proper noun portion of the Chinese translation is used as input to train the first Transformer model. The optimization objective is to minimize the cross-entropy loss function between the predicted boundary probability and the true boundary label.

[0066] Training strategy settings: Configure the Adam optimizer and use Noam learning rate scheduling; set gradient accumulation and mixed precision training. Training execution and monitoring: Batch size: 4096 tokens (approximately 64 sentences); Training steps: 100,000-500,000 steps; Validate every 1000 steps and save the best model; Monitor training loss, validation loss, and BLEU score to ensure the reliability and reproducibility of syllable segmentation results. The resulting trained first Transformer model provides stable machine learning output for confidence calculation of proper noun parts.

[0067] The proper noun portion of the Chinese translation to be processed is input into the first Transformer model. The first Transformer model outputs a predicted boundary probability p for every two adjacent letters in the proper noun portion. If the predicted boundary probability p is greater than the segmentation threshold, the position is determined as the syllable boundary, and the syllable segmentation result of the machine learning model is obtained.

[0068] In another possible embodiment, the specific method for syllable segmentation using a machine learning model is as follows: Collect bilingual parallel corpora. Each corpus includes the foreign proper name, the proper name part of the corresponding Chinese translation, and the ground truth boundary label corresponding to the proper name part of the Chinese translation. Convert the proper name parts of both the foreign proper name and the Chinese translation into sub-word sequences. Construct a dataset. Divide the dataset into training, validation, and test sets in an 8:1:1 ratio. Ensure that each set is balanced in terms of domain distribution.

[0069] The machine learning model includes a first Transformer model and a second Transformer model, and the basic parameters of the first Transformer model and the second Transformer model are configured. The first Transformer model takes as input a sequence of words from the proper noun portion of the Chinese translation, and its optimization objective is to minimize the cross-entropy loss function between the predicted boundary probabilities and the true boundary labels. It outputs a sequence of predicted boundary probabilities for the syllable boundaries of the proper noun portion of the Chinese translation, thus obtaining the syllable segmentation result. The second Transformer model takes as input the syllable segmentation result output by the first Transformer model, and its optimization objective is to maximize the cross-entropy loss function of the character similarity between the reconstructed Chinese translation and the true Chinese translation. It outputs the probability distribution of Chinese characters at each position in the proper noun portion of the Chinese translation, thus obtaining the character sequence of the reconstructed Chinese translation. The first and second Transformer models are trained jointly.

[0070] The second Transformer model adds a semantic verification layer to the syllable segmentation results of the first Transformer model: only the segmentation method that can restore the standard Chinese translation will be strengthened, avoiding the problem that the segmentation position may be correct but the semantics may be inaccurate in simple boundary prediction, thereby learning more essential syllable segmentation rules and improving the overall accuracy.

[0071] The proper noun portion of the Chinese translation to be processed is input into the first Transformer model. The first Transformer model outputs the predicted boundary probability p for every two adjacent letters in the foreign proper noun, representing the confidence that the position is a syllable boundary. If the confidence is greater than the segmentation confidence threshold, the position is determined as a syllable boundary, and the syllable segmentation result of the machine learning model is obtained.

[0072] Therefore, when the benchmark database has sufficient data and the labeled data used for training is high-quality manually labeled data, the first Transformer model is sufficient; however, when the sample size is insufficient, it is necessary to use both the first Transformer model and the second Transformer model.

[0073] Step 404: Calculate the quality evaluation score F based on the weight and confidence level of the place name components; In one possible embodiment, the quality assessment score F = (A × B + C × D + E × G) × 100% + σ; where A represents the confidence level of the proper noun part, B represents the weight of the proper noun, C represents the confidence level of the common noun part, D represents the weight of the common noun, E represents the confidence level of the adjective part, G represents the weight of the adjective, and σ represents the correction parameter. B + D + G = 1.

[0074] Step 405: If the quality evaluation score F is greater than the quality evaluation threshold F set Data that is deemed excellent is classified as such; otherwise, it is classified as data requiring approval.

[0075] In one possible embodiment, the quality evaluation threshold F set For dynamic thresholds, , Represents the basic threshold, where Indicates the language adjustment factor. This represents the country adjustment factor. This represents the data distribution adjustment coefficient.

[0076] In one possible embodiment, If the language is English, =1; Country of origin: United Kingdom =1; when the proportion of high-frequency words in the dataset is less than 5%. The value is 0.90; when the proportion of high-frequency words in the dataset is between 5% and 10%, The value is 0.95; when the proportion of high-frequency words in the dataset is between 10% and 15%, the higher the proportion of high-frequency words, the more regular the dataset, and the lower the translation difficulty. The higher the value, the higher the quality evaluation threshold F. set The larger, The value is 0.98; when the proportion of high-frequency words in the dataset exceeds 15%, The value is 1.

[0077] For example, in a dataset where the language is English, the country is the United Kingdom, and high-frequency words account for more than 15% of the total number of words, the quality assessment threshold F is... set Data with a quality evaluation score (F) above 85% is considered excellent, eliminating the need for further manual cleaning and certification by auditing experts, significantly reducing their workload and improving efficiency. Data with a quality evaluation score (F) below 85% is considered data requiring verification, necessitating expert review; the lower the score, the more rigorous the review process.

[0078] Step 5: Traverse the dataset to be tested and obtain the quality evaluation score for each foreign place name.

[0079] Step 6: Obtain the score for the foreign language place name translation software based on the quality evaluation score of each foreign language place name.

[0080] In one possible embodiment, the score of the foreign language place name translation software is the arithmetic mean of the quality evaluation scores for each foreign language place name.

[0081] The above description is merely an embodiment of this application and does not constitute any limitation on this application. Any simple modifications, alterations, or equivalent structural changes made to the above embodiments based on the technical essence of this application shall still fall within the protection scope of the technical solution of this application.

Claims

1. A quality evaluation method for foreign language place name translation software, characterized in that, Includes the following steps: Step 1: Construct a place name database, which includes a place name result library and a component matching library. The place name result library includes a benchmark dataset and a dataset to be tested. Step 2: Label the foreign place names in the dataset to be tested. Step 3: Input the foreign place names in the dataset to be tested into the foreign place name translation software one by one. If the corresponding standard Chinese translation can be found directly from the benchmark dataset, the quality score F is the highest score, and proceed to step 5; otherwise, proceed to step 4. Step 4: Translate and calculate the quality evaluation score: Step 401: The foreign language place name translation software outputs the Chinese translation of the current foreign language place name and obtains the place name components corresponding to the Chinese translation; Step 402: Obtain the weights of the regional name components in the Chinese translations based on the weight mapping table; Step 403: Assign reliability scores to the regional name components of the Chinese translations; Step 404: Calculate the quality evaluation score F based on the weight and confidence level of the place name components; Step 405: If the quality evaluation score F is greater than the quality evaluation threshold F set Data that is deemed excellent is classified as such; otherwise, it is classified as data requiring approval. Step 5: Traverse the dataset to be tested and obtain the quality evaluation score for each foreign place name; Step 6: Obtain the score for the foreign language place name translation software based on the quality evaluation score of each foreign language place name.

2. The quality evaluation method for a foreign language place name translation software according to claim 1, characterized in that: The specific method for assigning reliability scores to local component scores is as follows: Step 4031: If the place name component can be directly queried from the benchmark dataset, the confidence level is 100%; otherwise, proceed to step 4032. Step 4032: For the generic term portion of the place name component, determine the confidence level based on the element coding class code matching-confidence level mapping table; generic terms often have standard translations, while proper names rely more on transliteration rules. Step 4033: For the proper noun portion of the place name component, compare the syllable segmentation results of the rule model and the machine learning model. If the syllable segmentation results are consistent, the confidence level A = 100%; if the segmentation positions and the number of syllables are inconsistent, the confidence level... Where q represents the difference in the number of letters in the syllable segmentation results output by the rule model and the machine learning model, λ represents the penalty coefficient, λ∈[0.5,1], n1 represents the number of syllable segments output by the rule model, n2 represents the number of syllable segments output by the machine learning model, and L avg This indicates the average number of letters per syllable. Q represents the total number of letters in the proper noun; syllable segmentation results show inconsistent segmentation positions but consistent number of syllable segments, with a confidence level of [missing information]. .

3. The quality evaluation method for a foreign language place name translation software according to claim 1, characterized in that: The specific method for syllable segmentation of the proper noun portion of Chinese translations using rule-based models is as follows: Standardize the proper noun portion of the Chinese translation; The official transliteration table was split to construct a transliteration dictionary with key-value pairs of Chinese transliteration units and foreign language original syllables; Candidate syllable boundaries are detected based on phonetic features. Using candidate syllable boundaries as constraints, a transliteration dictionary is retrieved using a forward maximum matching algorithm to perform syllable segmentation.

4. The quality evaluation method for a foreign language place name translation software according to claim 1, characterized in that: The specific method for segmenting the proper noun portion of Chinese translations using machine learning models is as follows: Collect corpora, each corpus including the proper noun portion of the Chinese translation and ground truth boundary annotations; convert the proper noun portion of the Chinese translation into a sequence of subwords; construct the dataset; Configure the basic parameters of the first Transformer model; Given the input dataset, train the first Transformer model with the optimization objective being to minimize the cross-entropy loss function between the predicted boundary probabilities and the true boundary labels. The proper noun portion of the Chinese translation to be processed is input into the first Transformer model. The first Transformer model outputs a predicted boundary probability p for every two adjacent letters in the proper noun portion. If the predicted boundary probability p is greater than the segmentation threshold, the position is determined as the syllable boundary, and the syllable segmentation result of the machine learning model is obtained.

5. The quality evaluation method for a foreign language place name translation software according to claim 1, characterized in that: The specific method for syllable segmentation using machine learning models is as follows: Collect bilingual parallel corpora. Each corpus includes a foreign proper name, the proper name portion of the corresponding Chinese translation, and the ground truth boundary label corresponding to the proper name portion of the Chinese translation. Convert both the foreign proper name and the proper name portion of the Chinese translation into subword sequences. Construct a dataset. The machine learning model includes a first Transformer model and a second Transformer model, and the basic parameters of the first Transformer model and the second Transformer model are configured. The first Transformer model takes as input a sequence of sub-words from the proper noun portion of the Chinese translation, and its optimization objective is to minimize the cross-entropy loss function between the predicted boundary probabilities and the true boundary labels. It outputs a sequence of predicted boundary probabilities for the syllable boundaries of the proper noun portion of the Chinese translation, thus obtaining the syllable segmentation result. The second Transformer model takes as input the syllable segmentation result output by the first Transformer model, and its optimization objective is to maximize the cross-entropy loss function of the character similarity between the reconstructed Chinese translation and the true Chinese translation. It outputs the probability distribution of Chinese characters at each position in the proper noun portion of the Chinese translation, thus obtaining the character sequence of the reconstructed Chinese translation. The first and second Transformer models are trained jointly. The proper noun portion of the Chinese translation to be processed is input into the first Transformer model. The first Transformer model outputs the predicted boundary probability p for every two adjacent letters in the foreign proper noun, representing the confidence that the position is a syllable boundary. If the confidence is greater than the segmentation confidence threshold, the position is determined as a syllable boundary, and the syllable segmentation result of the machine learning model is obtained.

6. The quality evaluation method for a foreign language place name translation software according to claim 1, characterized in that: Step 404: The specific method for calculating the quality evaluation score F based on the weight and confidence level of the place name components is as follows: Quality evaluation score F = (A×B+C×D+E×G)×100%+σ; where A represents the confidence level of the proper noun part, B represents the weight of the proper noun, C represents the confidence level of the generic noun part, D represents the weight of the generic noun, E represents the confidence level of the adjective part, G represents the weight of the adjective, and σ represents the correction parameter.

7. The quality evaluation method for a foreign language place name translation software according to claim 1, characterized in that: Quality evaluation threshold F set For dynamic thresholds, ,in Indicates the basic threshold. Indicates the language adjustment factor. This represents the country adjustment factor. This represents the data distribution adjustment coefficient.

8. The quality evaluation method for a foreign language place name translation software according to claim 1, characterized in that: In step two, after labeling the foreign place names in the dataset to be tested, the foreign place names are divided into W major categories, including at least a set of proper names consisting only of proper names. Then, the dataset to be tested is preprocessed. The specific method for preprocessing the dataset to be tested is as follows: Step 201: Segment the foreign place names in the dataset to be tested according to words, filter out high-frequency words, determine the composition of high-frequency words based on the land cover type, and add high-frequency words to the place name database; Step 202: For the set of proper names, create a nested dictionary: {Foreign place names: {Chinese place names 1: f1; ...; Chinese place names n: f n The system iterates through the proper noun set to obtain multiple Chinese place names that contradict the same foreign place name and their frequency of occurrence. It then determines the standard Chinese place name corresponding to the foreign place name and adds it to the place name database.

9. The quality evaluation method for a foreign language place name translation software according to claim 1, characterized in that: In step 402, the weight mapping table is a mapping table of place name components-place name types-weights.

10. A quality evaluation method for foreign language place name translation software according to claim 1, characterized in that: The place names in the dataset to be tested are labeled with their components and divided into five categories: those consisting only of proper names; those consisting of proper names and generic names; those consisting of adjectives or locative words and proper names; those consisting of adjectives or locative words and proper names and generic names; and those consisting of adjectives or locative words and generic names.

Images