Model training method, translation method, electronic device and computer program product

Abstract

The invention provides a model training method, a translation method, electronic equipment and a computer program product, and relates to the technical field of interface design. The model training method comprises the following steps: constructing an initial model for language translation; extracting visual features of various fonts in the font database; based on historical region structure parameters of a historical region in the historical design interface, determining semantic constraint conditions for constraining the length of the historical translation text; and performing joint training on the initial model according to the visual features and the semantic constraint conditions to obtain a constraint translation model. The translation method comprises the following steps: determining initial text information and regional structure parameters according to a text region in a design interface; processing according to the initial text information and the regional structure parameters through a constraint translation model to obtain a translated text; and filling each translated text into the corresponding text region.

Description

Technical Field

[0001] This application relates to the field of interface design technology, and more specifically, to a model training method, a translation method, an electronic device, and a computer program product. Background Technology

[0002] Currently, when designing UI (User Interface) interfaces, due to differences in different languages ​​and requirements, for example, when a designer completes a Chinese design draft, it usually needs to be translated into multiple languages ​​such as English due to different client requirements. Currently, this is usually done through machine translation or manual adaptation.

[0003] However, when using machine translation tools such as Google Translate, the differences in expression between different types of languages ​​can cause the translated text to exceed the original UI container boundaries. Human translation is time-consuming and labor-intensive, and requires further adjustments to font size, resulting in low efficiency and poor quality when converting the UI interface to a different language, which cannot meet current design and usage requirements. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a model training method, a translation method, an electronic device, and a computer program product to improve the problems of low efficiency and poor performance of UI interfaces in language conversion in the prior art.

[0005] To address the aforementioned problems, firstly, embodiments of this application provide a model training method, the method comprising: Construct an initial model for language translation; Extract visual features of various fonts from a font database; Based on the historical region structure parameters in the historical design interface, semantic constraints are determined to constrain the length of the historical translated text. Based on the visual features and the semantic constraints, the initial model is jointly trained to obtain a constrained translation model.

[0006] In the above implementation process, considering that the font database contains various types of fonts, and that different fonts have different visual characteristics due to different design requirements, these visual characteristics can affect the size, length, and other visual aspects of text information. Furthermore, the semantic content during translation also influences the size, length, and other visual aspects of text information. Therefore, visual features of various fonts in the font database can be extracted, and semantic constraints on the length of historical translated text can be determined based on the historical region structure parameters in the historical design interface. By combining visual and translation content perspectives, the initial language translation model can be jointly trained based on visual features and semantic constraints to obtain a corresponding constrained translation model. This constrained translation model can limit text length and visual aspects during the translation process, ensuring accuracy of the translated content while improving the consistency of the visual appearance of the text before and after translation, and optimizing the text length.

[0007] Optionally, the extraction of visual features of multiple fonts from the font database includes: Determine the visual format parameters of various fonts in the font database at different font sizes; wherein, the visual format parameters include: character width, character height, character spacing, and line height; The visual features of each font are obtained by normalizing and vector aggregation based on various visual format parameters.

[0008] In the above implementation process, we can first determine various types of visual format parameters for multiple fonts in the font database under different font sizes. Then, we can perform normalization and vector aggregation on these various visual format parameters to obtain the visual features corresponding to each font. This allows us to measure and process the various visual format parameters representing visual conditions into visual features of a unified format, which can then be indexed and used in the subsequent processing of the constrained translation model. This enables us to determine the font, font size, and other parameters affecting the visual appearance of the translated text through these visual features.

[0009] Optionally, the step of jointly training the initial model based on the visual features and the semantic constraints to obtain a constrained translation model includes: Based on the aforementioned semantic constraints and the accuracy threshold of semantic translation, the first loss during semantic translation is determined; A second loss is determined based on the visual features and the historical region structure parameters of the historical region to determine the degree of matching between the historical translated text and the historical region. Determine the combined loss based on the first loss and the second loss; The initial model is jointly trained based on the joint loss to obtain the constrained translation model.

[0010] In the above implementation process, considering the accuracy requirements of translation, a first loss for semantic translation can be determined based on the defined semantic constraints and the accuracy threshold during semantic translation. A second loss is determined by combining visual features and historical region structure parameters as regional constraints to determine the degree of matching between the text structure of the historical translated text and the historical region. Combining the first and second losses, a joint loss that takes into account the requirements of text translation accuracy, length, and visual structure is determined. The initial model is then jointly trained based on the joint loss to obtain the corresponding constrained translation model. This ensures that the constrained translation model can generate translated text that meets the actual regional structure requirements of the text region, has high text content accuracy, and consistent visual structure before and after, thus satisfying the translation needs of various different scenarios.

[0011] Secondly, embodiments of this application also provide a translation method, the method comprising: Based on the text area in the design interface, determine the initial text information and area structure parameters; The translated text is obtained by processing the initial text information and the region structure parameters using a constrained translation model; wherein the constrained translation model is determined based on the method described in any one of the first aspects above. Each translated text is filled into the corresponding text area.

[0012] In the above implementation process, when designing a design interface for language conversion, the initial text information and region structure parameters can be determined based on the text area to be translated within the design interface. By constraining the translation model and processing the initial text information and region structure parameters, a translated text is obtained that meets the actual region structure requirements of the text area, has high text content accuracy, and maintains consistent visual structure. This translated text is then filled into the corresponding text area, completing the translation process. The system can perform translation processing based on the actual text content and region conditions of the text area to be translated, and by using a constrained translation model, the resulting translated text meets the region constraints of the text area. No manual translation or adjustment is required, effectively improving the efficiency and effectiveness of language conversion for the design interface and meeting the design and usage requirements of various application scenarios.

[0013] Optionally, determining the initial text information and region structure parameters based on the text region in the design interface includes: Obtain the design interface to be translated from the design software; Identify one or more of the text regions in the design interface; Identify the region structure parameters for each of the text regions; wherein the region structure parameters include the shape, length, and width of the text region; Read the initial text information in the text area; wherein the initial text information includes: text content and text format; the text format includes at least one of: font size, font name, line height, and line number.

[0014] In the above implementation process, the design interface to be translated can be obtained from various design software. One or more text regions are identified from the design interface using image recognition. Based on the identified text regions, the shape, length, width, and other region structure parameters of each text region are further identified, and the initial text information in each text region is read. The initial text information includes the text content and text formatting that affects the visual appearance. This allows for the simultaneous processing of multiple text regions in the design interface to be translated, effectively improving the efficiency and effectiveness of determining the initial text information and region structure parameters.

[0015] Optionally, the step of processing the initial text information and the region structure parameters through a constrained translation model to obtain the translated text includes: The region structure parameters and the initial text information are substituted into the constrained translation model for processing to obtain the initial translated text output by the constrained translation model; Based on the region structure parameters, determine whether the initial translated text matches the corresponding text region; If it is determined that the initial translated text matches the text region, then the initial translated text shall be used as the translated text; If it is determined that the initial translated text does not match the text region, then the constrained translation model is used to reprocess the text according to the region structure parameters and the initial text information to obtain the adjusted translated text; wherein, the translated text is the adjusted translated text that matches the text region. Wherein, the similarity between the first text format of the initial text information and the second text format of the translated text is higher than a preset similarity threshold.

[0016] In the above implementation process, the region structure parameters and initial text information can be used as input data and fed into the constrained translation model for translation processing to obtain the initial translated text. To improve the effectiveness of the translated text, the matching between the initial translated text and the text region can be determined based on the region structure parameters. If they match, the initial translated text is directly used as the translated text. If they do not match, it indicates that the initial translated text may still have out-of-bounds issues, and the constrained translation model can be used to re-translate it based on the region structure parameters and initial text information to obtain an adjusted translated text. The matching between the adjusted translated text and the text region is then determined, and the adjusted translated text that matches the text region is the final translated text. Furthermore, the similarity between the first text format of the initial text information and the second text format of the translated text is higher than a preset similarity threshold to ensure that the visual structure of the initial text information and the translated text remains basically consistent before and after translation. This allows for effective verification of the output results of the constrained translation model, resulting in translated text that meets the actual region structure requirements of the text region, has high text content accuracy, and maintains consistent visual structure before and after translation, further optimizing the translation effect.

[0017] Optionally, determining whether the initial translated text matches the corresponding text region based on the region structure parameters includes: Determine the visual parameters of the initial translated text; wherein the visual parameters include the text length and text width of the initial translated text; If the visual parameters are determined to be no more than the region structure parameters, then the initial translated text is determined to match the text region. If the visual parameters exceed the region structure parameters, then the initial translated text does not match the text region.

[0018] In the above implementation process, when determining whether the initial translated text matches the text region, visual parameters such as the actual text length and width of the initial translated text can be determined first. These visual parameters are then compared with the region structure parameters. If the visual parameters do not exceed the region structure parameters, it indicates that the initial translated text will not exceed the region boundary within the text region, and the initial translated text can be placed normally within the text region, meaning the initial translated text matches the text region. If the visual parameters exceed the region structure parameters, it indicates that the initial translated text will exceed the region boundary within the text region, and the initial translated text cannot be placed normally within the text region, meaning the initial translated text does not match the text region. Being able to determine whether the initial translated text matches the text region based on the actual structure of the initial translated text and the text region effectively improves the efficiency and effectiveness of verifying the initial translated text.

[0019] Optionally, the method further includes: Determine whether the historical design interface contains multiple duplicate text regions with the same initial text information and region structure parameters; If multiple repeated text regions exist, the first translated text corresponding to the multiple repeated text regions is filled into the multiple repeated text regions.

[0020] In the above implementation process, considering the correlation between multiple design interfaces, the current design interface and historical design interfaces may contain multiple duplicate text regions with the same initial text information and region structure parameters. Therefore, it is possible to first determine whether duplicate text regions exist in the processed and currently being processed historical design interfaces. If multiple duplicate text regions exist, the first determined translation text from each of the multiple duplicate text regions is directly used to fill the multiple duplicate text regions. This eliminates the need for repeated translation processing of multiple duplicate text regions, effectively improving the efficiency of language conversion for design interfaces and reducing the computational cost required for translation.

[0021] Secondly, embodiments of this application also provide an electronic device, which includes a memory and a processor. The memory stores program instructions, and when the processor reads and runs the program instructions, it executes the steps in any of the above-described implementations of the model training method or translation method.

[0022] Thirdly, embodiments of this application also provide a computer program product, which includes a computer program / instruction. When the computer program / instruction is executed by a processor, it implements the steps in any of the above-described model training method or translation method.

[0023] In summary, the embodiments of this application provide a model training method, a translation method, an electronic device, and a computer program product that can perform translation processing based on the actual text content and regional conditions of the text region to be translated. Furthermore, by constraining the translation model, the resulting translated text satisfies the regional limitations of the text region, eliminating the need for manual translation and adjustment. This effectively improves the efficiency and effectiveness of language conversion for design interfaces and meets the design and usage requirements of various application scenarios. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 A block diagram illustrating an electronic device provided in an embodiment of this application; Figure 2 A schematic flowchart illustrating a model training method provided in an embodiment of this application; Figure 3 A detailed flowchart of step S220 provided for an embodiment of this application; Figure 4 A detailed flowchart of step S240 provided for an embodiment of this application; Figure 5 A flowchart illustrating a translation method provided in an embodiment of this application; Figure 6 A detailed flowchart of step S310 provided for an embodiment of this application; Figure 7 A detailed flowchart of step S320 provided for an embodiment of this application; Figure 8 A detailed flowchart of step S322 provided for an embodiment of this application; Figure 9 A flowchart illustrating another translation method provided in this application embodiment; Figure 10 This is a schematic diagram of the initial state of the design interface of an in-vehicle system. Figure 11 This is a schematic diagram illustrating the state of machine translation of the design interface in existing technologies. Figure 12 This is a schematic diagram showing the translated state of the design interface provided in an embodiment of this application.

[0026] Icons: 100 - Electronic device; 111 - Memory; 112 - Memory controller; 113 - Processor; 114 - Peripheral interface; 115 - Input / output unit; 116 - Display unit. Detailed Implementation

[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of the embodiments of this application.

[0028] In existing technologies, when using machine translation tools such as Google Translate, the differences in expression between different types of languages ​​can cause the translated text to exceed the boundaries of the original UI container. For example, when translating "automatic unlock when approaching the door" into English, the corresponding machine translation is "Automatic locking close to the door". When the English content is filled into the interactive box of Chinese content, the English content is longer than the Chinese content, causing the English content to exceed the boundaries of the interactive box. Human translation requires a high time and effort cost and further adjustments to font size, resulting in low efficiency and poor effect when converting the language of the UI interface, which cannot meet the current design and usage requirements.

[0029] To address the aforementioned issues, this application provides a model training method and a translation method applicable to electronic devices. These electronic devices can be servers, personal computers (PCs), tablets, smartphones, personal digital assistants (PDAs), in-vehicle systems, and other electronic devices with logical computing capabilities. The method can perform translation processing based on the actual text content and regional conditions of the text region to be translated. Furthermore, by constraining the translation model, the resulting translated text satisfies the regional limitations of the text region, eliminating the need for manual translation and adjustment. This effectively improves the efficiency and effectiveness of language conversion for design interfaces, meeting the design and usage requirements of various application scenarios.

[0030] Optionally, please refer to Figure 1 , Figure 1 This is a block diagram illustrating an electronic device according to an embodiment of this application. The electronic device 100 may include a memory 111, a memory controller 112, a processor 113, a peripheral interface 114, an input / output unit 115, and a display unit 116. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device 100. For example, the electronic device 100 may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0031] The aforementioned memory 111, memory controller 112, processor 113, peripheral interface 114, input / output unit 115, and display unit 116 are electrically connected directly or indirectly to each other to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The aforementioned processor 113 is used to execute executable modules stored in the memory.

[0032] The memory 111 can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory 111 stores programs. After receiving execution instructions, the processor 113 executes the programs. The methods executed by the electronic device 100 as defined in any embodiment of this application can be applied to the processor 113, or implemented by the processor 113.

[0033] The aforementioned processor 113 may be an integrated circuit chip with signal processing capabilities. The processor 113 may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a digital signal processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor.

[0034] The peripheral interface 114 described above couples various input / output devices to the processor 113 and the memory 111. In some embodiments, the peripheral interface 114, the processor 113, and the memory controller 112 can be implemented on a single chip. In other instances, they can be implemented on separate chips.

[0035] The input / output unit 115 described above is used to provide user input data. The input / output unit 115 can be, but is not limited to, a mouse and a keyboard.

[0036] The aforementioned display unit 116 provides an interactive interface (e.g., a user interface) between the electronic device 100 and the user, or displays image data for the user's reference. In this embodiment, the display unit can be a liquid crystal display (LCD) or a touch display. If it is a touch display, it can be a capacitive touchscreen or a resistive touchscreen that supports single-point and multi-point touch operations. Supporting single-point and multi-point touch operations means that the touch display can sense touch operations generated simultaneously from one or more locations on the touch display and pass the sensed touch operations to the processor for calculation and processing. In this embodiment, the display unit 116 can display various data such as design interfaces, initial text information, and translated text.

[0037] The electronic device in this embodiment can be used to execute various steps in the model training and translation methods provided in the embodiments of this application. The implementation process of the model training and translation methods is described in detail below through several embodiments.

[0038] Please see Figure 2 , Figure 2 This is a flowchart illustrating a model training method provided in an embodiment of this application. The method may include steps S210-S240.

[0039] Step S210: Construct an initial model for language translation.

[0040] In this process, an initial model capable of translating different languages ​​can be built first. This includes various types of translation models such as phrase translation models (e.g., Phrase-Based MT, PBMT), syntax translation models (e.g., Syntax-Based MT), and deep learning-based machine translation models (e.g., Seq2Seq, Transformer, LLM). The appropriate initial model can be selected based on the actual needs of the application scenario.

[0041] It should be noted that the initial model can be a multimodal model, which combines multiple different types of input data to produce data output.

[0042] Step S220: Extract visual features of various fonts from the font database.

[0043] Step S230: Based on the historical region structure parameters in the historical design interface, determine the semantic constraints that constrain the length of the historical translated text.

[0044] Considering that the font database contains various font types, and that different fonts have different visual characteristics due to varying design requirements, these visual characteristics can affect the size, length, and other visual aspects of text information. Furthermore, the semantic content during translation also influences the size, length, and other visual aspects of text information. Therefore, we can extract the visual characteristics of various fonts from the font database and, based on the historical region structure parameters in the historical design interface, determine the semantic constraints that limit the length of historically translated text.

[0045] Optionally, the visual features of the font may include various feature data that affect the visual appearance of the font, such as font size, height, and spacing.

[0046] Optionally, the historical design interface can be data that meets design requirements and was manually designed or adjusted in a historical context. The historical design interface can include interfaces in different languages. The historical area is an area for placing text in different types of languages, such as interactive box areas. The historical area structure parameters can include various information such as the shape, length, and width of the historical area. For example, the historical area structure parameters are a rectangular area with a length of 10cm and a width of 5cm. The historical area can include the original text information of the original language version and the historical translated text obtained after translation.

[0047] It should be noted that semantic constraints can include restrictions on the abbreviation or shortening of text content. For example, abbreviating "Left" to "L" in English, or abbreviating "Automatic locking close to the door" to "NearDoor Unlock". Semantic constraints can be abbreviated or shortened from multiple perspectives such as entities, grammar, and structure to limit the length of historical translated texts.

[0048] Step S240: Based on visual features and semantic constraints, the initial model is jointly trained to obtain a constrained translation model.

[0049] In this approach, the initial language translation model can be jointly trained by combining visual and translation content perspectives, based on visual features and semantic constraints, to obtain a corresponding constrained translation model.

[0050] exist Figure 2 In the illustrated embodiment, by constraining the translation model, limitations can be imposed on text length, text visuals, etc., during the translation process. This ensures the accuracy of the translated content while improving the consistency of the text visuals before and after translation and optimizing the text length.

[0051] Optionally, please refer to Figure 3 , Figure 3The following is a detailed flowchart of step S220 provided in an embodiment of this application. Step S220 may include steps S221-S222.

[0052] Step S221: Determine the visual format parameters of various fonts in the font database at different font sizes.

[0053] First, we can determine the visual format parameters of various fonts in the font database for different font sizes. These visual format parameters can include various parameters that affect the visual appearance of the font, such as character width, character height, character spacing, and line height.

[0054] It should be noted that the font database can include a variety of regular fonts with permissions, such as SimSun, Microsoft YaHei, Arial, Helvetica, etc. When building the font database, each font can be metrically tested to determine the visual format parameters of each font at different font sizes (e.g., 10-24px). Corresponding data tables such as FontMetrics can also be built to classify and store various visual format parameters for subsequent processing.

[0055] Step S222: Normalize and aggregate vectors based on various visual format parameters to obtain the visual features of each font.

[0056] In order to unify the processing of various visual format parameters, considering the differences between different types of visual format parameters, normalization and vector aggregation can be performed on various visual format parameters to obtain the visual features corresponding to each font.

[0057] Optionally, each visual format parameter can be normalized. The normalization process can be achieved by mapping the pixel values ​​measured in the visual format parameters to a range of 2-1, calculating statistical features such as average length and standard deviation according to font and character category (Chinese, English, numbers, punctuation), and aggregating them according to the following dimensions: font category (e.g., SimSun, Heiti); character type (e.g., Chinese, English, numbers, punctuation). Each combination generates a set of "visual feature vectors", including the mean character width, standard deviation, and unit line height ratio, which are used as visual features for subsequent model use.

[0058] Optionally, if there is an updated font, the font in the font database can be modified, and updated visual features can be extracted based on the updated font.

[0059] exist Figure 3In the illustrated embodiment, various visual format parameters characterizing visual conditions can be measured and processed into visual features of a unified format, so as to facilitate indexing and use in the subsequent processing of the constrained translation model, thereby determining parameters affecting visual conditions such as font and font size of the translated text through visual features.

[0060] Optionally, please refer to Figure 4 , Figure 4 The following is a detailed flowchart of step S240 provided in an embodiment of this application. Step S240 may include steps S241-S244.

[0061] Step S241: Based on semantic constraints and the accuracy threshold of semantic translation, determine the first loss during semantic translation.

[0062] Considering the accuracy requirements of translation, the first loss during semantic translation can be determined based on the defined semantic constraints and the accuracy threshold during semantic translation.

[0063] Optionally, the accuracy threshold can be set to a value such as 95% to keep the translation highly accurate, and the first loss can be denoted as L_translation.

[0064] Step S242: Based on visual features and historical region structure parameters, determine the second loss to determine the degree of matching between historical translated text and historical regions.

[0065] Among them, a second loss can be determined by combining visual features and historical region structure parameters as regional constraints to determine the degree of matching between historical translated texts and historical regions.

[0066] Optionally, the degree of matching between the text structure of the historical translation text and the historical region can include: whether the historical translation text can be properly placed into the historical region. If it can be properly placed into the historical region, then it matches; if it cannot be properly placed into the historical region, for example, if the text goes out of bounds, then it does not match. The second loss can be denoted as L_visual_fit.

[0067] Step S243: Determine the combined loss based on the first loss and the second loss.

[0068] Among these, the first loss and the second loss can be combined to determine the joint loss that takes into account the requirements for text translation accuracy, length, and visual structure.

[0069] Optionally, the joint loss can be Loss = L_translation + λ × L_visual_fit, where λ is a coefficient for adaptively adjusting the magnitude of the second loss.

[0070] Step S244: Jointly train the initial model based on the joint loss to obtain the constrained translation model.

[0071] Among them, the initial model can be jointly trained according to the joint loss to obtain the corresponding constrained translation model, so that the constrained translation model can generate translated text that meets the actual regional structure requirements of the text region, has high accuracy of text content, and has consistent visual structure before and after, thus meeting the translation needs of various different scenarios.

[0072] Optionally, based on a large-scale multilingual translation corpus, the initial model constructed is a multimodal input model. The training method for joint training can include: the input vector can include: the embedded representation of the original text token in the historical initial text information corresponding to the historical translated text, translation contextual cue words (such as UI element semantics), and the embedding vector of the visual features of the fonts in the historical initial text information (FontMetrics Embedding). The model structure of the initial model can adopt a multi-layer Transformer or mBART architecture. Training objectives can include: translation semantic accuracy (determined based on the first loss) and the rationality of visual layout (determined based on the second loss).

[0073] exist Figure 4 In the illustrated embodiment, joint training can enable the trained constrained translation model to automatically balance semantic accuracy and structural layout adaptability when generating translated text.

[0074] Please see Figure 5 , Figure 5 This is a flowchart illustrating a translation method provided in an embodiment of this application. The method may include steps S310-S330.

[0075] Step S310: Determine the initial text information and region structure parameters based on the text area in the design interface.

[0076] When designing a design interface for translation into different languages, the initial text information and region structure parameters can be determined based on the text area to be translated in the design interface.

[0077] Optionally, the initial text information and region structure parameters of the text area in the design interface can be determined through methods such as image recognition, plug-in access for data extraction, etc.

[0078] Step S320: By constraining the translation model, the initial text information and regional structure parameters are processed to obtain the translated text.

[0079] One approach is to constrain the translation model, combining initial text information and region structure parameters, and based on the actual translation language requirements, such as translating Chinese into English, to obtain translated text that meets the actual region structure requirements of the text region, has high text content accuracy, and consistent visual structure before and after translation.

[0080] It should be noted that the constrained translation model is determined based on any one of the above training methods.

[0081] Step S330: Fill each translated text into the corresponding text area.

[0082] When performing language conversion on the design interface, the initial content in the text area can be removed using design software, and the translated text can be filled into the corresponding text area to complete the translation process.

[0083] Optionally, the corresponding translated text can be obtained through a data extraction plugin, and the translated text can be dynamically filled into the text area.

[0084] Optionally, exceptions at each stage can be captured uniformly, returning explicit error codes and information; a retry mechanism can be added for critical model calls. Performance monitoring can also be performed: tracking the latency of each interface and the duration of model calls to optimize bottlenecks. To improve the scalability of the translation constraint model, more fonts or custom font libraries can be integrated in the future, supporting dynamic uploading of font files and automatic measurement of visual structural parameters.

[0085] exist Figure 5 In the embodiments shown, translation processing can be performed based on the actual text content and regional conditions of the text region to be translated. Furthermore, the translation processing is performed through a constrained translation model, and the resulting translated text meets the regional restrictions of the text region. No manual translation or adjustment is required, which effectively improves the efficiency and effect of language conversion for the design interface and meets the design and usage requirements of various application scenarios.

[0086] Optionally, please refer to Figure 6 , Figure 6 The following is a detailed flowchart of step S310 provided in an embodiment of this application. Step S310 may include steps S311-S314.

[0087] Step S311: Obtain the design interface to be translated from the design software.

[0088] Designers can use various types of design software to create corresponding design interfaces. These interfaces can be UI interfaces for various applications, such as interactive interfaces for controlling door opening and closing functions in vehicle systems.

[0089] Optionally, the design interface to be translated can be obtained from the data interface of the design software based on the data extraction plugin.

[0090] Alternatively, the design software may include various tools or software capable of interface design, such as Figma, MasterGo, and Instant Design.

[0091] Step S312: Identify one or more text regions in the design interface.

[0092] One or more text regions can be identified from the design interface using image recognition.

[0093] For example, image recognition can be used to identify pixels throughout the design interface to determine one or more text regions based on the color differences of the pixels.

[0094] Step S313: Identify the region structure parameters of each text region.

[0095] Based on the defined text regions, the region structure parameters of each text region can be further identified. These parameters include the shape, length, and width of the text region.

[0096] For example, the region structure parameters can include: containerWidth: number; / / width of the text region (px), containerHeight: number; / / height of the text region (px).

[0097] Step S314: Read the initial text information in the text area.

[0098] Based on the defined text area, the initial text information in each text area can be read. The initial text information may include: text content and text format. The text format may include at least one of the following: font size, font name, line height, and line number.

[0099] Optionally, the initial text information may include fontSize:number; / / font size (px); fontFamily:string; / / font name; lineHeight:number; / / line height (multiple); maxLines:number; / / maximum number of lines; language:string; / / target language code, i.e., text content.

[0100] It should be noted that when there are multiple text areas in the design interface, you can select the text areas to process them one by one, or you can process all the text areas at the same time.

[0101] Alternatively, the initial text information and region structure parameters of multiple text areas in the design interface can be obtained directly from the data interface of the design software through a data extraction plugin.

[0102] exist Figure 6 In the illustrated embodiment, multiple text regions in the design interface to be translated can be processed simultaneously, effectively improving the efficiency and effectiveness of determining initial text information and region structure parameters.

[0103] Optionally, please refer to Figure 7 , Figure 7 The following is a detailed flowchart of step S320 provided in an embodiment of this application. Step S320 may include steps S321-S324.

[0104] Step S321: Substitute the region structure parameters and initial text information into the constrained translation model for processing to obtain the initial translated text output by the constrained translation model.

[0105] In this process, the region structure parameters and initial text information can be used as input data and fed into the constrained translation model for translation processing to obtain the initial translated text output.

[0106] Optionally, the constrained translation model can perform visual feature matching based on the initial input text information to determine matches with the same font and size as the initial text information. If no identical matches are found, approximate vectors can be obtained through interpolation or approximate font mapping. For example, the font or size closest to the initial text information can be determined as the match. The constrained translation model can perform translation processing based on the text content (including corresponding UI prompts) in the initial text information, combined with the constraints of region structure parameters. The translation process can include: the translation encoder in the constrained translation model fuses the visual constraints of region structure parameters as an additional channel; the translation decoder ranks candidate translation sentences based on the degree of visual matching during the beam search stage; and the constrained translation model can calculate the length and width of each candidate translation sentence and adjust the character layout and content in conjunction with the region structure parameters. The constrained translation model controls the number and arrangement logic of output characters based on font display characteristics. For example, when the font of the initial input text is Helvetica, the font size is 14px, and the width of the region structure parameter is 120px, the constrained translation model can automatically evaluate the optimal number and length of translation tokens in this scenario and generate the language expression most suitable for this text region as the initial translation text.

[0107] For example, the structure of the initial translated text output may include: - { "translated_text": "Text content adapted to the current initial translation text"; "estimated_width": 117.8 (The initial text width of the translated text) "predicted_lines": 2 (Number of lines in the initial translated text) }

[0108] Step S322: Based on the region structure parameters, determine whether the initial translated text matches the corresponding text region.

[0109] In order to improve the effectiveness of the translated text, the matching between the initial translated text and the text region can be determined based on the region structure parameters of the text region.

[0110] Optionally, it can be determined whether the initial translated text can be placed normally into the text area. If it can be placed normally, then it is matched; if it cannot be placed normally and there is an out-of-bounds situation, then it is not matched.

[0111] Step S323: If it is determined that the initial translation text matches the text region, then the initial translation text shall be used as the translation text.

[0112] Step S324: If it is determined that the initial translation text does not match the text region, the translation model is constrained to reprocess the text according to the region structure parameters and the initial text information to obtain the adjusted translation text.

[0113] If a match is found, the initial translated text is used directly as the translated text. If no match is found, it indicates that the initial translated text may still have out-of-bounds issues. In this case, the translation model can be re-processed based on the region structure parameters and the initial text information to obtain the adjusted translated text. The model then continues to determine whether the adjusted translated text matches the text region. The adjusted translated text that matches the text region is the final translated text.

[0114] It should be noted that the similarity between the first text format of the initial text information and the second text format of the translated text is higher than a preset similarity threshold. The first and second text formats can include font, font size and other format information of the text. If there is no font or font size in the font database that is the same as the font of the initial text information, then the font or font size with the highest similarity to the font of the initial text information is queried from the font database as the second text format, so that the visual structure of the initial text information and the translated text before and after translation is basically consistent.

[0115] exist Figure 7In the illustrated embodiment, the output of the constrained translation model can be effectively verified, resulting in translated text that meets the actual regional structure requirements of the text region, has high accuracy in text content, and maintains consistent visual structure before and after, thus further optimizing the translation effect.

[0116] Optionally, please refer to Figure 8 , Figure 8 The following is a detailed flowchart of step S322 provided in an embodiment of this application. Step S322 may include steps S3221-S3223.

[0117] Step S3221: Determine the visual parameters of the initial translated text.

[0118] In determining whether the initial translated text matches the text region, visual parameters such as the overall text length and text width of the initial translated text can be determined first, and then the visual parameters and region structure parameters can be compared.

[0119] Optionally, the text length can be determined based on the font size, number, spacing, etc. of the characters in the initial translation text, and the text width can be determined based on the font size, line height, and number of lines. For example, if a line of characters goes out of bounds and the text area is wide enough to accommodate two lines of characters, the constrained translation model can adjust the line of characters into two lines of characters to match the text area.

[0120] Step S3222: If the visual parameters are determined to be no more than the region structure parameters, then the initial translated text is determined to match the text region.

[0121] If the visual parameters do not exceed the region structure parameters, that is, the length of the initial translated text is less than or equal to the length of the text region in the region structure parameters, and the width of the text is less than or equal to the width of the text region, then it indicates that the initial translated text will not exceed the boundary of the region within the text region, and the initial translated text can be placed normally within the text region, that is, the initial translated text matches the text region.

[0122] Step S3223: If the visual parameters exceed the region structure parameters, then the initial translated text does not match the text region.

[0123] If the visual parameter exceeds the region structure parameter, that is, the length of the initial translation text is greater than the length of the text region in the region structure parameter, or the width of the text is greater than the width of the text region, it indicates that the initial translation text will exceed the boundary of the text region and cannot be placed normally within the text region, that is, the initial translation text does not match the text region.

[0124] exist Figure 8In the illustrated embodiment, it is possible to determine whether the initial translated text matches the text region based on the initial translated text and the actual structure of the text region, which effectively improves the efficiency and effectiveness of verifying the initial translated text.

[0125] Optionally, please refer to Figure 9 , Figure 9 This is a flowchart illustrating another translation method provided in an embodiment of this application. The method may further include steps S340-S350.

[0126] Step S340: Determine whether there are multiple duplicate text regions in the historical design interface that have the same initial text information and region structure parameters.

[0127] Considering the interconnected nature of multiple design interfaces, there may be multiple duplicate text regions with the same initial text information and regional structure parameters in the current design interface and historical design interfaces. Therefore, it is possible to first determine whether there are duplicate text regions in the processed and currently being processed historical design interfaces.

[0128] Optionally, the existence of duplicate text regions can be determined by comparing data. If the initial text information and region structure parameters of two text regions are exactly the same, then they are determined to be duplicate text regions.

[0129] Step S350: If there are multiple duplicate text regions, fill the multiple duplicate text regions with the first translated text corresponding to the multiple duplicate text regions.

[0130] If multiple duplicate text regions exist, the first determined translation text in each of the multiple duplicate text regions is directly used to fill the multiple duplicate text regions.

[0131] It should be noted that when designing the UI of a software, the identity information of each processed translated text and its corresponding duplicate text area, such as number, can be stored in the database. It is possible to first determine whether there is a duplicate text area before translating. If a duplicate text area is found, the database can be queried through the identity information of the duplicate text area to directly obtain the corresponding first translated text for filling.

[0132] exist Figure 9 In the illustrated embodiment, there is no need to repeatedly translate multiple repeating text regions, which effectively improves the efficiency of language conversion when designing the interface and reduces the computational cost required for translation.

[0133] Optionally, please refer to Figures 10-12 , Figure 10This is a schematic diagram of the initial state of the design interface for an in-vehicle system. Figure 11 This is a schematic diagram illustrating the state of machine translation of the design interface in existing technologies. Figure 12 This is a schematic diagram showing the translated state of the design interface provided in an embodiment of this application. According to... Figure 10 It can be seen that this design interface is a UI interface for selecting and setting various functions for unlocking car doors in the vehicle system. Figure 10 Chinese content in multiple text areas (such as...) Figure 10 When converting information such as "left rear door child lock," "right rear door child lock," etc., into English, the existing machine translation technology produces results such as... Figure 11 As shown, machine-translated English content is often quite long, resulting in numerous issues such as text overflow and incomplete text display, which negatively impacts user viewing and usability. In contrast, the design interface obtained by translating using the method provided in this application's embodiments is as follows: Figure 12 As shown, it can constrain the translation of text content within a text area, maintaining semantic consistency with the original Chinese content while reducing the length of the English content. This allows the translated text to fill the text area normally for users to view and use, effectively improving the efficiency and effectiveness of UI design.

[0134] This application also provides a computer program product, which includes a computer program / instructions. When the computer program / instructions are executed by a processor, they implement the steps in any of the above-described model training method or translation method.

[0135] In the several embodiments provided in this application, it should be understood that the disclosed device can also be implemented in other ways. The device embodiments described above are merely illustrative; for example, the block diagrams in the accompanying drawings illustrate the possible architecture, functions, and operations of the device according to various embodiments of this application. In this regard, each block in the block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram, and combinations of block diagrams, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0136] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0137] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0138] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application. It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0139] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

[0140] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A model training method, characterized in that, The method comprises: building an initial model for language translation; extracting visual features of multiple fonts in a font database; determining a semantic constraint condition for constraining the length of a historical translation text based on historical region structure parameters of a historical region in a historical design interface; training the initial model based on the visual features and the semantic constraint condition to obtain a constraint translation model.

2. The method of claim 1, wherein, The extraction of the visual features of multiple fonts in the font database comprises: determining visual format parameters of multiple fonts in the font database under different font sizes; wherein the visual format parameters comprise character width, character height, inter-character spacing, and line height; performing normalization processing and vector aggregation processing on multiple visual format parameters to obtain the visual features of each font.

3. The method of claim 1, wherein, The training of the initial model based on the visual features and the semantic constraint condition to obtain a constraint translation model comprises: determining a first loss during semantic translation based on the semantic constraint condition and an accuracy threshold of semantic translation; determining a second loss of the matching degree between the historical translation text and the historical region based on the visual features and the historical region structure parameters of the historical region; determining a joint loss based on the first loss and the second loss; training the initial model based on the joint loss to obtain the constraint translation model.

4. A method of translation, characterized by, The method comprises: determining initial text information and region structure parameters based on text regions in a design interface; processing the initial text information and the region structure parameters based on a constraint translation model to obtain translation texts; wherein the constraint translation model is determined based on the method of any one of claims 1-3; filling each translation text into a corresponding text region.

5. The method of claim 4, wherein, The determination of initial text information and region structure parameters based on text regions in a design interface comprises: obtaining the design interface to be translated from design software; identifying one or more text regions in the design interface; identifying the region structure parameters of each text region; wherein the region structure parameters comprise the shape, length, and width of the text region; reading the initial text information in the text region; wherein the initial text information comprises text content and text format; and the text format comprises at least one of font size, font name, line height, and line number.

6. The method of claim 4, wherein, The processing of the initial text information and the region structure parameters based on a constraint translation model to obtain translation texts comprises: processing the region structure parameters and the initial text information in the constraint translation model to obtain initial translation texts output by the constraint translation model; determining whether the initial translation texts match the corresponding text regions based on the region structure parameters; if it is determined that the initial translation texts match the text regions, taking the initial translation texts as the translation texts; If it is determined that the initial translation text does not match the text region, the initial text information and the region structure parameter are reprocessed by the constraint translation model to obtain an adjusted translation text, wherein the translation text is the adjusted translation text matching the text region. The similarity between the first character format of the initial text information and the second character format of the translation text is higher than a preset similarity threshold.

7. The method of claim 6, wherein, The method further comprises: determining a visual parameter of the initial translation text, wherein the visual parameter comprises a text length and a text width of the initial translation text; If it is determined that the visual parameter does not exceed the region structure parameter, it is determined that the initial translation text matches the text region; If it is determined that the visual parameter exceeds the region structure parameter, it is determined that the initial translation text does not match the text region.

8. The method according to any one of claims 4-7, characterized in that, The method further comprises: determining whether there are multiple repeated text regions with the same initial text information and region structure parameter in a historical design interface; If there are multiple repeated text regions, first translation texts corresponding to the multiple repeated text regions are filled into the multiple repeated text regions.

9. An electronic device, comprising: The electronic device comprises a memory and a processor, the memory stores program instructions, and the processor executes the program instructions to perform the steps in the method of any one of claims 1-3 or 4-8.

10. A computer program product, characterised in that, The computer program product comprises computer programs / instructions, which are executed by the processor to implement the steps in the method of any one of claims 1-3 or 4-8.

Images