Partial text translation method for game interface, storage medium and computer program product

Abstract

The application discloses a game interface local text translation method, a storage medium and a computer program product, and relates to the technical field of game interface translation, wherein the method comprises the following steps: moving a floating icon in response to a floating icon dragging instruction of a user; collecting a game interface image carrying the floating icon, and identifying position information of an original text block and position information of the floating icon in the game interface image; determining a text block of interest of the user according to the position information of the floating icon and the position information of the original text block in the game interface image; calling a translation model to translate text information in the text block of interest, and generating and displaying a translated text block. The application reduces the user operation difficulty of game interface local text translation, and significantly improves the user experience.

Description

Technical Field

[0001] This application relates to the field of game interface translation technology, and in particular to a method, storage medium, and computer program product for partial text translation of a game interface. Background Technology

[0002] In game interfaces, there is often text in a non-native language, which users need to translate into a language they can understand.

[0003] In mobile game interface text translation technologies, global text translation and global text display methods are commonly used. However, these methods suffer from high resource consumption and low efficiency. Therefore, existing technologies have proposed a path for local text translation in game interfaces. The main idea is to respond to the user's text box drag command, capture the text block of interest, and translate and display it. However, this method requires the user to first select a base point and manually adjust the text box size, which affects the user experience. Summary of the Invention

[0004] The main purpose of this application is to provide a method, storage medium, and computer program product for translating partial text in a game interface, aiming to solve the technical problem of poor user experience in existing methods for translating partial text in a game interface.

[0005] To achieve the above objectives, this application proposes a method for partial text translation of a game interface, comprising: Move the floating icon in response to the user's drag command. Collect game interface images with floating icons, and identify the position information of the original text blocks and the position information of the floating icons in the game interface images; Based on the position information of the floating icon in the game interface image and the position information of the original text block, determine the text block of interest to the user; The translation model is invoked to translate the text information in the text block of interest, and the translated text block is generated and displayed.

[0006] In one embodiment, determining the text block of interest for the user based on the position information of the floating icon in the game interface image and the position information of the original text block includes: When the floating icon is located within the bounding box of one of the original text blocks, the original text block surrounding the floating icon is determined as the text block of interest, wherein the position information of the original text block is represented by the bounding box; And / or, Calculate the distance between the floating icon and each original text block, and identify the original text blocks whose distance from the floating icon is less than the distance threshold as text blocks of interest.

[0007] In one embodiment, moving the floating icon in response to a user's floating icon drag command includes: The active floating icon is moved in response to the user's floating icon drag command, and the passive floating icon is moved by an equal directional travel amount that is a preset multiple of the active floating icon. The step of determining the text block of interest for the user based on the position information of the floating icon in the game interface image and the position information of the original text block includes: Based on the position information of the active floating icon in the game interface image and the position information of the original text block, determine the text block of interest to the user.

[0008] In one embodiment, determining the text block of interest for the user based on the position information of the floating icon in the game interface image and the position information of the original text block includes: The movement direction of the floating icon is calculated based on the position information of the floating icon in multiple consecutive frames of the game interface image; Based on the movement direction of the floating icon and the position information of the original text block, the text block of interest to the user is determined.

[0009] In one embodiment, the step of calling a translation model to translate the text information in the text block of interest, and generating and displaying the translated text block includes: The translation model is invoked to translate the original text information of each original text block to obtain the corresponding translated text information; After the text block of interest is confirmed, the translated text information corresponding to the text block of interest will be displayed.

[0010] In one embodiment, the step of calling a translation model to translate the text information in the text block of interest, and generating and displaying the translated text block includes: After the text block of interest is identified, the translation model is invoked to translate the original text information of the text block of interest in order to obtain the translated text information of the text block of interest. The translated text information of the text block of interest is displayed.

[0011] In one embodiment, the step of calling a translation model to translate the text information in the text block of interest, and generating and displaying the translated text block includes: At least two monolingual text recognition models are loaded synchronously, and text recognition is performed on the same frame of game interface image or the same text block of interest in a multi-threaded parallel manner to extract the original text information of each original text block in the game interface image or the original text information of the text block of interest. The recognition results of each monolingual text recognition model are fused to output aggregated text information and location information.

[0012] In one embodiment, the step of calling a translation model to translate the text information in the text block of interest, and generating and displaying the translated text block includes: Obtain the location information of the gaming device; Based on the location information of the game device and the preset calling rules, the target translation model is invoked to translate the original text information of each original text block to obtain the corresponding translated text information.

[0013] In addition, to achieve the above objectives, this application also proposes a storage medium that is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the game interface partial text translation method described above.

[0014] In addition, to achieve the above objectives, this application also proposes a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the game interface partial text translation method described above.

[0015] One or more technical solutions proposed in this application have at least the following technical effects: This paper proposes a method for local text translation in game interfaces. The method moves the floating icon in response to the user's drag command, thus representing the user's region of interest (ROI) based on the icon's position. It acquires game interface images carrying the floating icon, identifies the position information of the original text blocks and the floating icon within the image, and locates each text block and icon. Based on the position information of the floating icon and the original text blocks, it determines the text blocks of ROI, filtering them for translation. Finally, it uses a translation model to translate the text information in the ROI, generating and displaying the translated text blocks. This method reduces the user's operational difficulty in local text translation of game interfaces and significantly improves the user experience. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the method for translating partial text in the game interface. Figure 2This is a scene illustration of a game interface. Figure 3 This is an illustration of another scene in the game interface; Figure 4 This is another scene illustration of the game interface; Figure 5 This is a schematic diagram of a scenario where there are dense blocks of text in the game interface.

[0019] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0020] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application. To better understand the technical solutions of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0021] This application provides a method for translating partial text in a game interface. In the first embodiment of this application's method for translating partial text in a game interface, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the game interface partial text translation method of this application. The game interface partial text translation method may include steps S10 to S40: Step S10: Move the floating icon in response to the user's floating icon drag command.

[0022] In one feasible implementation, a floating icon is configured on the operating interface or game interface of an electronic device (phone or tablet). Users can long press the floating icon, drag it to the target position in the game interface, and then release it to input the floating icon drag command.

[0023] Step S20: Acquire a game interface image carrying a floating icon, and identify the position information of the original text block and the position information of the floating icon in the game interface image.

[0024] Specifically, after acquiring the game interface image, text blocks and floating icons are identified and located based on image processing technology.

[0025] Step S30: Determine the text block of interest to the user based on the position information of the floating icon in the game interface image and the position information of the original text block.

[0026] It should be noted that the game interface image refers to one frame or multiple consecutive frames of the game interface image, and correspondingly, the position information of the floating icon is one or more.

[0027] In one possible implementation, step S30 may include step S311 or S312: Step S311: When the floating icon is located within the bounding box of one of the original text blocks, the original text block surrounding the floating icon is determined as the text block of interest, wherein the position information of the original text block is represented by the bounding box.

[0028] In one specific implementation, after acquiring the game interface image, the image coordinates of the center of the floating icon and the image coordinates of the bounding box of the original text block are collected. Based on the image coordinates of the center of the floating icon and the image coordinates of the bounding box of the original text block, the original text block in which the floating icon is located is identified. After the identification is completed, the original text block surrounding the floating icon is determined as the text block of interest.

[0029] Step S312: Calculate the distance value between the floating icon and each original text block, and determine the original text blocks whose distance value from the floating icon is less than the distance threshold as text blocks of interest.

[0030] In one specific implementation, after acquiring the game interface image, the image coordinates of the center of the floating icon and the image coordinates of the bounding box of the original text block are collected. Based on the image coordinates of the center of the floating icon and the image coordinates of the bounding box of the original text block, it is identified which original text block has a distance value less than a preset distance threshold between the floating icon and the original text block. After the identification is completed, the original text block with a distance value less than the distance threshold from the floating icon is identified as the text block of interest.

[0031] It should be noted that the distance value can be represented in pixels. The distance between the floating icon and the original text block can be configured as the distance between the center points, or as the shortest distance from the center of the floating icon to the bounding box of the original text block.

[0032] For example, such as Figure 2 As shown, the game interface image includes original text block B and original text block A. When the floating icon is located within the bounding box of original text block B or the distance value between the floating icon and the original text block is less than the distance threshold, original text block B is determined as the text block of interest.

[0033] It is understandable that, compared to selecting text blocks of interest in a text box, the method for determining text blocks of interest in this embodiment better meets the lightweight interaction requirements in game scenarios.

[0034] In one feasible implementation, step S10 may include step S110: Step S110: In response to the user's floating icon drag command, move the active floating icon and move the passive floating icon by an equal directional travel amount that is a preset multiple of the active floating icon. Accordingly, step S30 may include step S320: Step S320: Determine the text block of interest to the user based on the position information of the active floating icon and the position information of the original text block in the game interface image.

[0035] It should be noted that equidirectional travel is defined as travel in the same direction. For example, it represents the travel of an active floating icon moving one target unit vertically, and the travel of a passive floating icon moving two target units vertically. For example,... Figure 3 As shown, the game interface includes active floating icons and passive floating icons. When the user controls the active floating icon to move vertically upward, the passive floating icon moves vertically upward a greater distance.

[0036] Understandably, on large-screen electronic devices, the target text block may be far from the location of the floating icon. Users need to long-press the floating icon and move it significantly to select the target text block, which may reduce the user experience. The method for determining the text block of interest in this embodiment allows users to move the active floating icon slightly to move the passive floating icon into the bounding box of the text block of interest, or to make the distance between the passive floating icon and the text block of interest less than a distance threshold. This significantly reduces the user's physical movement distance and operation time, and makes it less likely to accidentally select other text blocks due to large drags. In addition, the active floating icon can be set to a size that is easy for users to select, while the passive floating icon can be set to transparent and / or small in size to reduce obstruction of the game screen.

[0037] In one specific implementation, the active floating icon and the passive floating icon are stacked. Preferably, the active floating icon and the passive floating icon are set on the left or right interface of the game interface, which can be configured by the user based on actual usage. Preferably, the active floating icon and the passive floating icon are automatically reset after being moved.

[0038] In another specific implementation, the active floating icon and the passive floating icon are set in different positions on the game interface. Preferably, the active floating icon is set on the left or right side of the game interface, and the passive floating icon is set concentrically with the game interface.

[0039] In another possible implementation, step S30 may include step S330 (steps S331~S332): Step S331: Calculate the motion direction of the floating icon based on the position information of the floating icon in multiple consecutive game interface images; Step S332: Determine the text block of interest to the user based on the movement direction of the floating icon and the position information of the original text block.

[0040] Specifically, after acquiring multiple consecutive frames of game interface images, the position information of the floating icon in the game images is identified, and the movement direction of the floating icon is determined by line fitting based on multiple point information, for example, based on the least squares method. Then, based on the movement direction of the floating icon and the position information of the original text block, the user's intention (which original text block to move the floating icon to) is identified or predicted. For example, when the original text block is located in the movement direction, or the distance from the original text block to the line containing the movement direction is less than a threshold, the original text block is identified as the text block of interest. It should be noted that the movement direction is directional, and only the original text block in front of the floating icon is considered, not the original text block behind the movement.

[0041] For example, such as Figure 4 As shown, the solid line represents the initial floating icon, and the dashed line represents the floating icon after it has moved. By using the position information of the floating icon in at least two frames of game interface images, the movement direction of the floating icon can be calculated, and the original text block B can be determined as the text block of interest to the user based on the movement direction.

[0042] Understandably, both of the aforementioned methods for determining text of interest rely on the instantaneous position of the floating icon. This means the user must move the floating icon completely to the vicinity or above the target text block to identify the text of interest and trigger translation. Furthermore, smaller text blocks require more time to select, impacting the real-time performance and smoothness of the translation. The method for determining text of interest in this embodiment can identify the text block of interest before the floating icon even reaches it. Compared to methods that rely on the final position or instantaneous distance of the floating icon, this shortens the recognition time. This reduced time can be used to complete the translation and display, thereby improving the smoothness and real-time performance of the translation and avoiding delayed translation.

[0043] In one specific implementation, step S332 may include the following steps: Each original text block is scored based on a scoring rule that includes distance, trajectory, and direction components, and the original text block with the highest score is identified as the text block of interest.

[0044] Among them, the direction component is used to quantify the consistency between the angle between the target vector pointing from the original position of the floating icon to the original text block and the direction of movement of the floating icon; The trajectory sub-item is used to quantify the spatial proximity between the current position of the floating icon and the original text block; The distance component is used to quantify the vertical distance from the center point of the original text block to the line containing the direction of movement.

[0045] Preferably, the spatial proximity of the trajectory sub-items is calculated based on the shortest distance between the center point of the floating icon and the bounding box of the original text block, so as to eliminate the influence of the size of the original text block on the recognition result.

[0046] It is understandable that if a single parameter (such as whether the distance is less than a threshold or whether it is on a straight line in the direction of movement) is used, misidentification of text blocks of interest may occur in scenarios where the original text blocks are relatively dense. The method for determining text blocks of interest in this implementation process determines text blocks of interest by using distance, direction, and trajectory components together, which significantly improves the recognition accuracy of text blocks of interest in scenarios where the original text blocks are dense.

[0047] For example, such as Figure 5 As shown, the original text block A and the original text block B are located in front of the direction of the floating icon's movement (to clearly show the lines, two original text blocks with significantly different orientations are used as an illustration). Based on the calculated d1 and d2, the trajectory components of the original text block A and the original text block B are calculated. Based on the calculated α and β, the direction components of the original text block A and the original text block B are calculated. Based on the calculated L1 and L2, the distance components of the original text block A and the original text block B are calculated. Finally, based on the preset weights and the values ​​of each component, the total score of the original text block is calculated. In this example, the weight of the direction component is set to 0.45, the weight of the trajectory component is set to 0.25, and the weight of the distance component is set to 0.2.

[0048] In one specific implementation, in the scheme employing step S110, step S331 may include the following steps: The motion direction of the passive floating icon is calculated based on the position information of the passive floating icon in multiple consecutive game interface images; Accordingly, step S332 may include the following steps: Based on the movement direction of the passive floating icon and the position information of the original text block, determine the text block of interest to the user.

[0049] It is understandable that the second and third implementation methods of S30 provided above, compared with the first implementation method, reduce the user's operating difficulty and significantly improve the user experience while ensuring the accuracy of the recognition of the text block of interest. It is worth noting that these two methods can also shorten the recognition speed of the text block of interest, thereby improving the fluency of text translation.

[0050] The above are only three feasible implementation methods of step S30 provided in this embodiment. This embodiment does not specifically limit the specific implementation method of step S30.

[0051] Step S40: Call the translation model to translate the text information in the text block of interest, and generate and display the translated text block.

[0052] Specifically, it can be configured to extract the original text information from the text block of interest after the text block of interest is determined, call the translation model to translate the original text information, generate a translated text block carrying the translated text information, and display the translated text block in the form of a floating window or covering the corresponding original text block.

[0053] In one possible implementation, step S40 may include step S410: Step S410: Call the translation model to translate the original text information of each original text block to obtain the corresponding translated text information; after the text block of interest is confirmed, display the translated text information corresponding to the text block of interest.

[0054] In another feasible implementation, step S40 may include step S420: Step S420: After confirming the text block of interest, call the translation model to translate the original text information of the text block of interest to obtain the translated text information of the text block of interest; then display the translated text information of the text block of interest.

[0055] It is understandable that the second implementation of S40 provided above, compared to the first implementation, only requires translating the text block of interest, thus reducing performance loss.

[0056] The above are only two feasible implementations of step S40 provided in this embodiment. This embodiment does not specifically limit the specific implementation of step S40.

[0057] In one feasible implementation, generating the translated text block in step S40 may include the following steps: At least two monolingual text recognition models are loaded synchronously, and text recognition is performed on the same frame of game interface image or the same text block of interest in a multi-threaded parallel manner to extract the original text information of each original text block in the game interface image or the original text information of the text block of interest. The recognition results of various monolingual text recognition models are fused to output aggregated text information and location information. In one specific implementation, three independent OCR models from Google ML Kit—Latin, Japanese, and Korean—are loaded simultaneously. These models are then used in a multi-threaded parallel manner to recognize the same frame of image. The recognition results are aligned using timestamps, and the aggregated text information and location information are output.

[0058] Understandably, configuring multiple monolingual translation models significantly improves the recognition rate of game text.

[0059] In one feasible implementation, generating the translated text block in step S40 may include the following steps: Obtain the location information of the game device; based on the location information of the game device and the preset calling rules, call the target translation model to translate the original text information of each original text block to obtain the corresponding translated text information.

[0060] In one specific implementation, after user authorization, a local or cloud-based translation model is selected based on regional configuration, network reachability, and model availability; if the game device is detected to be connected to a domestic Wi-Fi network, the local translation model is activated first.

[0061] This embodiment of a method for translating partial text in a game interface involves moving a floating icon in response to a user's drag command, thus representing the user's region of interest through the position of the floating icon. The method includes: acquiring game interface images carrying the floating icons; identifying the position information of the original text blocks and the floating icons within the game interface image; determining the user's text blocks of interest based on the position information of the floating icons and the original text blocks in the game interface image; and finally, calling a translation model to translate the text information in the text blocks of interest, generating and displaying the translated text blocks. This method reduces the user's operational difficulty in translating partial text in a game interface and significantly improves the user experience.

[0062] This application also provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the game interface partial text translation method in the above embodiments.

[0063] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory) with one or more wires electrically connected, optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, etc., or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by an instruction execution system or device, or a combination thereof. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0064] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the game interface partial text translation method described above.

[0065] The above are only some embodiments of this application and do not limit the scope of protection of this application. All equivalent structural transformations made under the technical concept of this application and using the content of this application specification and drawings, or direct / indirect applications in other related technical fields, are included in the scope of protection of this application.

Claims

1. A method for translating partial text in a game interface, characterized in that, The method includes: Move the floating icon in response to the user's drag command. Collect game interface images with floating icons, and identify the position information of the original text blocks and the position information of the floating icons in the game interface images; Based on the position information of the floating icon in the game interface image and the position information of the original text block, determine the text block of interest to the user; The translation model is invoked to translate the text information in the text block of interest, and the translated text block is generated and displayed. The step of determining the text block of interest for the user based on the position information of the floating icon in the game interface image and the position information of the original text block includes: The movement direction of the floating icon is calculated based on the position information of the floating icon in multiple consecutive frames of the game interface image; Based on the movement direction of the floating icon and the position information of the original text block, the text block of interest to the user is determined, including: Each original text block is scored based on a scoring rule that includes distance, trajectory, and direction components, and the original text block with the highest score is identified as the text block of interest. Among them, the direction component is used to quantify the consistency between the angle between the target vector pointing from the original position of the floating icon to the original text block and the direction of movement of the floating icon; The trajectory sub-item is used to quantify the spatial proximity between the current position of the floating icon and the original text block; The distance component is used to quantify the vertical distance from the center point of the original text block to the line containing the direction of movement.

2. The method for translating partial text in a game interface as described in claim 1, characterized in that, The step of determining the text block of interest for the user based on the position information of the floating icon in the game interface image and the position information of the original text block includes: When the floating icon is located within the bounding box of one of the original text blocks, the original text block surrounding the floating icon is determined as the text block of interest, wherein the position information of the original text block is represented by the bounding box; And / or, Calculate the distance between the floating icon and each original text block, and identify the original text blocks whose distance from the floating icon is less than the distance threshold as text blocks of interest.

3. The method for translating partial text in a game interface as described in claim 1, characterized in that, The method of moving the floating icon in response to the user's floating icon drag command includes: The active floating icon is moved in response to the user's floating icon drag command, and the passive floating icon is moved by an equal directional travel amount that is a preset multiple of the active floating icon. The step of determining the text block of interest for the user based on the position information of the floating icon in the game interface image and the position information of the original text block includes: Based on the position information of the active floating icon in the game interface image and the position information of the original text block, determine the text block of interest to the user.

4. The method for translating partial text in a game interface as described in claim 1, characterized in that, The process of calling the translation model to translate the text information in the text block of interest, generating and displaying the translated text block includes: The translation model is invoked to translate the original text information of each original text block to obtain the corresponding translated text information; After the text block of interest is confirmed, the translated text information corresponding to the text block of interest will be displayed.

5. The method for translating partial text in a game interface as described in claim 1, characterized in that, The process of calling the translation model to translate the text information in the text block of interest, generating and displaying the translated text block includes: After the text block of interest is identified, the translation model is invoked to translate the original text information of the text block of interest in order to obtain the translated text information of the text block of interest. The translated text information of the text block of interest is displayed.

6. The method for translating partial text in a game interface as described in claim 1, characterized in that, The process of calling the translation model to translate the text information in the text block of interest, generating and displaying the translated text block includes: At least two monolingual text recognition models are loaded synchronously, and text recognition is performed on the same frame of game interface image or the same text block of interest in a multi-threaded parallel manner to extract the original text information of each original text block in the game interface image or the original text information of the text block of interest. The recognition results of each monolingual text recognition model are fused to output aggregated text information and location information.

7. The method for translating partial text in a game interface as described in claim 1, characterized in that, The process of calling the translation model to translate the text information in the text block of interest, generating and displaying the translated text block includes: Obtain the location information of the gaming device; Based on the location information of the game device and the preset calling rules, the target translation model is invoked to translate the original text information of each original text block to obtain the corresponding translated text information.

8. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the game interface partial text translation method as described in any one of claims 1 to 7.

9. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the game interface partial text translation method as described in any one of claims 1 to 7.

Images