Text superimposed box detection method and device, and electronic equipment

Abstract

The application provides a text superframe detection method and device and electronic equipment, obtains an interface image of a target game interface, determines a target detection region in the interface image based on text frame information of the target game interface, performs image processing on the target detection region through a pre-trained text superframe detection model to obtain a processing result, and determines whether the text content displayed in the target game interface exceeds the corresponding text frame based on the processing result. In this way, the detection accuracy of the text superframe is improved by performing image processing on the detection region corresponding to the text frame in the interface image through the text superframe detection model established based on the deep learning network, and then determining whether the target game interface has the text superframe phenomenon.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically, to a method, apparatus, and electronic device for detecting text exceeding the bounding box. Background Technology

[0002] Game scenes typically include text boxes to display game-related content. When in-game text needs to be translated into other languages, the translated text may exceed the preset text box content, resulting in poor display quality.

[0003] In related technologies, rule-based judgment methods based on engine data or traditional image processing methods are commonly used to determine whether the displayed text in a text box exceeds the bounding box. Because the text position obtained from engine data is not accurate enough, and because the UI tree in a game screen contains a great deal of content, it is difficult to clearly identify other UI controls that should be compared with the text. Therefore, it is necessary to compare the text with the positions of its parent and sibling nodes in the UI control tree. Using rules based on engine data for bounding box logic judgment can lead to some false positives. Furthermore, due to the rich and diverse nature of game screens, using traditional image processing methods for bounding box detection will produce many false positives. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a method, apparatus and electronic device for detecting text overframes, so as to improve the detection accuracy of text overframes.

[0005] In a first aspect, embodiments of the present invention provide a text over-bound detection method, comprising: acquiring an interface image of a target game interface; determining a target detection region in the interface image based on text box information of the target game interface; performing image processing on the target detection region using a pre-trained text over-bound detection model to obtain a processing result; and determining, based on the processing result, whether the text content displayed in the target game interface exceeds the corresponding text box.

[0006] Secondly, embodiments of the present invention provide a text over-bound detection device, comprising: a target detection region determination module, configured to acquire an interface image of a target game interface and determine a target detection region in the interface image based on text box information of the target game interface; a model processing module, configured to perform image processing on the target detection region using a pre-trained text over-bound detection model to obtain a processing result; and a text over-bound determination module, configured to determine, based on the processing result, whether the text content displayed in the target game interface exceeds the corresponding text box.

[0007] Thirdly, embodiments of the present invention provide an electronic device, including a processor and a memory, wherein the memory stores machine-executable instructions that can be executed by the processor, and the processor executes the machine-executable instructions to implement the above-described text superframe detection method.

[0008] Fourthly, embodiments of the present invention provide a machine-readable storage medium storing machine-executable instructions. When the machine-executable instructions are invoked and executed by a processor, the machine-executable instructions cause the processor to implement the above-described text overframe detection method.

[0009] The embodiments of the present invention bring the following beneficial effects:

[0010] The aforementioned text over-bounding detection method, apparatus, and electronic device acquire an interface image of a target game interface, determine the target detection region in the interface image based on the text box information of the target game interface, perform image processing on the target detection region using a pre-trained text over-bounding detection model to obtain the processing result, and determine whether the text content displayed in the target game interface exceeds the corresponding text box based on the processing result. In this method, the text over-bounding detection model based on a deep learning network performs image processing on the detection region corresponding to the text box in the interface image, thereby determining whether the target game interface has experienced text over-bounding, thus improving the accuracy of text over-bounding detection.

[0011] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention are realized and obtained in accordance with the structures particularly pointed out in the description, claims and drawings.

[0012] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0013] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0014] Figure 1 A flowchart of a text hyperframe detection method provided in an embodiment of the present invention;

[0015] Figure 2 A flowchart of the model training phase provided in an embodiment of the present invention;

[0016] Figure 3 A graphical user interface diagram provided for an embodiment of the present invention;

[0017] Figure 4A schematic diagram of a candidate region for adding random text to a text box, provided in an embodiment of the present invention;

[0018] Figure 5 A schematic diagram of a model structure provided in an embodiment of the present invention;

[0019] Figure 6 A flowchart of a model testing phase provided in an embodiment of the present invention;

[0020] Figure 7 A schematic diagram of a model output result provided in an embodiment of the present invention;

[0021] Figure 8 This is a schematic diagram of the structure of a text overframe detection device provided in an embodiment of the present invention;

[0022] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] With the widespread adoption of computers and mobile smartphones, gaming has become an increasingly common form of entertainment. In the current context of the gaming industry's development, the expansion of domestic games overseas is an inevitable trend, and game localization is a crucial element. With the rapid expansion of overseas business, more and more major game companies have realized the importance of game localization quality, with translation being paramount. Due to the different characteristics and lengths of various languages, UI text translated from Chinese into different languages ​​often suffers from numerous issues such as exceeding the text frame limit or UI overlap, especially when the target language is a Latin-based or Southeast Asian language. Troubleshooting these issues typically involves manual visual inspection within the game, a method that is inefficient and prone to omissions. Therefore, the ability to automatically and efficiently identify text exceeding the text frame limit in the game screen using game screen and game engine data has received significant attention during testing.

[0025] In related technologies, methods for detecting text overflow in user interfaces (UI) generally fall into two categories: rule-based methods based on game engine data and methods based on traditional image processing. Rule-based methods based on game engine data obtain the coordinates and size information of the text through a game engine, and compare the text position with the positions of its parent and sibling nodes in the UI tree to determine if the text overflows or overlaps with a text box or nearby UI elements. Methods based on traditional image processing acquire the text region from an image, extract the surrounding image as the object to be detected, and perform traditional image processing algorithms such as grayscale conversion, binarization, and edge detection on the image to determine if there are obvious button boxes or text box edges within the text region, thus determining whether a text overflow problem has occurred.

[0026] The two methods for detecting text overflow in user interfaces described above have the following problems in practical production applications:

[0027] 1. Inaccurate text position: Ideally, the text position obtained from the engine layer should be the target area of ​​the smallest envelope text. However, in practice, due to the consideration of reserving text position, the width and height of the obtained text are generally larger than the actual text width and height. Therefore, when performing logical judgment on whether the text exceeds the frame, this part of the error will cause significant false alarms.

[0028] 2. The comparison object of the text cannot be accurately defined: The text over-bounds in the screen is manifested by over-bounding or overlapping with its background control box, background UI box or nearby UI box. However, because there is a lot of content in the UI tree of the game screen, it is difficult to clearly find other UI that should be compared with the text. Therefore, it is necessary to compare the text with the position of its parent node and sibling nodes in the UI tree. This will cause some false alarms when the over-bounds logic is judged.

[0029] 3. Diverse and rich game visuals: The background frames of game text are diverse and varied. A clean, solid-color text background only occupies a portion of the game scene, and there is a possibility that the text may float on the game screen. In the process of using traditional image processing methods to detect super-boundary text, it is very likely that many edges will be extracted within the text area. When judging the rules, it is easy to misreport such text as having super-boundary issues.

[0030] The UI design styles are diverse: UI styles are not limited to regular rectangles, but may also include irregular UIs such as circles and rotations. Moreover, the UI design style of each game is different, which may conflict with the logic of judging the bounding box in the initial setting of the algorithm, causing a large number of false alarms.

[0031] Based on this, the present invention provides a text overlay detection method, apparatus and electronic device, which can be applied to text overlay detection scenarios in various games.

[0032] See Figure 1 First, a text hyperframe detection method provided by an embodiment of the present invention will be introduced, which includes the following steps:

[0033] Step S102: Obtain the interface image of the target game interface, and determine the target detection area in the interface image based on the text box information of the target game interface.

[0034] The aforementioned interface image can be obtained by taking a screenshot of the target game interface during game execution. The text box information is typically set by relevant technical personnel during the generation of the target game interface. This text box information can include the positions of the four vertices of the text box in the graphical user interface, the position of the center of the text box in the graphical user interface, and the size of the text box, such as 20×30 pixels.

[0035] Based on the text box information, the text box area in the target game interface can be determined. Furthermore, the image area in the interface image corresponding to the text box area can be defined as the target detection area. Alternatively, the image area can be expanded around the text box area, such as expanding it upwards, downwards, left, and right by 5 pixels respectively.

[0036] Step S104: The target detection region is processed by a pre-trained text hyperframe detection model to obtain the processing result.

[0037] The aforementioned text hyperframe detection models are typically built upon deep learning networks, neural networks, etc. These models require image processing of the target detection region to extract image features and classify the region based on these features, outputting the probability that the text content in the target detection region exceeds the corresponding text box. To reduce the introduction of useless image information and decrease the image processing workload of the text hyperframe detection model, only a local interface image of the target detection region can be input into the model.

[0038] The text hyperframe detection problem can be viewed as a binary classification problem in images. Several network architectures in deep learning are specifically designed to solve this problem. A suitable network architecture can be selected based on the characteristics of different architectures to build a text hyperframe detection model; the specific choice depends on the requirements and is not limited here.

[0039] When developing a text bounding box detection model, a large number of positive and negative samples are needed to ensure that the model can balance the image features of game interface images with text bounding and without bounding boxes. Positive samples refer to game interface images with text not bounding boxes, while negative samples refer to game interface images with text bounding boxes. When negative samples are difficult to collect during game execution, they can be generated manually. For example, text content bounding boxes can be added to the edges of text boxes in game interface images with text not bounding boxes to generate negative samples.

[0040] Step S106: Based on the processing result, determine whether the text content displayed in the target game interface exceeds the corresponding text box.

[0041] The output of a text hyperframe detection model can be the probability that text content in the detected area exceeds the corresponding text box, or it can be other features of the text content in the detected area, such as edge image features of the detected area. The specific content of the output depends on the deep learning network structure used by the text hyperframe detection model.

[0042] By setting specific thresholds for the processing results, such as probability thresholds or feature thresholds, the text content in the target game interface can be considered to fall into one of two categories: exceeding the frame or not exceeding the frame, depending on the content of the processing result. No restrictions are imposed here.

[0043] The aforementioned text over-bounding detection method acquires an image of the target game interface, determines the target detection region in the interface image based on the text box information of the target game interface, processes the target detection region using a pre-trained text over-bounding detection model to obtain the processing result, and determines whether the text content displayed in the target game interface exceeds the corresponding text box based on the processing result. In this method, the text over-bounding detection model based on a deep learning network processes the detection region corresponding to the text box in the interface image, thereby determining whether the target game interface has experienced text over-bounding, thus improving the accuracy of text over-bounding detection.

[0044] In practical implementation, the above text hyperframe detection model can be trained in the following ways:

[0045] (1) Based on a pre-defined deep learning network, establish an initial model. The deep learning network can be of various types, such as recurrent convolutional networks, ResNet18, etc., and can be selected according to requirements. Multiple deep learning networks can also be used in combination. The initial model adopts the network structure of the corresponding deep learning network.

[0046] (2) Determine training data from a preset sample set; the training data includes local interface images and annotation parameters of the local interface images; the local interface images display the text content corresponding to the text boxes set in the test game interface; the annotation parameters are used to indicate whether the text content displayed in the local interface images exceeds the corresponding text boxes.

[0047] Detecting whether text exceeds its bounding box can be viewed as a binary classification problem. Therefore, during model training, both text samples that exceed the bounding box and those that do not are needed to enable the model to learn and distinguish their features. Correspondingly, the sample set can include a positive sample set and a negative sample set. The annotation parameters of multiple local interface images in the positive sample set indicate that the text content does not exceed the corresponding text box; the annotation parameters of multiple local interface images in the negative sample set indicate that the text content exceeds the corresponding text box. When determining the training data, a first number of local interface images can be selected from the preset positive sample set, and a second number of local interface images can be selected from the preset negative sample set; the first number matches the second number; the selected local interface images and their annotation parameters are then used as the training data. This method of selecting the same number of positive and negative samples as training data avoids the model being affected by a large number of positive samples, reduces overfitting, and enhances the model's generalization ability.

[0048] Because the probability of text exceeding the text box during game execution is far less than the probability of it not exceeding the text box, the number of negative samples is usually small. In practice, a negative sample set can be generated as follows: For a local interface image in the positive sample set, generate random text content based on the display format of the text content displayed in the local interface image; then, add random text content to a specified position of the text content displayed in the local interface image, such as the edge of the text box or adjacent to existing text, to obtain the updated text content; finally, determine whether the updated text content displayed in the local interface image exceeds the corresponding text box; if it does, the local interface image is determined as a local interface image in the negative sample set. The above judgment process can be implemented manually or through traditional image processing methods; no restrictions are placed here.

[0049] (3) Input the local interface image into the initial model, and perform image processing on the local interface image through the initial model to obtain the processing result; the processing result includes the probability parameter of the text content displayed in the local interface image exceeding the corresponding text box. The above image processing methods usually include image enhancement processing, image feature extraction processing, etc., extracting single-level or multi-level features of the image, and outputting the probability parameter of the text in the image exceeding the corresponding text box based on the image features.

[0050] (3) Calculate the loss value of the initial model based on the preset loss function, the annotation parameters of the local interface image, and the processing results. In specific implementation, the cross-entropy function for binary classification can be used as the loss function, or a commonly used loss function for binary classification problems can be used. There are no restrictions here.

[0051] (4) Determine whether the loss value of the initial model has converged. If yes, determine the trained initial model as the text hyperbox detection model; otherwise, proceed to step (5).

[0052] (5) Based on the loss value, adjust the network parameters of the initial model and execute step (2). The network parameters can be adjusted using various network optimizers to make the loss value converge quickly.

[0053] The following embodiments provide text box information of a target game interface and determine the implementation method of target detection area in the interface image.

[0054] The information in the text boxes mentioned above is usually set by relevant personnel during the generation of the target game interface. It may include the position and size parameters of the text box in the target game interface. The position parameter may indicate the position of the center of the text box or the position of a vertex of the text box.

[0055] After obtaining the text box information, the image region corresponding to the text box in the interface image can be determined based on the position and size parameters of the text box set in the target game interface. Since other controls usually exist in the target game interface, and these controls may obscure the text box, the text content may not be displayed. Therefore, it is necessary to detect whether text content is displayed in the image region corresponding to the text box; if so, the image region corresponding to the text box is determined as the target detection region.

[0056] The following embodiments provide an implementation method for image processing of target detection regions using a pre-trained text hyperframe detection model.

[0057] After determining the target detection region, the interface image needs to be cropped based on the target detection region to obtain a local interface image. This local interface image includes the target detection region and can be the same size as the target detection region, or it can be slightly larger than the target detection region by appropriately expanding its surrounding area. Based on preset detection size parameters, the local interface image undergoes a size transformation; these parameters are the standard sizes input to the text super-boundary detection model. Finally, the processed local interface image is input into the pre-trained text super-boundary detection model, which performs image processing on the local interface image. The text super-boundary detection model typically extracts image features from the local interface image. Since it needs to detect whether the text content in the local interface image exceeds the bounding box, it usually focuses on detecting edge image features. The image processing method corresponds to the deep learning network backbone used by the text super-boundary detection model; specific settings can be configured according to requirements and are not limited here.

[0058] The following embodiment provides an implementation method for determining whether the text content displayed in the target game interface exceeds the corresponding text box based on the processing result.

[0059] When using deep learning networks to solve binary classification problems, the output is typically the probability of belonging to one of the two categories. Similarly, the output of a text hyperframe detection model can be the probability parameter of whether the text content displayed in the detected area exceeds the corresponding text box.

[0060] A probability threshold can be preset. This probability threshold can be determined by testing the text over-boundary detection model or by experience. After determining the probability threshold, it can be judged whether the probability parameter of the text content displayed in the target detection area exceeding the corresponding text box is greater than the preset probability threshold; if so, it is determined that the text content displayed in the target game interface exceeds the corresponding text box; if not, it is determined that the text content displayed in the target game interface does not exceed the corresponding text box.

[0061] This invention also provides another method for detecting text superframes, which... Figure 1 This method is implemented based on the method shown. It can detect whether text exceeds its background control box or background text box, and whether text overlaps with other user interface elements. It can adapt to a variety of game scenarios and UI design styles, accurately detects multilingual UI text exceeding the box, has few false positives, effectively assists manual verification, saves manpower, and has a wide range of applications.

[0062] This method is mainly implemented through the following steps:

[0063] 1. Game Scene Sample Collection: During game testing, a large number of real game screenshots and their corresponding engine UI tree information are saved to obtain the position information of text in the screen, including the top-left corner (x), top-left corner (y), text width (w), and text height (h). Using local images near the game text as unit images, normal text images and super-framed text images are distinguished and collected for model training.

[0064] 2. Generating Fake Text Over-Boundary Images: In real game scene images, the number of over-boundary text images is extremely small compared to the number of normal text images. This extreme imbalance in data is detrimental to the training of deep learning models and has a significant impact on the model's generalization performance. Therefore, a semi-automatic method is needed to generate effective over-boundary text sample images within real game image data. Based on real game image data and text position information, within a general text area, the text position and size are calibrated using OCR (Optical Character Recognition). Utilizing the font file and font size used in the game, strings with random language, length, content, and color are added to the top, bottom, left, and right positions of the text, automatically generating a large amount of fake data. Subsequently, valid over-boundary text image data is manually selected from the large amount of fake data to supplement the real over-boundary text image data, expanding the dataset and making the number of normal text images as balanced as possible with the number of over-boundary text images.

[0065] 3. Training the Hyperbox Detection Model: A deep network (such as ResNet18) is used as the backbone, connecting multiple fully connected layers to construct a binary classification network model for hyperbox detection in text images. Normal text image data from real game scenarios are positive samples (labeled 1), while hyperbox text image data from real game scenarios and artificially created valid hyperbox text image data are negative samples (labeled 0). The network gradient is calculated using the binary cross-entropy loss function, and the network weights are updated using the Adam optimizer. Even with the addition of some artificially created negative hyperbox samples, the imbalance between positive and negative samples still exists. Therefore, when sampling training samples, it is necessary to randomly select the same number of positive samples as negative samples for network training to improve the network's generalization performance.

[0066] 4. Inference for the Text Over-Boundary Detection Model: In practical production applications, automated text over-boundary detection based on images is performed. A local image of the text area obtained from the engine is used as input, and a trained deep learning model is employed to determine whether each piece of text has an over-boundary issue. Since game screenshots may have unexpected issues such as latency or occlusion, before performing text over-boundary detection, an OCR model is used to determine if text exists within the text area given by the engine. If no text exists, no detection is needed; if text exists, the over-boundary detection model is then used for judgment. This filters out unexpected situations and reduces false positives.

[0067] The above method can be divided into a training phase and a testing phase. The training phase and the testing phase will be introduced below.

[0068] The flowchart of the training phase is as follows: Figure 2 As shown, this is achieved through the following steps:

[0069] (1) Collect real game scene images and corresponding text information: During game testing, save a large number of game scene screenshots, and simultaneously extract the position information of all text in each screenshot from the game engine, including the coordinates (x, y) of the top-left corner of the text, the text width w, and the text height h. An example of information annotation is shown below. Figure 3 As shown, the gray boxes mark the specific location and range of the text in the image based on the text information obtained from the game engine.

[0070] (2) Labeling whether local images of real text exceed the frame: The collected images within the game scene are distinguished, and each text box is used as a unit to distinguish whether each piece of text exceeds the frame. Normal text images are labeled as 1, and text images exceeding the frame are labeled as 0. For example... Figure 4 As shown, four candidate text addition areas are provided around the text "purchase", indicated by dashed boxes and solid boxes indicating the text box range corresponding to the text.

[0071] (3) Semi-automatic generation of artificial text bounding box image data: Since the number of text bounding boxes in real game scenes is extremely small compared to the number of normal text, such extremely imbalanced data is not conducive to the training and fitting of data features by deep networks. Therefore, data augmentation is required. Based on the game language, font, font size, and real game images, strings of random length, random content, and random color are generated in multiple candidate regions around the original game text. Valid bounding box text sample images are selected from these to expand the negative sample dataset. Because the text position obtained from the game engine may be inaccurate, an OCR model is needed to accurately locate the text position, generating text in the candidate regions at the top, bottom, left, and right of the game text.

[0072] (4) Training of the text bounding box detection binary classification model: The text bounding box detection problem in the game is regarded as a deep network image binary classification problem. To minimize the loss of image information, the text range provided by the game engine is cropped from the game scene image. The cropped image is extended 20 pixels to the left and right and 10 pixels to the top and bottom based on the engine text position coordinates. The size of all network input images is uniformly adjusted to 256 pixels wide and 64 pixels high.

[0073] The binary classification convolutional neural network model uses ResNet18 as the backbone network, connecting a flat1 fully connected layer with 128 neurons, a flat2 fully connected layer with 32 neurons, and a flat3 fully connected layer with 2 neurons, all connected to a softmax layer as the output layer. The output is the probability of classifying the image as 0 or 1, i.e., the probability of classifying the image as an over-bounds sample or a normal sample. The network structure is as follows: Figure 5 As shown in Table 1, the structural parameters of the output of this model are as follows.

[0074] Table 1

[0075]

[0076] The network training uses the Adam optimizer, with the binary cross-entropy function as the loss function, as shown in the following equation:

[0077]

[0078] In the above formula, represents the label of sample i, with 1 for positive class and 0 for negative class, and represents the probability that sample i is predicted to be positive.

[0079] Although some artificial hyperframe samples were added for training, the number of hyperframe text images is still relatively small compared to the number of normal sample images. Therefore, when sampling samples for training the model, the same number of positive and negative sample images need to be extracted for each training session to avoid the model being affected by a large number of positive samples, reduce overfitting, and enhance the model's generalization ability.

[0080] After training the model, the testing phase begins, such as... Figure 6 As shown, this stage is achieved through the following steps:

[0081] (1) Acquisition of game scene images and corresponding text information to be tested: During production testing, game screenshots and image text information obtained by the engine need to be collected. This process is similar to the sample acquisition process during training, and will not be described in detail here.

[0082] (2) OCR model determines whether text exists in the text region: Due to pop-up occlusion, text information or screen asynchrony, or overlapping of game UI layers, the text information obtained by the engine will select some areas in the screen where there is no text. This part of the sample can use the OCR model to determine whether there is text in the region. If there is no text, it is considered dirty data of engine text information and is directly filtered without further detection; if there is text, it is considered valid text image data and then the super-frame detection model is used for subsequent judgment.

[0083] (7) Text Over-Boundary Detection Binary Classification Model Inference: For valid text in an image, a local image of the text location is extracted as input to the network model. The output is a value between 0 and 1, representing the probability that the text image is a normal sample (i.e., the text does not have an over-boundary problem). A threshold of 0.5 is used. If the output probability is greater than 0.5, the text is considered to have no over-boundary problem; if the output probability is less than or equal to 0.5, the text is considered to have an over-boundary problem. The image is provided, and the over-boundary text locations are marked for manual review. If all text in an image is judged as normal text, the automated test is passed, and no manual review is required. The over-boundary detection result of a complete image is as follows: Figure 7 As shown, the marked values ​​represent the probability that the model determines this text to be a normal sample.

[0084] The aforementioned method, based on game scene images, is used to detect text exceeding the bounding box issue in game scenes. It utilizes real game image data and data augmentation methods to train the model, which boasts high recall and low false positives. It quickly, efficiently, and accurately determines whether text exceeding the bounding box exists within the test image, performing the judgment image-by-image and text-by-text. This significantly improves the speed of image review and text exceeding the bounding box issue, enhancing tester efficiency, reducing time and manpower costs, and minimizing the possibility of missed issues. It is suitable for automated testing of large-scale game runs.

[0085] For the above method embodiments, see Figure 8 The embodiment of the present invention shown provides a text overlay detection device, which includes:

[0086] The target detection region determination module 802 is used to acquire the interface image of the target game interface and determine the target detection region in the interface image based on the text box information of the target game interface.

[0087] The model processing module 804 is used to perform image processing on the target detection region using a pre-trained text hyperframe detection model to obtain the processing result;

[0088] The text over-bound determination module 806 is used to determine, based on the processing results, whether the text content displayed in the target game interface exceeds the corresponding text box.

[0089] The aforementioned text over-bounding detection device acquires an interface image of a target game interface, determines the target detection region in the interface image based on the text box information of the target game interface, performs image processing on the target detection region using a pre-trained text over-bounding detection model, and obtains the processing result. Based on the processing result, it determines whether the text content displayed in the target game interface exceeds the corresponding text box. In this method, the text over-bounding detection model based on a deep learning network performs image processing on the detection region corresponding to the text box in the interface image, thereby determining whether the target game interface has experienced text over-bounding, thus improving the accuracy of text over-bounding detection.

[0090] The aforementioned device further includes: an initial model building module for building an initial model based on a preset deep learning network; a training data determination module for determining training data from a preset sample set; the training data includes local interface images and annotation parameters of the local interface images; the local interface images display the text content corresponding to the text boxes set in the test game interface; the annotation parameters are used to indicate whether the text content displayed in the local interface images exceeds the corresponding text boxes; a training data processing module for inputting the local interface images into the initial model, performing image processing on the local interface images through the initial model, and obtaining processing results; the processing results include probability parameters of the text content displayed in the local interface images exceeding the corresponding text boxes; a loss value calculation module for calculating the loss value of the initial model based on a preset loss function, the annotation parameters of the local interface images, and the processing results; a network parameter adjustment module for adjusting the network parameters of the initial model based on the loss value; and continuing to execute the step of determining training data from the preset sample set until the loss value converges, thus determining the trained initial model as a text hyperbox detection model.

[0091] The aforementioned sample set includes a positive sample set and a negative sample set; the annotation parameters of multiple local interface images in the positive sample set indicate that the text content does not exceed the corresponding text box; the annotation parameters of multiple local interface images in the negative sample set indicate that the text content exceeds the corresponding text box; the training data determination module is further used to: select a first number of local interface images from the preset positive sample set; select a second number of local interface images from the preset negative sample set; the first number matches the second number; and determine the selected multiple local interface images and the annotation parameters of the local interface images as training data.

[0092] The aforementioned device further includes: a random text generation module, used to generate random text content based on the display format of the text content displayed in the local interface image for a local interface image in the positive sample set; a text addition module, used to add random text content at a specified position of the text content displayed in the local interface image to obtain updated text content; an over-frame judgment module, used to determine whether the updated text content displayed in the local interface image exceeds the corresponding text box; and a negative sample set determination module, used to determine the local interface image as a local interface image in the negative sample set if it exceeds the text box.

[0093] The aforementioned text box information includes the position and size parameters of the text box in the target game interface; the target detection region determination module is also used to: determine the image region in the interface image corresponding to the text box based on the position and size parameters of the text box set in the target game interface; detect whether there is text content displayed in the image region corresponding to the text box; if so, determine the image region corresponding to the text box as the target detection region.

[0094] The aforementioned model processing module is also used for: taking a screenshot of the interface image based on the target detection region to obtain a local interface image; the local interface image includes the target detection region; performing size transformation processing on the local interface image based on preset detection size parameters; and inputting the processed local interface image into a pre-trained text super-frame detection model to perform image processing on the local interface image through the text super-frame detection model.

[0095] The above processing results include the probability parameter of the text content displayed in the target detection area exceeding the corresponding text box; the text exceeding the box determination module is also used to: determine whether the probability parameter of the text content displayed in the target detection area exceeding the corresponding text box is greater than a preset probability threshold; if yes, determine that the text content displayed in the target game interface exceeds the corresponding text box; if no, determine that the text content displayed in the target game interface does not exceed the corresponding text box.

[0096] This embodiment also provides an electronic device, including a processor and a memory. The memory stores machine-executable instructions that can be executed by the processor. The processor executes the machine-executable instructions to implement the above-described text super-frame detection method. Specifically:

[0097] The process involves acquiring an image of the target game interface, determining the target detection region based on the text box information of the target game interface, performing image processing on the target detection region using a pre-trained text overbox detection model, and obtaining the processing result. Based on the processing result, it is determined whether the text content displayed in the target game interface exceeds the corresponding text box.

[0098] In the above method, a text over-frame detection model based on a deep learning network is used to process the detection area corresponding to the text box in the interface image, thereby determining whether the target game interface has experienced text over-frame phenomenon, thus improving the detection accuracy of text over-frame.

[0099] Optionally, the above text hyperframe detection model is trained in the following way: An initial model is established based on a preset deep learning network; training data is determined from a preset sample set; the training data includes local interface images and their annotation parameters; the local interface images display the text content corresponding to the text boxes set in the test game interface; the annotation parameters are used to indicate whether the text content displayed in the local interface images exceeds the corresponding text boxes; the local interface images are input into the initial model, and the initial model performs image processing on the local interface images to obtain processing results; the processing results include probability parameters of the text content displayed in the local interface images exceeding the corresponding text boxes; the loss value of the initial model is calculated based on a preset loss function, the annotation parameters of the local interface images, and the processing results; the network parameters of the initial model are adjusted based on the loss value; the step of determining training data from the preset sample set is continued until the loss value converges, and the trained initial model is determined as the text hyperframe detection model.

[0100] Optionally, the aforementioned sample set includes a positive sample set and a negative sample set; the annotation parameters of multiple local interface images in the positive sample set indicate that the text content does not exceed the corresponding text box; the annotation parameters of multiple local interface images in the negative sample set indicate that the text content exceeds the corresponding text box; the step of determining training data from the preset sample set includes: selecting a first number of local interface images from the preset positive sample set; selecting a second number of local interface images from the preset negative sample set; matching the first number with the second number; and determining the selected multiple local interface images and the annotation parameters of the local interface images as training data.

[0101] Optionally, the negative sample set is generated as follows: for the local interface images in the positive sample set, random text content is generated based on the display format of the text content displayed in the local interface images; random text content is added to a specified position of the text content displayed in the local interface images to obtain updated text content; it is determined whether the updated text content displayed in the local interface images exceeds the corresponding text box; if it exceeds the text box, the local interface image is determined as a local interface image in the negative sample set.

[0102] Optionally, the aforementioned text box information includes the position and size parameters of the text box in the target game interface; the step of determining the target detection region in the interface image based on the text box information of the target game interface includes: determining the image region corresponding to the text box in the interface image based on the position and size parameters of the text box set in the target game interface; detecting whether text content is displayed in the image region corresponding to the text box; if so, determining the image region corresponding to the text box as the target detection region.

[0103] Optionally, the above steps of performing image processing on the target detection region using a pre-trained text hyperframe detection model include: taking a screenshot of the interface image based on the target detection region to obtain a local interface image; the local interface image includes the target detection region; performing size transformation processing on the local interface image based on preset detection size parameters; and inputting the processed local interface image into the pre-trained text hyperframe detection model to perform image processing on the local interface image through the text hyperframe detection model.

[0104] Optionally, the above processing result includes a probability parameter that the text content displayed in the target detection area exceeds the corresponding text box; based on the processing result, the step of determining whether the text content displayed in the target game interface exceeds the corresponding text box includes: determining whether the probability parameter that the text content displayed in the target detection area exceeds the corresponding text box is greater than a preset probability threshold; if yes, determining that the text content displayed in the target game interface exceeds the corresponding text box; if no, determining that the text content displayed in the target game interface does not exceed the corresponding text box.

[0105] See Figure 9 As shown, the electronic device includes a processor 100 and a memory 101. The memory 101 stores machine-executable instructions that can be executed by the processor 100. The processor 100 executes the machine-executable instructions to implement the above-described text overframe detection method.

[0106] Furthermore, Figure 9 The electronic device shown also includes a bus 102 and a communication interface 103, with the processor 100, the communication interface 103 and the memory 101 connected via the bus 102.

[0107] The memory 101 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 103 (which can be wired or wireless), such as the Internet, wide area network, local area network, or metropolitan area network. The bus 102 may be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 9 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0108] Processor 100 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of processor 100 or by instructions in software form. Processor 100 may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this invention can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software module can reside in a readily available storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory 101, and the processor 100 reads the information from memory 101 and, in conjunction with its hardware, completes the steps of the method described in the foregoing embodiments.

[0109] This embodiment also provides a machine-readable storage medium storing machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions cause the processor to implement the above-described text over-frame detection method.

[0110] The present invention provides a text hyperframe detection method, apparatus, and electronic device, including a computer-readable storage medium storing program code. The program code includes instructions that can be used to execute the methods described in the preceding method embodiments. Specifically:

[0111] The process involves acquiring an image of the target game interface, determining the target detection region based on the text box information of the target game interface, performing image processing on the target detection region using a pre-trained text overbox detection model, and obtaining the processing result. Based on the processing result, it is determined whether the text content displayed in the target game interface exceeds the corresponding text box.

[0112] In the above method, a text over-frame detection model based on a deep learning network is used to process the detection area corresponding to the text box in the interface image, thereby determining whether the target game interface has experienced text over-frame phenomenon, thus improving the detection accuracy of text over-frame.

[0113] Optionally, the above text hyperframe detection model is trained in the following way: An initial model is established based on a preset deep learning network; training data is determined from a preset sample set; the training data includes local interface images and their annotation parameters; the local interface images display the text content corresponding to the text boxes set in the test game interface; the annotation parameters are used to indicate whether the text content displayed in the local interface images exceeds the corresponding text boxes; the local interface images are input into the initial model, and the initial model performs image processing on the local interface images to obtain processing results; the processing results include probability parameters of the text content displayed in the local interface images exceeding the corresponding text boxes; the loss value of the initial model is calculated based on a preset loss function, the annotation parameters of the local interface images, and the processing results; the network parameters of the initial model are adjusted based on the loss value; the step of determining training data from the preset sample set is continued until the loss value converges, and the trained initial model is determined as the text hyperframe detection model.

[0114] Optionally, the aforementioned sample set includes a positive sample set and a negative sample set; the annotation parameters of multiple local interface images in the positive sample set indicate that the text content does not exceed the corresponding text box; the annotation parameters of multiple local interface images in the negative sample set indicate that the text content exceeds the corresponding text box; the step of determining training data from the preset sample set includes: selecting a first number of local interface images from the preset positive sample set; selecting a second number of local interface images from the preset negative sample set; matching the first number with the second number; and determining the selected multiple local interface images and the annotation parameters of the local interface images as training data.

[0115] Optionally, the negative sample set is generated as follows: for the local interface images in the positive sample set, random text content is generated based on the display format of the text content displayed in the local interface images; random text content is added to a specified position of the text content displayed in the local interface images to obtain updated text content; it is determined whether the updated text content displayed in the local interface images exceeds the corresponding text box; if it exceeds the text box, the local interface image is determined as a local interface image in the negative sample set.

[0116] Optionally, the aforementioned text box information includes the position and size parameters of the text box in the target game interface; the step of determining the target detection region in the interface image based on the text box information of the target game interface includes: determining the image region corresponding to the text box in the interface image based on the position and size parameters of the text box set in the target game interface; detecting whether text content is displayed in the image region corresponding to the text box; if so, determining the image region corresponding to the text box as the target detection region.

[0117] Optionally, the above steps of performing image processing on the target detection region using a pre-trained text hyperframe detection model include: taking a screenshot of the interface image based on the target detection region to obtain a local interface image; the local interface image includes the target detection region; performing size transformation processing on the local interface image based on preset detection size parameters; and inputting the processed local interface image into the pre-trained text hyperframe detection model to perform image processing on the local interface image through the text hyperframe detection model.

[0118] Optionally, the above processing result includes a probability parameter that the text content displayed in the target detection area exceeds the corresponding text box; based on the processing result, the step of determining whether the text content displayed in the target game interface exceeds the corresponding text box includes: determining whether the probability parameter that the text content displayed in the target detection area exceeds the corresponding text box is greater than a preset probability threshold; if yes, determining that the text content displayed in the target game interface exceeds the corresponding text box; if no, determining that the text content displayed in the target game interface does not exceed the corresponding text box.

[0119] See Figure 9 As shown, the electronic device includes a processor 100 and a memory 101. The memory 101 stores machine-executable instructions that can be executed by the processor 100. The processor 100 executes the machine-executable instructions to implement the above-described text overframe detection method.

[0120] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system and apparatus described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0121] Furthermore, in the description of the embodiments of the present invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in the present invention based on the specific circumstances.

[0122] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0123] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0124] Finally, it should be noted that the above embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for detecting text exceeding the bounding box, characterized in that, include: Acquire an interface image of the target game interface, and determine the target detection area in the interface image based on the text box information of the target game interface; The target detection region is processed using a pre-trained text hyperframe detection model to obtain the processing result; Based on the processing result, determine whether the text content displayed in the target game interface exceeds the corresponding text box; The text hyperframe detection model is trained in the following way: An initial model is built based on a pre-defined deep learning network; Training data is determined from a preset sample set; the training data includes local interface images and annotation parameters of the local interface images; the local interface images display the text content corresponding to the text boxes set in the test game interface; the annotation parameters are used to indicate whether the text content displayed by the local interface images exceeds the corresponding text boxes. The local interface image is input into the initial model, and the initial model performs image processing on the local interface image to obtain a processing result; the processing result includes a probability parameter that the text content displayed in the local interface image exceeds the corresponding text box; Based on the preset loss function, the annotation parameters of the local interface image, and the processing results, the loss value of the initial model is calculated. Based on the loss value, the network parameters of the initial model are adjusted; the step of determining training data from a preset sample set continues until the loss value converges, and the trained initial model is determined as a text hyperframe detection model.

2. The method according to claim 1, characterized in that, The sample set includes a positive sample set and a negative sample set; the annotation parameters of multiple local interface images in the positive sample set indicate that the text content does not exceed the corresponding text box; the annotation parameters of multiple local interface images in the negative sample set indicate that the text content exceeds the corresponding text box. The steps for determining training data from a pre-defined sample set include: Select a first number of local interface images from a preset set of positive samples; A second number of local interface images are selected from a preset set of negative samples; the first number matches the second number. The selected local interface images and their annotation parameters are used as training data.

3. The method according to claim 2, characterized in that, The negative sample set is generated in the following way: For a local interface image in the positive sample set, random text content is generated based on the display format of the text content displayed in the local interface image; Add the random text content to a specified position in the text content displayed in the local interface image to obtain the updated text content; Determine whether the updated text content displayed in the local interface image exceeds the corresponding text box; If the number of cases exceeds the limit, the local interface image will be identified as a local interface image in the negative sample set.

4. The method according to claim 1, characterized in that, The text box information includes the position and size parameters of the text box in the target game interface; The step of determining the target detection region in the interface image based on the text box information of the target game interface includes: Based on the position and size parameters of the text box set in the target game interface, determine the image area in the interface image corresponding to the text box; Detect whether text content is displayed in the image area corresponding to the text box; If so, the image region corresponding to the text box is determined as the target detection region.

5. The method according to claim 1, characterized in that, The steps of image processing of the target detection region using a pre-trained text hyperframe detection model include: Based on the target detection region, the interface image is cropped to obtain a partial interface image; the partial interface image includes the target detection region. Based on preset detection size parameters, the local interface image is subjected to size transformation processing; The processed local interface image is input into a pre-trained text hyperframe detection model, and the local interface image is processed by the text hyperframe detection model.

6. The method according to claim 1, characterized in that, The processing result includes a probability parameter that the text content displayed in the target detection area exceeds the corresponding text box; Based on the processing result, the step of determining whether the text content displayed in the target game interface exceeds the corresponding text box includes: Determine whether the probability parameter of the text content displayed in the target detection area exceeding the corresponding text box is greater than a preset probability threshold; If so, determine that the text content displayed on the target game interface exceeds the corresponding text box; If not, it is determined that the text content displayed on the target game interface does not exceed the corresponding text box.

7. A text over-frame detection device, characterized in that, include: The target detection region determination module is used to acquire the interface image of the target game interface and determine the target detection region in the interface image based on the text box information of the target game interface. The model processing module is used to perform image processing on the target detection region using a pre-trained text hyperframe detection model to obtain the processing result; The text over-bound determination module is used to determine, based on the processing result, whether the text content displayed in the target game interface exceeds the corresponding text box; The device further includes: The initial model building module is used to build an initial model based on a pre-defined deep learning network. The training data determination module is used to determine training data from a preset sample set; the training data includes local interface images and annotation parameters of the local interface images; the local interface images display the text content corresponding to the text boxes set in the test game interface; the annotation parameters are used to indicate whether the text content displayed in the local interface images exceeds the corresponding text boxes. The training data processing module is used to input the local interface image into the initial model, and perform image processing on the local interface image through the initial model to obtain the processing result; the processing result includes the probability parameter that the text content displayed in the local interface image exceeds the corresponding text box; The loss value calculation module is used to calculate the loss value of the initial model based on a preset loss function, the annotation parameters of the local interface image, and the processing results. The network parameter adjustment module is used to adjust the network parameters of the initial model based on the loss value; continue to execute the step of determining training data from a preset sample set until the loss value converges, and determine the trained initial model as a text hyperbox detection model.

8. An electronic device, characterized in that, The method includes a processor and a memory, the memory storing machine-executable instructions that can be executed by the processor, the processor executing the machine-executable instructions to implement the text superframe detection method according to any one of claims 1-6.

9. A machine-readable storage medium, characterized in that, The machine-readable storage medium stores machine-executable instructions, which, when invoked and executed by a processor, cause the processor to implement the text superframe detection method according to any one of claims 1-6.

Images