Image-based Terminal Interface Recognition Method, Device, Equipment, and Medium

Abstract

The present application provides an image-based terminal interface recognition method, device, equipment and medium. Among them, the method includes: using at least one replacement term to replace at least one original term included in the original terminal interface respectively to obtain a replaced replacement terminal interface; acquiring a screenshot image of the replacement terminal interface; performing recognition on the screenshot image to obtain at least one target term; for each of the target terms, acquiring the replacement term corresponding to the target term; in the case where the replacement term and the target term do not match, determining the target term as an abnormally displayed term; and marking the abnormally displayed term on the replacement terminal interface, effectively improving the efficiency of terminal interface recognition.

Description

Technical Field

[0001] The present invention relates to the technical field of application program recognition, and in particular, to a method, device, equipment, and medium for recognizing a terminal interface based on an image. Background Art

[0002] When an App (Application) is released overseas, it needs to support multiple languages. Therefore, it is necessary to detect the language performance of the application program App. However, in the related art, after the language of the application program App is switched, it is necessary for testers to manually browse and identify, which not only takes time and effort, but also seriously affects the delivery time. Summary of the Invention

[0003] In view of the above-mentioned defects or deficiencies in the prior art, it is desirable to provide a method, device, equipment, and medium for recognizing a terminal interface based on an image, which can effectively improve the efficiency of recognizing the terminal interface.

[0004] On the one hand, an embodiment of the present application provides a method for recognizing a terminal interface based on an image, the method including:

[0005] Using at least one replacement entry to respectively replace at least one original entry included in the original terminal interface to obtain a replaced replacement terminal interface;

[0006] Obtaining a screenshot image of the replacement terminal interface;

[0007] Performing recognition on the screenshot image to obtain at least one target entry;

[0008] For each of the target entries, obtaining the replacement entry corresponding to the target entry;

[0009] In the case where the replacement entry and the target entry do not match, determining the target entry as an abnormally displayed entry;

[0010] Marking the abnormally displayed entry on the replacement terminal interface.

[0011] On the one hand, an embodiment of the present application provides a device for recognizing a terminal interface based on an image, the device including:

[0012] A replacement module, configured to use at least one replacement entry to respectively replace at least one original entry included in the original terminal interface to obtain a replaced replacement terminal interface;

[0013] A first acquisition module, configured to obtain a screenshot image of the replacement terminal interface;

[0014] A recognition module, configured to perform recognition on the screenshot image to obtain at least one target entry;

[0015] A second acquisition module, configured to acquire, for each of the target entries, a replacement entry corresponding to the target entry;

[0016] A determination module, configured to determine the target entry as an abnormally displayed entry when the replacement entry does not match the target entry;

[0017] A reminder module, configured to mark the abnormally displayed entry on the replacement terminal interface.

[0018] On the one hand, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the method described in the embodiment of the present application is implemented.

[0019] On the one hand, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the method described in the embodiment of the present application is implemented.

[0020] The method, device, equipment, and medium for image-based terminal interface recognition provided by the embodiment of the present application perform entry replacement on the original terminal interface, then take a screenshot of the replaced replacement terminal interface, obtain the target entry in the screenshot through screenshot recognition, and identify the abnormally displayed entry for the target entry. Thus, the present application can convert the manual recognition technology for the terminal interface into automated graphic and text recognition by using graphic and text recognition technology, effectively improving the recognition efficiency of character display anomalies on the application program (App) terminal interface. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and / or additional aspects and advantages of the present invention will become apparent and easy to understand from the following description of the embodiments in conjunction with the drawings, where:

[0022] Figure 1 Shows an implementation environment architecture diagram of the image-based terminal interface recognition method provided by the embodiment of the present application;

[0023] Figure 2 Shows a flowchart of the image-based terminal interface recognition method provided by an embodiment of the present application;

[0024] Figure 3 Shows a flowchart of the image-based terminal interface recognition method provided by another embodiment of the present application;

[0025] Figure 4 Shows a schematic diagram of the original terminal interface provided by the embodiment of the present application;

[0026] Figure 5 Shows the embodiment of the present application forFigure 4 Schematic diagram of the replacement terminal interface after replacement;

[0027] Figure 6 Schematic flow chart showing a method for identifying a terminal interface based on an image according to another embodiment of the present application;

[0028] Figure 7 Schematic flow chart showing a method for identifying a terminal interface based on an image according to still another embodiment of the present application;

[0029] Figure 8 Schematic diagram showing a reminder message according to an embodiment of the present application;

[0030] Figure 9 Exemplary structural block diagram showing a device for identifying a terminal interface based on an image according to an embodiment of the present application;

[0031] Figure 10 Schematic diagram showing the structure of a computer system of an electronic device or a server suitable for implementing an embodiment of the present application. Detailed implementation manners

[0032] Embodiments of the present invention will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, where like or similar reference numerals denote like or similar elements or elements having the same or similar functions throughout. The embodiments described by referring to the accompanying drawings are exemplary and are intended to explain the present invention and should not be construed as limiting the present invention.

[0033] For the specific implementation environment of the terminal interface identification method proposed in the present application, please refer to Figure 1 . Figure 1 Schematic diagram of the implementation environment architecture of a method for identifying a terminal interface based on an image according to an embodiment of the present application.

[0034] As Figure 1 shown, the implementation environment architecture includes: a terminal device 101 and a server 102.

[0035] The terminal device 101 is used to display the original terminal interface, the replacement terminal interface, the reminder message, etc. to the tester. The terminal device can be a desktop computer, a laptop computer, a smart phone, a tablet computer, an e-book reader, smart glasses, a smart watch, etc., but is not limited thereto.

[0036] The server 102 can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Among them, the server 102 is used to provide replacement entries to the terminal device 101 and execute the recognition strategy for target entry anomalies.

[0037] The terminal device 101 is directly or indirectly connected to the server 102 through a wired or wireless communication method. Optionally, the above-mentioned wireless network or wired network uses standard communication technologies and / or protocols. The network is usually the Internet, or can be any network, including but not limited to any combination of a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a private network or a virtual private network.

[0038] The image-based terminal interface recognition method proposed in this application can be implemented by an image-based terminal interface recognition device, and the image-based terminal interface recognition device can be installed on the terminal device or the server.

[0039] To further illustrate the technical solutions provided in the embodiments of the present application, the following will be described in detail in combination with the accompanying drawings and specific implementation manners. Although the embodiments of the present application provide the method operation instruction steps shown in the following embodiments or drawings, more or fewer operation instruction steps may be included in the method based on routine or non-creative labor. In the steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided in the embodiments of the present application. When the method is actually processed or executed by the device, it can be executed in the method order shown in the embodiments or drawings or executed in parallel.

[0040] Please refer to Figure 2 , Figure 2 which shows a schematic flowchart of an image-based terminal interface recognition method provided in an embodiment of the present application. As Figure 2 shown, the method includes:

[0041] Step 201, using at least one replacement entry to replace at least one original entry included in the original terminal interface respectively, to obtain a replaced replacement terminal interface.

[0042] Among them, the replacement terminal interface is used to display at least one replacement entry. That is, after the original terminal interface of the application program is replaced by the replacement entry, the replaced replacement terminal interface is formed to display the replacement entry, so as to facilitate the recognition of character display anomalies based on the entries displayed in the replaced replacement terminal interface.

[0043] Among them, the original terminal interface can be the terminal interface to be subjected to language replacement. Among them, the application program App can complete function tests in the language type to which the original terminal interface belongs. For example, the original terminal interface can be a terminal interface marked in Chinese or an English-marked terminal interface. Developers conduct function tests on the application program App under Chinese labels or English labels, and each control can execute its due functions. For the overseas release of the application program App, it is only that language replacement has not been carried out, so that the application program App can meet the language needs of users in the target country or region.

[0044] It should also be noted that there can be multiple original terminal interfaces, that is, as the controls in the application program App are called, the displayed terminal interface will change. When performing language replacement, it is necessary to perform entry replacement and abnormal display entry recognition on each interface in the application program App, so as to effectively ensure the reliability of the application program App.

[0045] It should be understood that the original terminal interface contains at least one original entry, that is, the original terminal interface contains at least one piece of text information. When performing language conversion detection, it is necessary to perform conversion recognition on all the text information in the application program App.

[0046] Furthermore, after obtaining the original terminal interface, at least one replacement entry corresponding to at least one original entry in the original terminal interface can also be obtained according to the current original terminal interface. Among them, the replacement entry can be obtained according to each original entry in the original terminal interface. It should also be noted that the replacement entry is not the entry obtained by translating the original entry into the target language, but a machine-readable string obtained by encoding the entry. Using the replacement entry to replace the original entry can quickly and accurately reverse-check the specific information of the abnormal display entry when the target entry is an abnormal display entry, thereby effectively reducing the translation and proofreading time in the process of character display anomaly recognition and improving the efficiency of character display anomaly function recognition.

[0047] In some implementation manners, the original entries in the code file, configuration file or resource file can be replaced to generate the replaced replacement terminal interface.

[0048] Step 202, obtain a screenshot image of the replacement terminal interface.

[0049] That is to say, after obtaining the replacement entries, at least one original entry is replaced with at least one replacement entry to obtain a replaced replacement terminal interface, and then the replaced replacement terminal interface is captured, so that the detection of the terminal interface can be converted into image recognition.

[0050] Among them, each replaced replacement terminal interface can be captured through the built-in screenshot function in the terminal device to obtain a screenshot image, or a third-party program can be used to take a picture of the replaced replacement terminal interface to obtain a screenshot image.

[0051] Step 203: Identify at least one target entry for the screenshot image.

[0052] Optionally, at least one target entry included in the screenshot can be obtained by using the text and image recognition technology, so as to effectively convert the manual reading detection of the application App interface into the text and image recognition and character recognition using a computer device, effectively improving the time for recognizing the application App interface and improving the recognition efficiency. Specifically, the screenshot can be subjected to text and image recognition to obtain at least one target entry included in the screenshot. Among them, the text and image recognition is OCR (Optical Character Recognition) recognition, which refers to analyzing and recognizing an image file of text materials to obtain text information.

[0053] Step 204: For each target entry, obtain the replacement entry corresponding to the target entry.

[0054] That is to say, in order to detect the accuracy of the entries in the replaced replacement terminal interface, after at least one target entry is obtained by recognizing the screenshot image of the replaced replacement terminal interface, it is necessary to obtain the replacement entry corresponding to the target entry according to the target entry, so as to judge whether there is an abnormal character display when replacing the entries in the terminal interface by comparing the target entry and the replacement entry.

[0055] Step 205: When the replacement entry does not match the target entry, determine the target entry as an abnormally displayed entry.

[0056] Specifically, after the replacement entry corresponding to the target entry is obtained by using the target entry, the replacement entry and the target entry are subjected to matching recognition. Among them, an entry generally includes multiple characters. Therefore, in the embodiment of the present application, it can be determined whether the replacement entry and the target entry match by characters, and when the replacement entry and the target entry match, the target entry is determined as a normally displayed entry, and when the replacement entry and the target entry do not match, the target entry is determined as an abnormally displayed entry.

[0057] Step 206: Mark the abnormally displayed entry on the replaced replacement terminal interface.

[0058] Among them, the marked content may include at least one of the position of the abnormally displayed entry in the screenshot image, the error content displayed by the abnormally displayed entry, and the replacement entry corresponding to the abnormally displayed entry.

[0059] That is to say, after identifying the abnormally displayed entry on the replacement terminal interface, it is also necessary to mark the abnormally displayed entry on the replacement terminal interface, so that the abnormally displayed entry can be obtained by identifying the mark in the replacement terminal interface, thereby effectively improving the acquisition efficiency of developers for the abnormally displayed entry.

[0060] Specifically, this application uses at least one replacement entry to replace each of the at least one original entries included in the original terminal interface respectively, to obtain the replacement terminal interface after replacement, then takes a screenshot of the replacement terminal interface, obtains the screenshot image of the replacement terminal interface, identifies at least one target entry for the screenshot image, then obtains the replacement entry corresponding to each target entry, matches and identifies the replacement entry and the target entry, and when the replacement entry and the target entry do not match, determines the target entry as the abnormally displayed entry, and marks the abnormally displayed entry on the replacement terminal interface.

[0061] The image-based terminal interface recognition method, device, electronic device and storage medium provided by the embodiments of this application replace the entries of the original terminal interface, then take a screenshot of the replacement terminal interface after replacement, identify the abnormally displayed entries included in the screenshot image for the screenshot image, and mark the abnormally displayed entries on the replacement terminal interface. In this way, it can quickly and simply implement the comparison between the interface entries recognized by OCR and the original entries, and improve the detection speed of the display characters of the terminal interface. Therefore, this application can convert the manual recognition technology of the terminal interface into an automated graphic and text recognition and character recognition by using the graphic and text recognition technology, and effectively improve the recognition efficiency of the character display abnormality of the application App terminal interface.

[0062] As a feasible embodiment, as Figure 3 shown, the image-based terminal interface recognition method includes:

[0063] Step 301, perform signature encoding on each original entry to obtain a string result.

[0064] It should be noted that although the original entry can be in Chinese or English, since English is more easily applied when writing and converting program codes, therefore, when performing entry replacement, English can be preferentially selected as the original entry, and Chinese can be used as the original entry when performing function detection of controls, so as to facilitate engineers whose mother tongue is Chinese to perform function detection.

[0065] Among them, each original entry is signed and encoded to obtain a string result, which may include: generating a signature string according to the original entry, equally dividing the signature string according to a preset length to obtain multiple character arrays; processing each character array cyclically to obtain an index corresponding to the character array; splicing the multiple indexes to obtain a string result.

[0066] Specifically, for encoding according to the original entry, the MD5 Message-Digest Algorithm can be used to encode the original entry to obtain a keyword for compressing the entry corresponding to the original entry, which may include: generating a 32-bit signature string for the original entry using the MD5 algorithm, dividing the 32-bit signature string into 4 segments, with 8 characters in each segment, then processing the 4 segments of signature strings cyclically, taking 8 characters in each segment, regarding the 8 characters in each segment as a hexadecimal string and performing a bitwise AND operation with 0x3fffffff (30 1s), and then dividing the obtained 30-bit characters into 6 segments. Through a shift operation, perform a bitwise AND operation (0x0000003D) on each 5 bits with the character array respectively to obtain the index of the entry in the character array and take out the concatenated string. In this way, 4 six-bit strings can be obtained from one MD5 string, and the first six-bit string is taken as the string result.

[0067] Step 302: Obtain a keyword for compressing the entry corresponding to the original entry according to the string result and the target language identifier, where the target language identifier is used to indicate the target language after replacement.

[0068] It should be noted that the target language can be the language type to be replaced. For example, it can be multiple language types different from the language type of the original entry, such as Japanese, French, etc. The target language identifier is used to indicate the target language after replacement, that is, the language detection environment corresponding to the current replacement entry or target entry can be determined according to the target language identifier.

[0069] For example, the first letter of the English corresponding to the language type to be detected can be used as the target language identifier. For example, when the language type to be detected is Japanese, the target language identifier can be "j", and when the language type to be detected is French, the target language identifier can be "f". It should be understood that the target language identifier is only used to assist developers in identifying for which language the current display abnormal entry has a replacement exception during detection. Therefore, developers can also encode multiple target languages to be detected to obtain the corresponding target language identifier, that is, the setting of the target language identifier only needs to satisfy the developers' identification of the generation environment of the abnormal display entry.

[0070] Specifically, after obtaining the original entry, perform signature encoding on each original entry to obtain a string result, obtain at least one target language to be detected, and obtain the target language identifier corresponding to each target language. For each target language, concatenate the string result and the target language identifier to obtain a compressed keyword for the entry that conforms to the currently detected target language. When the target entry is determined to be an abnormally displayed entry, identify the target language identifier in the compressed keyword for the entry to determine the target language to which the target entry belongs, that is, the language detection environment in which the abnormally displayed entry is located, so that when performing multi-language recognition on the terminal interface, the language environment in which character display abnormalities occur can be accurately identified.

[0071] Step 303, generate a replacement entry corresponding to the compressed keyword for the entry according to the entry concatenation rule, where the entry concatenation rule makes the difference between the replacement entry and the length of the target character corresponding to the replacement entry less than a preset threshold, and the target character length is used to indicate the character length of the target language after replacement.

[0072] Among them, the preset threshold is the acceptable difference between the length of the replacement entry and the character length of the target language, such as 1 or 2.

[0073] That is to say, in order to make the length of the replacement entry more conform to the length in the actual application of the target language and improve the accuracy of entry recognition, it is necessary to construct the character length required for the target language to express the meaning of the original entry.

[0074] As a feasible embodiment, generating a replacement entry corresponding to the compressed keyword for the entry according to the entry concatenation rule includes: obtaining the target character length and locator corresponding to the target language; determining the redundant character length corresponding to the replacement entry according to the target character length and the compressed character length corresponding to the compressed keyword for the entry; if the redundant character length is less than or equal to the preset length, concatenate the compressed keyword for the entry and the locator to obtain the replacement entry; if the redundant character length is greater than the preset length, concatenate the compressed keyword for the entry, the locator, and the padding characters corresponding to the redundant character length to obtain the replacement entry.

[0075] Specifically, the length relationship between the target language and the language to which the original entry belongs can be obtained according to the current language type to be replaced, and the target character length of the target language can be determined according to the length relationship, that is, the actual entry length that will be generated if the original entry is replaced with the target language, so as to compensate for the length of the replacement entry and ensure that the replacement test can meet the length requirements in actual replacement. In some embodiments, the length of the original entry and the length of the replacement entry used to replace the original entry are kept the same, which can provide the highest test accuracy to the greatest extent.

[0076] Among them, the length relationship between the target language and the language to which the original entry belongs can be used to determine common phrases in a large number of target languages and the lengths of the corresponding original entries in the original language. Optionally, sample data of multiple original entries are obtained, the sample data of the original entries are translated into translation samples in the target language, and then the average length of the sample data of the original entries and the average length of the translation samples in the target language are obtained respectively. Based on the average length of the sample data of the original entries and the average length of the translation samples in the target language, the length relationship between the target language and the language to which the original entry belongs is further obtained.

[0077] Specifically, after determining the target character length, the target character length can be compared with the entry compression keyword to determine the redundant character length corresponding to the replacement entry. Among them, the redundant character length can be the difference between the target character length and the entry compression keyword, that is, subtracting the entry compression keyword length from the target character length can obtain the redundant character length. Then it is judged whether the target character length is greater than the preset length, where the preset length can be the minimum length that requires length compensation, such as 1 character. Therefore, if the target character length is greater than the preset length, it means that when performing entry replacement, only using the entry compression keyword for entry replacement cannot accurately express the placeholder situation of the target language, and it is easy to have problems with abnormal entries due to the excessive length of the target language when the actual product goes global. Therefore, it is necessary to generate padding characters corresponding to the redundant character length to pad the replacement entry, that is, to make the length of the obtained replacement entry basically match the target character length of the target language, that is, to splice the entry compression keyword, the locator and the padding characters corresponding to the redundant character length to obtain the replacement entry. If the target character length is less than or equal to the preset length, it means that the length of the entry compression keyword can already cover the length required by the target language, and there is no need to use the redundant character length for additional length compensation, and the entry compression keyword and the locator can be directly spliced to obtain the replacement entry.

[0078] Among them, the padding characters can be generated using English letters, that is, the number of English letters that meet the redundant character length is selected as the padding characters, and the English letters in the padding characters can be the same or different. Preferably, the content of the original entry that exceeds the entry compression keyword and the locator part can be used to generate the padding characters, so that when the entry compression keyword is abnormal during entry replacement or OCR recognition, developers can also infer the content of the abnormally displayed entry using the content of the padding characters.

[0079] Among them, the locator is a symbol used to determine the position of the compressed keyword of the entry during machine recognition. That is to say, since the compressed keyword of the entry and the padding character are usually expressed in English, when the two are concatenated and displayed, it is impossible to accurately identify the compressed keyword of the target entry in the entry, and thus it is impossible to accurately obtain the replacement entry corresponding to the target entry. Especially when the positions of multiple target entries are close and their lengths reach the maximum length of the display interface, it is very difficult to distinguish the number of target entries included in this area. Therefore, in this application, by setting the locator, during the recognition process, the position of the compressed keyword of the entry can be accurately obtained by recognizing the locator, so as to achieve the purpose of accurately identifying the keyword of the entry, and effectively avoid the possibility of misidentifying the padding character as the compressed keyword of the entry. Optionally, the position of the locator can be set according to the actual situation. In addition to setting the locator after the compressed keyword of the entry proposed in this application, it can also be set before the compressed keyword of the entry, etc.

[0080] Specifically, the replacement entry replace_value = the compressed keyword of the entry shortkey(7 digits)+value[7:]. Among them, "+" is the locator. When performing image recognition on the screenshot of the replacement terminal interface, the compressed keyword of the entry is found through the "+" in the entry, so as to determine the corresponding replacement entry replace_value according to the compressed keyword of the entry. Then, the target entry is recognized by matching the replacement entry replace_value and the target entry. It should be understood that value[7:] represents the content of the padding character corresponding to the length of the redundant character after the 7-digit compressed keyword shortkey of the entry.

[0081] For example, as Figure 4 and Figure 5 shown, among which, Figure 4When the language to which the original entry 401 belongs is English and the target language is also English, the original entry 401 for joining the meeting is "Join Meeting". By encrypting "Join Meeting", the compressed keyword 502 of the entry "yyee6re" is obtained. Then, "Join Meeting" including spaces is 12 characters. The compressed keyword of the entry shortkey (7 digits) plus the locator "+" is a total of 8 digits. Subtracting the length of the compressed keyword of the entry and the locator from the length of the original entry gives the redundant character length. Therefore, it is determined that the redundant character length corresponding to the replacement entry is 4, that is, 12 - 8. According to the original entry, the padding character 504 corresponding to the redundant character length is "ting". The compressed keyword 501 of the entry, the locator 503, and the padding character 504 are concatenated to obtain the replacement label 501 "yyee6re+ting". In the figure, "Sign Up / Log In" obtains the replacement label "v6BJF3e+Log In" after calculation. It should be understood that Figure 5 is a schematic representation of entry replacement, and no abnormal display entry appears.

[0082] It should be noted that since the last digit of the compressed keyword of the entry is occupied by the target language identifier, and the length of the redundant character length needs to be determined according to the target character length of the target language, when multiple language tests need to be performed in the same application App, entry replacement needs to be performed for each language. When performing entry replacement each time, the content and length of the replacement entry can vary dynamically according to the target language.

[0083] Optionally, for entries with high usage frequencies, such as "Settings", "Return", "Help", etc., the corresponding replacement entries can be stored in advance according to the entry attributes. After obtaining the original entry, the stored replacement entry can be directly retrieved according to the entry content, effectively reducing the computational workload of encoding the original entry by the terminal device and / or the server to obtain the replacement entry and improving the acquisition efficiency of the replacement entry.

[0084] Step 304, use at least one replacement entry to replace at least one original entry included in the original terminal interface respectively, and obtain the replaced replacement terminal interface.

[0085] Step 305, obtain a screenshot image of the replacement terminal interface.

[0086] Step 306, perform recognition on the screenshot image to obtain at least one target entry.

[0087] Step 307, for each target entry, obtain the replacement entry corresponding to the target entry.

[0088] Step 308, in the case where the replacement entry and the target entry do not match, determine the target entry as an abnormally displayed entry.

[0089] Step 309, mark the abnormally displayed entry on the replacement terminal interface.

[0090] Thus, the image-based terminal interface recognition method proposed in this application can generate different replacement entries for each language, which can meet the recognition under the terminal interfaces of multiple languages, effectively avoid the problem that the replacement is normal in a short language environment but abnormal in a long language environment for the terminal interface, and further improve the reliability of using this application for terminal interface recognition.

[0091] As an embodiment, as Figure 6 shown, the image-based terminal interface recognition method includes the following steps:

[0092] Step 601, use at least one replacement entry to replace each of the at least one original entry included in the original terminal interface respectively to obtain a replaced replacement terminal interface.

[0093] Step 602, obtain a screenshot image of the replacement terminal interface.

[0094] Step 603, perform recognition on the screenshot image to obtain at least one target entry.

[0095] Step 604, for each target entry, recognize the locator in the target entry.

[0096] Step 605, according to the locator, recognize the entry compression keyword in the target entry.

[0097] Based on the above embodiment, since the target entry is obtained by replacing with the replacement entry, therefore, the locator "+" of the replacement entry should be included in the target entry. After performing recognition on the screenshot image to obtain the target entry, the locator in each target word can be obtained. Then, according to the position of the locator, the 7 characters before the locator can be obtained to get the entry compression keyword corresponding to the target entry to which the locator belongs.

[0098] Step 606, according to the entry compression keyword, obtain the replacement entry corresponding to the target entry.

[0099] Generally speaking, no abnormality will occur to the entry compression keyword during entry replacement. Therefore, after obtaining the entry compression keyword, it can be matched with the entry compression keyword of the replacement entry obtained in the foregoing solution to obtain the replacement entry corresponding to the target entry.

[0100] It should be understood that if an abnormality occurs in the keyword compression of the replacement entry during entry replacement, the corresponding replacement entry cannot be obtained. At this time, it can be considered that the target entry cannot match the replacement entry, that is, there is no matching replacement entry for the target entry. Therefore, the target entry is abnormally displayed as this entry.

[0101] Step 607, in the case where the replacement entry and the target entry do not match, determine that the target entry is an abnormally displayed entry.

[0102] Among them, when it is determined that the target entry contains the first detection character, determine that the target entry is an abnormally displayed entry; when it is determined that the target entry does not contain the first detection character, detect whether the replacement entry contains the second detection character; when the replacement entry contains the second detection character, convert the replacement entry into an entry that conforms to the preset rule; and compare the entry that conforms to the preset rule with the target entry to obtain a comparison result. Based on the comparison result, determine that the replacement entry and the target entry do not match, and determine the target entry as an abnormally displayed entry; when the replacement entry does not contain the second detection character, determine that the target entry is an abnormally displayed entry.

[0103] Furthermore, when the comparison result indicates that the entry that conforms to the preset rule does not match the target entry, determine that the replacement entry and the target entry do not match; when the comparison result indicates that the entry that conforms to the preset rule matches the target entry, determine that the replacement entry and the target entry match.

[0104] Among them, the first detection character is a character displayed on the interface due to an abnormal reason. The first detection character can be a display character default generated by the system according to a specific abnormal reason during entry replacement. For example, when the length of the replacement entry is much greater than the length that can be displayed at the current position, the system will default to using an ellipsis to replace all the lengths that cannot be displayed. It should be understood that the first detection character includes but is not limited to an ellipsis.

[0105] In some embodiments, the characters on the interface may not be fixed and can be changed according to the dynamic information during the operation of the application. For example, the characters on the interface may display the account of the current user, the number of current users, and other dynamic information. The second detected character represents a character that displays different contents on the interface according to the actual application scenario. The second detected character can be a placeholder for the information input by the user or a placeholder for the number of users. The corresponding relationship between the placeholder and its corresponding content is the preset rule. By replacing the placeholder in the replacement entry with its corresponding content according to the preset rule, the replacement entry can be converted into an entry that conforms to the preset rule. For example, the user nickname is usually set by the user himself. In the main page of the application App, the personal settings interface, and other interfaces, there are usually entries that display the user nickname, such as "Hello, Andy", and the corresponding content in the code file for this entry is "Hello, %", where "%" is the placeholder for the user nickname. When displaying the entry according to the code file, it is necessary to first read the nickname data of the current user from the resource file according to the placeholder "%", and then splice the nickname data with the entry data "Hello," in the code file to get "Hello, Andy" and display it. Another example is that in a group environment, in addition to the group name (meeting name) in the title bar, it is usually necessary to display the number of people in the current group, and this number changes with the actual number of people who join. At this time, "#" can be used in the code file to occupy the position for the number of participants. When displaying the entry according to the code file, it is necessary to first obtain the information of the participants and count the actual number of participants, and then replace "#" in the code file with the actual number of participants. It should be understood that in this example, the corresponding relationship between "%" and the user nickname, and / or, the corresponding relationship between "#" and the number of participants is the preset relationship. It should also be understood that the second detected character includes but is not limited to placeholders.

[0106] As a feasible embodiment, when generating replacement entries according to the original entries, signature encoding is performed on each original entry to obtain a string result. According to the string result and the target language identifier, the compressed keyword of the entry corresponding to the original entry is obtained. The target character length corresponding to the target language is obtained. According to the target character length and the compressed character length corresponding to the compressed keyword of the entry, the initial redundant character length corresponding to the replacement entry is determined, where the initial redundant character length is the length of the entry corresponding to the original entry in the code file. Then, it is identified whether the code corresponding to the original entry in the code file contains a placeholder. If it does not contain a placeholder, the initial redundant character is used as the redundant character length, and it is further determined whether the redundant character length is greater than the preset length and the corresponding splicing operation of the replacement entry is performed. If it contains a placeholder, it is determined that the redundant character length is greater than the preset length. When the initial redundant character length is less than or equal to the preset length, the placeholder is used as the padding character for splicing. When the initial redundant character length is greater than the preset length, the placeholder is spliced into the padding character according to its position in the code corresponding to the original entry to form the final padding character. The compressed keyword of the entry, the locator, and the final padding character are spliced to obtain the replacement entry. Thus, when detecting the replacement and display of entries in this application, it can simulate the replacement scenario when the entries are actually used, enhance the accuracy of entry replacement recognition, and avoid the inability to recognize the abnormality generated during the replacement of the placeholder by the entry. That is to say, when the text information recorded in the replacement entry and the text information recorded in the target entry are consistent, it indicates that the application App will not generate an entry truncation abnormality during language replacement, and can well meet the requirement of changing the language and satisfy overseas requirements. However, when the replacement entry and the target entry do not match, there are multiple possibilities. Among them, the replacement entry generated by operation in a special scenario may include a second detection character, where the second detection character can be used to express preset text information. When performing entry replacement, the background script will convert the second detection character into the preset text information that matches the second detection character and then perform the replacement. Therefore, the content recorded in the target entry cannot be recognized only by using the replacement entry. At this time, it is necessary to convert the replacement entry into an entry that conforms to the preset rules, compare the entry that conforms to the preset rules with the target entry to obtain a comparison result, and then determine the target entry as an abnormally displayed entry according to the comparison result.

[0107] Theoretically, when the replacement entry record does not match the target entry record, it can be explained that the target entry has a high probability of truncation anomaly. However, since the calculation for further detecting the second detection character will consume a large amount of computing resources, it is possible to first identify whether the first detection character exists in the target entry. If it exists, it indicates that the target entry has definitely experienced truncation anomaly, that is, there is no need to identify the second detection character for this target entry, effectively improving the recognition efficiency of the terminal interface. It should also be understood that in the case where the target entry does not contain the first detection character and the replacement entry does not contain the second detection character, it can also indicate that the target entry is an abnormally displayed entry. Among them, in this case, it may include abnormally displayed entries caused by errors in the entry compression keyword.

[0108] Step 608, mark the abnormally displayed entry on the replacement terminal interface.

[0109] Therefore, the image-based terminal interface recognition method proposed in this application can determine whether the target entry matches the replacement entry through various matching rules between the replacement entry and the target entry, thereby effectively improving the accuracy of identifying character display anomalies in the target entry.

[0110] As a specific embodiment, as Figure 7 shown, the image-based terminal interface recognition includes the following steps:

[0111] Step 701, obtain a screenshot image of the replacement terminal interface, and identify at least one target entry for the screenshot image.

[0112] Step 702, for each of the target entries, obtain the entry compression keyword shortkey in the target entry.

[0113] Step 703, obtain the corresponding replacement entry key and the padding character value corresponding to the redundant character length according to the entry compression keyword.

[0114] Step 704, determine whether the replacement entry and the target entry are the same.

[0115] If so, determine that the target entry has no anomaly; if not, execute step 705.

[0116] Step 705, determine whether the target entry contains an ellipsis.

[0117] If it contains, determine that the target entry is an abnormally displayed entry; if not, execute step 706.

[0118] Step 706, determine whether the replacement entry contains a placeholder.

[0119] If it is included, execute step 707; if not, determine that the target entry is an abnormally displayed entry.

[0120] Step 707: Determine whether the placeholder in the replacement entry and the target entry meet the preset rules.

[0121] If they meet the rules, determine that the target entry is normal; if not, determine that the target entry is an abnormally displayed entry.

[0122] Thus, this application can automatically identify the abnormality of the entries on the terminal interface of the application layer sequence App by performing text recognition on the replacement entry and the target entry, greatly improving the efficiency of interface recognition.

[0123] As a feasible embodiment, mark the abnormally displayed entry on the replacement terminal interface, including: identifying the position of the abnormally displayed entry in the screenshot image; based on the position, mark the abnormally displayed entry in the screenshot image, and the marking includes reminder information corresponding to the abnormally displayed entry.

[0124] That is to say, after identifying the abnormally displayed entry, in order to facilitate the developer to quickly and accurately locate the abnormally displayed entry and repair its language replacement function, this application further identifies the position of the abnormally displayed entry in the screenshot and marks the abnormally displayed entry according to its position in the screenshot, such as reminder information corresponding to the abnormally displayed entry. For example, as Figure 8 shown, the entry containing an ellipsis is the abnormally displayed entry, and a reminder is given through the reminder information 801 corresponding to the abnormally displayed entry.

[0125] In some embodiments, only the marked image can be used as reminder information for feedback reminder, so that the developer can determine the entry with an abnormality according to the frame in the marked image. Preferably, since there are a large number of entry compression keywords and redundant English entries in the replacement terminal interface of the replacement, and there may be multiple entry applications in the same interface layout in the application App, it is impossible for the developer to know in which application scenario the abnormally displayed entry occurs only based on the marked image. Therefore, the attribute of the target entry with an abnormality can be further analyzed, and the analysis result and the marked image are combined to obtain reminder information, so that the developer can more quickly understand the application scenario where the abnormality occurs according to the analysis result.

[0126] In summary, the terminal interface recognition method, device, electronic device, and storage medium provided by the embodiments of the present application replace the original terms in the original terminal interface, then take a screenshot of the replaced terminal interface after replacement, identify the abnormally displayed terms included in the screenshot image for the screenshot image, and mark the abnormally displayed terms on the replaced terminal interface. Thus, the present application can convert the manual recognition technology for the terminal interface into automated graphic and text recognition and character recognition by using graphic and text recognition technology, effectively improving the recognition efficiency of character display anomalies in the application program (App) terminal interface.

[0127] Further, please refer to Figure 9 , Figure 9 which shows an exemplary structural block diagram of the image-based terminal interface recognition device provided by the embodiments of the present application. As Figure 9 shown, the device 10 includes:

[0128] A replacement module 11, configured to replace at least one original term included in the original terminal interface with at least one replacement term respectively to obtain a replaced terminal interface after replacement;

[0129] A first acquisition module 12, configured to acquire a screenshot image of the replaced terminal interface;

[0130] An identification module 13, configured to identify at least one target term for the screenshot image;

[0131] A second acquisition module 14, configured to acquire a replacement term corresponding to each target term for each target term;

[0132] A determination module 15, configured to determine that the target term is an abnormally displayed term when the replacement term and the target term do not match;

[0133] A reminder module 16, configured to mark the abnormally displayed term on the replaced terminal interface.

[0134] In some embodiments, the second acquisition module 14 is further configured to:

[0135] Identify the locator in the target term;

[0136] Identify the term compression keyword in the target term according to the locator;

[0137] Acquire the replacement term corresponding to the target term according to the term compression keyword.

[0138] In some embodiments, the determination module 15 is further configured to:

[0139] Determine that the target term is an abnormally displayed term when it is determined that the target term contains a first detection character;

[0140] When it is determined that the target entry does not contain the first detection character, the replacement entry is detected for containing the second detection character;

[0141] When the replacement entry contains the second detection character, the replacement entry is converted into an entry that conforms to a preset rule; and the entry that conforms to the preset rule is compared with the target entry to obtain a comparison result. Based on the comparison result, it is determined that the replacement entry and the target entry do not match, and the target entry is determined as an abnormally displayed entry;

[0142] When the replacement entry does not contain the second detection character, the target entry is determined as an abnormally displayed entry.

[0143] In some embodiments, the determination module 15 is further configured to:

[0144] When the comparison result indicates that the entry that conforms to the preset rule does not match the target entry, it is determined that the replacement entry and the target entry do not match;

[0145] When the comparison result indicates that the entry that conforms to the preset rule matches the target entry, it is determined that the replacement entry and the target entry match.

[0146] In some embodiments, the replacement module 11 is further configured to:

[0147] Perform signature encoding on each original entry to obtain a string result;

[0148] According to the string result and the target language identifier, an entry compression keyword corresponding to the original entry is obtained, and the target language identifier is used to indicate the target language after replacement;

[0149] Generate a replacement entry corresponding to the entry compression keyword according to the entry splicing rule, where the entry splicing rule makes the length difference between the replacement entry and the original entry corresponding to the replacement entry less than a preset threshold.

[0150] In some embodiments, the replacement module 11 is further configured to:

[0151] Obtain the target character length and locator corresponding to the target language;

[0152] Determine the redundant character length corresponding to the replacement entry according to the target character length and the compressed character length corresponding to the entry compression keyword;

[0153] If the redundant character length is less than or equal to the preset length, splice the entry compression keyword and the locator to obtain the replacement entry;

[0154] If the redundant character length is greater than the preset length, splice the entry compression keyword, the locator, and the padding characters corresponding to the redundant character length to obtain the replacement entry.

[0155] In some embodiments, the replacement module 11 is further configured to:

[0156] Generate a signature string according to the original entry, and equally divide the signature string according to a preset length to obtain a plurality of character arrays;

[0157] Process each character array in a loop to obtain an index corresponding to the character array;

[0158] Concatenate the multiple indexes to obtain a string result.

[0159] In some embodiments, the reminder module 14 is further configured to:

[0160] Identify the position of the abnormally displayed entry in the screenshot image;

[0161] Based on the position, label the abnormally displayed entry in the screenshot image, and the label includes reminder information corresponding to the abnormally displayed entry.

[0162] It should be understood that the various units or modules described in the device correspond to the respective steps in the method described above. Therefore, the operation instructions and features described above for the method also apply to the device and the modules included therein, and will not be repeated here. The device can be pre-implemented in a browser or other security application of an electronic device, or can be loaded into a browser or other security application of an electronic device by means of downloading, etc. The corresponding modules in the device can cooperate with the modules in the electronic device to implement the solutions of the embodiments of the present application.

[0163] The terminal interface recognition method, device, electronic device and storage medium provided by the embodiments of the present application. The device replaces entries in the original terminal interface, then takes a screenshot of the replaced terminal interface, identifies the abnormally displayed entries included in the screenshot image for the screenshot image, and labels the abnormally displayed entries on the replaced terminal interface. Thus, the present application can convert the manual recognition technology of the terminal interface into an automated graphic and text recognition and character recognition by using graphic and text recognition technology, effectively improving the recognition efficiency of character display abnormalities in the application program App terminal interface.

[0164] Among the several modules or units mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of the two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0165] Next, refer to Figure 10 , Figure 10 which shows a schematic structural diagram of a computer system of an electronic device or a server suitable for implementing the embodiments of the present application.

[0166] As Figure 10 shown, the computer system includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage section 1008 into a random access memory (RAM) 1003. In the RAM 1003, various programs and data required for the operation instructions of the system are also stored. The CPU 1001, ROM 1002, and RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0167] The following components are connected to the I / O interface 1005: an input section 1006 including a keyboard, a mouse, etc.; an output section 1007 including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, a modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed so that a computer program read from it can be installed into the storage section 1008 as needed.

[0168] Specifically, according to an embodiment of the present application, the process described above with reference to the flowchart Figure 2 can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program codes for performing the method shown in the flowchart. In such an embodiment, the computer program includes program codes for performing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 1009, and / or installed from the removable medium 1011. When the computer program is executed by a central processing unit (CPU) 1001, the above functions defined in the system of the present application are executed.

[0169] It should be noted that the computer-readable medium shown in this application can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this application, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on a computer-readable medium can be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination of the above.

[0170] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operation instructions of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram can represent a module, a program segment, or a part of code, and the foregoing module, program segment, or part of code contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two connected blocks can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that executes the specified functions or operation instructions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0171] The units or modules involved in the embodiments described in this application can be implemented in software or in hardware. The described units or modules can also be provided in a processor. For example, it can be described as: A processor includes a replacement module, an acquisition module, an identification module, and a reminder module. Among them, the names of these units or modules do not, in some cases, limit the units or modules themselves. For example, the replacement module can also be described as "replacing at least one original entry included in the original terminal interface with at least one replacement entry to obtain a replaced replacement terminal interface".

[0172] As another aspect, this application also provides a computer-readable storage medium. The computer-readable storage medium can be included in the electronic device described in the above embodiments, or can exist alone without being assembled into the electronic device. The above computer-readable storage medium stores one or more programs, and when the above programs are executed by one or more processors, they are used to implement the terminal interface identification method described in this application.

[0173] The above description is only a preferred embodiment of this application and an explanation of the technical principles applied. Those skilled in the art should understand that the scope of disclosure involved in this application is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the foregoing disclosure concept. For example, the technical solutions formed by mutually replacing the above features with the (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. An image-based terminal interface recognition method, characterized in that, It includes the following steps: Using at least one replacement entry to replace at least one original entry included in the original terminal interface respectively, to obtain a replaced replacement terminal interface; Obtaining a screenshot image of the replacement terminal interface; Performing recognition on the screenshot image to obtain at least one target entry; For each of the target entries, obtaining the replacement entry corresponding to the target entry; In the case where the replacement entry and the target entry do not match, determining the target entry as an abnormally displayed entry; Annotating the abnormally displayed entry on the replacement terminal interface; Before using at least one replacement entry to replace at least one original entry included in the original terminal interface, the method further includes: Performing signature encoding on each of the original entries to obtain a string result; According to the string result and a target language identifier, obtaining an entry compression keyword corresponding to the original entry, where the target language identifier is used to indicate the target language after replacement; Generating a replacement entry corresponding to the entry compression keyword according to an entry splicing rule, where the entry splicing rule makes the difference in the character lengths between the replacement entry and the target character length corresponding to the replacement entry less than a preset threshold, and the target character length is used to indicate the character length of the target language after replacement.

2. The method according to claim 1, wherein The step of, for each of the target entries, obtaining the replacement entry corresponding to the target entry includes: Identifying the locator in the target entry; According to the locator, identifying the entry compression keyword in the target entry; According to the entry compression keyword, obtaining the replacement entry corresponding to the target entry.

3. The method according to claim 2, wherein The step of, in the case where the replacement entry and the target entry do not match, determining the target entry as an abnormally displayed entry includes: When it is determined that the target entry contains a first detection character, determining the target entry as an abnormally displayed entry; When it is determined that the target entry does not contain the first detection character, detecting whether the replacement entry contains a second detection character; When the replacement entry contains the second detection character, converting the replacement entry into an entry that conforms to a preset rule; and comparing the entry that conforms to the preset rule with the target entry to obtain a comparison result, and based on the comparison result, determining that the replacement entry and the target entry do not match, and determining the target entry as an abnormally displayed entry; When the replacement entry does not contain the second detection character, determining the target entry as an abnormally displayed entry.

4. The method according to claim 3, characterized in that Based on the comparison result, determining that the replacement entry and the target entry do not match includes: When the comparison result indicates that the entry that conforms to the preset rule does not match the target entry, determining that the replacement entry and the target entry do not match; When the comparison result indicates that the entry that conforms to the preset rule matches the target entry, determining that the replacement entry and the target entry match.

5. The method according to claim 1, wherein The step of generating a replacement entry corresponding to the entry compression keyword according to an entry splicing rule includes: Obtaining the target character length and the locator corresponding to the target language; Determine the redundant character length corresponding to the replacement entry according to the target character length and the compressed character length corresponding to the entry compression keyword; If the redundant character length is less than or equal to the preset length, splice the entry compression keyword and the locator to obtain the replacement entry; If the redundant character length is greater than the preset length, splice the entry compression keyword, the locator, and the padding characters corresponding to the redundant character length to obtain the replacement entry.

6. The method according to claim 1, wherein Perform signature encoding on each of the original entries to obtain a string result, including: Generate a signature string according to the original entry, and equally divide the signature string into multiple character arrays according to the preset length; Process each of the character arrays in a loop to obtain an index corresponding to the character array; Splice the multiple indexes to obtain the string result.

7. The method according to claim 1, wherein The annotating the abnormal display entry on the replacement terminal interface includes: Identify the position of the abnormal display entry in the screenshot image; Based on the position, annotate the abnormal display entry in the screenshot image, and the annotation includes reminder information corresponding to the abnormal display entry.

8. An image-based terminal interface recognition device, characterized in that, Include: A replacement module for using at least one replacement entry to replace at least one original entry included in the original terminal interface respectively to obtain a replaced replacement terminal interface; A first acquisition module for acquiring a screenshot image of the replacement terminal interface; An identification module for identifying at least one target entry for the screenshot image; A second acquisition module for acquiring a replacement entry corresponding to each target entry; A determination module for determining the target entry as an abnormal display entry when the replacement entry does not match the target entry; A reminder module for annotating the abnormal display entry on the replacement terminal interface; The replacement module is further configured to: Perform signature encoding on each of the original entries to obtain a string result; Obtain an entry compression keyword corresponding to the original entry according to the string result and the target language identifier, where the target language identifier is used to indicate the target language after replacement; Generate a replacement entry corresponding to the entry compression keyword according to the entry splicing rule, where the entry splicing rule makes the difference between the replacement entry and the target character length corresponding to the replacement entry less than a preset threshold, and the target character length is used to indicate the character length of the target language after replacement.

9. The device according to claim 8, characterized in that, The second acquisition module is further configured to: Identify the locator in the target entry; Identify the entry compression keyword in the target entry according to the locator; Obtain a replacement entry corresponding to the target entry according to the entry compression keyword.

10. The device according to claim 9, characterized in that, The determination module is further configured to: Determine the target entry as an abnormal display entry when it is determined that the target entry contains a first detection character; When it is determined that the target entry does not contain the first detection character, detect whether the replacement entry contains a second detection character; When the replacement entry contains a second detection character, convert the replacement entry into an entry that conforms to a preset rule; and compare the entry that conforms to the preset rule with the target entry to obtain a comparison result, and based on the comparison result, determine that the replacement entry and the target entry do not match, and determine the target entry as an abnormally displayed entry; When the replacement entry does not contain a second detection character, determine that the target entry is an abnormally displayed entry.

11. The device according to claim 10, wherein The determining module is further configured to: When the comparison result indicates that the entry that conforms to the preset rule does not match the target entry, determine that the replacement entry and the target entry do not match; When the comparison result indicates that the entry that conforms to the preset rule matches the target entry, determine that the replacement entry and the target entry match.

12. The device according to claim 8, characterized in that, The replacement module is further configured to: Obtain the target character length and locator corresponding to the target language; Determine the redundant character length corresponding to the replacement entry according to the target character length and the compressed character length corresponding to the entry compression keyword; If the redundant character length is less than or equal to a preset length, splice the entry compression keyword and the locator to obtain the replacement entry; If the redundant character length is greater than the preset length, splice the entry compression keyword, the locator, and the padding characters corresponding to the redundant character length to obtain the replacement entry.

13. The device according to claim 8, characterized in that, The replacement module is further configured to: Generate a signature string according to the original entry, and equally divide the signature string into multiple character arrays according to a preset length; Process each of the character arrays in a loop to obtain an index corresponding to the character array; Splice the multiple indexes to obtain the string result.

14. The device according to claim 8, characterized in that, The reminder module is further configured to: Identify the position of the abnormally displayed entry in the screenshot image; Based on the position, annotate the abnormally displayed entry in the screenshot image, and the annotation includes reminder information corresponding to the abnormally displayed entry.

15. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the image-based terminal interface recognition method according to any one of claims 1-7.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the image-based terminal interface recognition method according to any one of claims 1-7.

Images