User interface test method, terminal device and computer program product

Abstract

The invention is suitable for the technical field of image processing, and provides a user interface test method, terminal equipment and a computer program product, the method comprises the following steps: obtaining a user interface image to be tested; performing character recognition on the user interface image to obtain target data; the target data comprises data content and a data position; the data position refers to the position of the target data in the user interface image; testing the target data in different dimensions based on the data content and the data position to obtain test results of the user interface image in different dimensions; the different dimensions comprise a character dimension and a translation dimension. According to the method and the device, accurate testing of the UI in the dimensions of characters, translation and the like is realized, and the testing quality is improved. Meanwhile, the problem that common testers test the small language UI page is solved, intervention of a professional language team is not needed, the application range is widened, and practicability is improved.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and in particular, relates to a user interface testing method, terminal equipment and computer program product. Background Art

[0002] With the continuous development of the globalization of domestic automobiles, more and more cars are no longer limited to domestic sales, but are often sold all over the world. Therefore, the in-vehicle system integrates multiple languages ​​to meet the needs of users in different regions. When testing the classical Chinese of the user interface (UI) in the in-vehicle system, the language ability of the tester is limited, which results in the limitation of the classical Chinese test of the UI interface.

[0003] At present, the correctness of the classical Chinese in the existing UI interface languages ​​needs to be manually checked by a professional language team, which has a small scope of application, low practicality and high labor costs; ordinary testers cannot understand the classical Chinese in small languages, which reduces the quality of testing. Summary of the invention

[0004] The embodiments of the present application provide a user interface testing method, terminal device and computer program product to solve the problems existing in the prior art, such as the need for manual inspection by a professional language team, a small scope of application, low practicality and high labor costs; and the inability of ordinary testers to understand the classical Chinese of small languages, which reduces the problem of test quality.

[0005] In a first aspect, an embodiment of the present application provides a user interface testing method, comprising:

[0006] Acquire a user interface image of the vehicle system to be tested;

[0007] Performing text recognition on the user interface image to obtain target data; the target data includes data content and data location; the data location refers to the location of the target data in the user interface image;

[0008] The target data is tested in different dimensions based on the data content and the data location to obtain test results of the user interface image in the different dimensions; the different dimensions include a text dimension and a translation dimension.

[0009] Optionally, performing text recognition on the user interface image to obtain target data includes:

[0010] Naming the user interface image based on a naming rule to obtain an image file containing the image name;

[0011] Inputting the image file into a text recognition model for recognition to obtain initial data; the initial data includes the image name, original content, data coordinates, and data confidence;

[0012] If the data confidence is greater than or equal to the set confidence, the initial data is processed based on the data processing rules to obtain the target data.

[0013] Optionally, the performing tests on the target data in different dimensions based on the data content and the data position to obtain test results of the user interface image in the different dimensions includes:

[0014] For the text dimension, obtaining historical version information corresponding to the target data; the historical version information includes historical content and historical location of the target data;

[0015] Testing the target data based on the historical version information to obtain a test result of the user interface image in the text dimension;

[0016] With respect to the translation dimension, the target data is input into a translation model for processing to obtain translation data corresponding to the target data;

[0017] A target image including the translation data and the data content is output based on the data position, and the target image is determined as a test result of the user interface image in the translation dimension; and a size of the target image is the same as a size of the user interface image.

[0018] Optionally, the target data includes target version information; the target data is at least two, the text dimension includes a content dimension, the content dimension includes a content quantity dimension and an actual content dimension; the test result of the user interface image under the content dimension is obtained according to the following method:

[0019] Determine the historical quantity of target data under the historical version information;

[0020] Compare the historical quantity with the quantity of target data under the target version information to obtain a test result of the user interface image under the content quantity dimension;

[0021] The data content of all target data under the target version information is compared with the data content of all target data under the historical version information to obtain a test result of the user interface image under the actual content dimension.

[0022] Optionally, the data position includes a starting point, an end point, and a center point of the target data, the historical position includes a historical starting point, a historical end point, and a historical center point of the target data, the text dimension includes a position dimension, and the test result of the user interface image under the position dimension is obtained according to the following method:

[0023] Compare the starting point, the end point and the center point of the target data with the historical starting point, the historical end point and the historical center point respectively, to obtain a first result of the user interface image in the position dimension;

[0024] Compare the font size of the data content with the font size of the historical content to obtain a second result of the user interface image in the position dimension;

[0025] Based on the first result and the second result, a test result of the user interface image in the position dimension is determined.

[0026] Optionally, the text dimension includes a font dimension, and the test result of the user interface image under the font dimension is obtained according to the following method:

[0027] Performing cropping processing on the user interface image based on the data position to obtain a first image containing only the target data;

[0028] Based on the historical position, a historical interface image corresponding to the historical version information is cropped to obtain a second image containing only the target data;

[0029] Performing grayscale processing on the first image and the second image respectively to obtain a third image corresponding to the first image and a fourth image corresponding to the second image;

[0030] Calculating the similarity between the third image and the fourth image to obtain a first structural similarity index;

[0031] Based on the first structural similarity index, a test result of the user interface image in the font dimension is determined.

[0032] Optionally, the text dimension includes a color dimension, and the test result of the user interface image under the color dimension is obtained according to the following method:

[0033] Performing cropping processing on the user interface image based on the data position to obtain a first image containing only the target data;

[0034] Based on the historical position, a historical interface image corresponding to the historical version information is cropped to obtain a second image containing only the target data;

[0035] Performing three-channel color separation processing on the first image and the second image respectively to obtain a first sub-image of the first image in different channel colors and a second sub-image of the second image in the different channel colors;

[0036] For any channel color, calculate the similarity between the first sub-image and the second sub-image under the any channel color to obtain a second structural similarity index;

[0037] Based on the second structural similarity index under the different channel colors, a test result of the user interface image under the color dimension is determined.

[0038] Optionally, after performing the test of different dimensions on the target data based on the data content and the data position to obtain the test results of the user interface image in the different dimensions, the method further includes:

[0039] For any dimension, if the test result of any dimension is different from the target data under the historical version information, the target data is marked based on the data position, and the marked user interface image is output.

[0040] In a second aspect, an embodiment of the present application provides a user interface testing device, comprising:

[0041] A first acquisition unit, used to acquire a user interface image to be tested;

[0042] A first recognition unit is used to perform text recognition on the user interface image to obtain target data; the target data includes data content and data position; the data position refers to the position of the target data in the user interface image;

[0043] The first testing unit is used to test the target data in different dimensions based on the data content and the data location to obtain test results of the user interface image in the different dimensions; the different dimensions include text dimension and translation dimension.

[0044] In a third aspect, an embodiment of the present application provides a terminal device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements a user interface testing method as described in any one of the first aspects above.

[0045] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the user interface testing method as described in any one of the above-mentioned first aspects is implemented.

[0046] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a terminal device, enables the terminal device to execute the user interface testing method described in any one of the above-mentioned first aspects.

[0047] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0048] A user interface testing method provided in an embodiment of the present application obtains a user interface image to be tested; performs text recognition on the user interface image to obtain target data; the target data includes data content and data position; the data position refers to the position of the target data in the user interface image; based on the data content and data position, the target data is tested in different dimensions to obtain test results of the user interface image in different dimensions; the different dimensions include text dimension and translation dimension. Compared with the prior art, the present application can achieve accurate testing of UI interface in dimensions such as text and translation by obtaining target data including data content and data position through text recognition, thereby improving the test quality. At the same time, it solves the problem of ordinary testers who do not have the function of small languages ​​to test small language UI pages, without the need for the intervention of professional language teams, expanding the scope of application and increasing practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0050] Figure 1 is a flowchart of an implementation of a user interface testing method provided in an embodiment of the present application;

[0051] Figure 2 is a flowchart of an implementation of a user interface testing method provided by another embodiment of the present application;

[0052] Figure 3 It is a schematic diagram of a data structure provided by an embodiment of the present application;

[0053] Figure 4 is a schematic diagram of target data provided by an embodiment of the present application;

[0054] Figure 5 is a flowchart of an implementation of a user interface testing method provided in yet another embodiment of the present application;

[0055] Figure 6is a flowchart of an implementation of a user interface testing method provided by another embodiment of the present application;

[0056] Figure 7 is a flowchart of an implementation of a user interface testing method provided by another embodiment of the present application;

[0057] Figure 8 is a schematic diagram of a target image provided by an embodiment of the present application;

[0058] Fig. 9 is a structural schematic diagram of a user interface testing device provided in one embodiment of the present application;

[0059] Fig.10 It is a structural diagram of a terminal device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0060] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0061] In actual applications, with the continuous development of the globalization of domestic automobiles, more and more cars are no longer limited to domestic sales, but are often sold all over the world. Therefore, the in-vehicle system integrates multiple languages ​​to meet the needs of users in different regions. When testing the classical Chinese of the user interface (UI) in the in-vehicle system, the language ability of the tester is limited, which results in the limitation of the classical Chinese test of the UI interface.

[0062] At present, the correctness of the classical Chinese in the existing UI interface languages ​​needs to be manually checked by a professional language team, which has a small scope of application, low practicality and high labor costs; ordinary testers cannot understand the classical Chinese in small languages, which reduces the quality of testing.

[0063] Based on this, this application proposes a user interface testing method that does not require the intervention of a professional language team. Even ordinary testers who do not have minority language capabilities can achieve accurate testing of multiple dimensions such as content, font, position, color, and translation of the UI interface.

[0064] See also Figure 1 , Figure 1 This is a flowchart of a user interface testing method provided in an embodiment of the present application. In the embodiment of the present application, the execution subject of the user interface testing method is a terminal device. The terminal device includes but is not limited to: a notebook, a desktop computer, and a computer.

[0065] like Figure 1 As shown, the user interface testing method provided in an embodiment of the present application may include S101 to S103, which are described in detail as follows:

[0066] In S101, a user interface image to be tested is obtained.

[0067] In the embodiment of the present application, when relevant personnel need to test the UI interface of the vehicle computer, they can establish a connection with the debugging tool (Android Debug Bridge, ADB) of the vehicle computer through their own terminal device and obtain the root permission of the vehicle computer.

[0068] In practical applications, ADB is a tool for debugging Android devices, which allows users to fully operate and manage Android devices through computers. ADB is a command line tool based on C / S architecture, through which the PC can communicate with Android devices.

[0069] In the car computer industry, the use of ADB tools is very common. Since most modern car computers use the Android system, ADB has become an important tool for debugging car computer applications or programs. Through ADB, developers can easily debug applications in car computers, perform system upgrades, install software, and other operations.

[0070] In an embodiment of the present application, after obtaining the root permission of the vehicle computer, the terminal device can perform a screenshot operation on the UI interface of the vehicle computer to obtain a screenshot of the UI interface of the vehicle computer, that is, the user interface image to be tested.

[0071] In S102, text recognition is performed on the user interface image to obtain target data; the target data includes data content and data position; the data position refers to the position of the target data in the user interface image.

[0072] In the embodiment of the present application, after obtaining the user interface image, the terminal device can input the user interface image into the text recognition model for recognition to obtain the target data in the user interface image. The target data includes but is not limited to data content and data location. The data location refers to the location of the target data in the user interface image.

[0073] It should be noted that the target data specifically refers to the data in the user interface image that needs to be tested.

[0074] In the embodiment of the present application, the target data may be one or more.

[0075] It should be noted that a target data may include a group of continuous characters, and a group of continuous characters may include at least one character.

[0076] Therefore, after the terminal device inputs the user interface image into the text recognition model for recognition and obtains all the text contained in the user interface image, the terminal device can traverse and detect the left and right spacing between each text, and determine at least two texts whose left and right spacing is less than the set spacing as continuous text. The set spacing can be determined according to actual needs and is not limited here.

[0077] Based on this, the terminal device can obtain multiple groups of continuous characters in the user interface image. Afterwards, the terminal device can determine a group of continuous characters as a target data.

[0078] In practical applications, the Optical Character Recognition (OCR) model is a technology that can recognize and extract printed and handwritten text from digital images and PDFs. This technology automatically extracts text information from digital images or PDF files by scanning them.

[0079] In one embodiment of the present application, in order to improve the accuracy of obtaining target data and improve the accuracy of subsequent tests, the terminal device can specifically perform the following steps: Figure 2 Steps S201 to S203 shown in the figure obtain target data, which are described in detail as follows:

[0080] In S201, the user interface image is named based on a naming rule to obtain an image file containing the image name.

[0081] In this embodiment, in order to facilitate subsequent search and comparison of the UI interface under the corresponding historical version to improve the test accuracy, the terminal device can name the acquired user interface image based on the naming rule to obtain an image file containing the image name. The naming rule can be determined according to actual needs and is not limited here.

[0082] In some possible embodiments, the naming rule may include: naming with "project-version-module-page-time (screenshot time)", and any two adjacent words are connected with a separator "-".

[0083] For example, when the project corresponding to a user interface image is Car01, the version is x1x2x3x4xxx, the module is loveCar, the page is homePage, and the screenshot time is 20240909102501, the image name can be: Car01-x1x2x3x4xxx-loveCar-homePage-20240909102501.png.

[0084] In S202, the image file is input into a text recognition model for recognition to obtain initial data; the initial data includes the image name, original content, data coordinates, and data confidence.

[0085] In this embodiment, after obtaining the above image file, in order to obtain accurate target data, the terminal device can input the above image file into a text recognition model for recognition to obtain initial data, wherein the initial data includes image name, original content, data coordinates and data confidence.

[0086] It should be noted that the initial data is multiple.

[0087] In this embodiment, in order to improve the test efficiency, the terminal device may compare the data confidence of the initial data with the set confidence to determine whether the initial data needs to be tested.

[0088] In one embodiment of the present application, when the terminal device detects that the data confidence of the initial data is less than the set confidence, it indicates that the initial data does not need to be tested, that is, the initial data is invalid data. Therefore, the terminal device can discard the initial data and continue to perform confidence detection on the next initial data.

[0089] In another embodiment of the present application, when the terminal device detects that the data confidence of the initial data is greater than or equal to the set confidence, step S203 may be executed.

[0090] In S203, if the data confidence is greater than or equal to the set confidence, the initial data is processed based on the data processing rule to obtain the target data.

[0091] In this embodiment, when the terminal device detects that the data confidence of the initial data is greater than or equal to the set confidence, it indicates that the initial data needs to be tested, that is, the initial data is valid data. Therefore, in order to improve the subsequent test efficiency and test accuracy, the terminal device can process the initial data based on the data processing rule to obtain the target data. The data processing rule can be determined according to actual needs and is not limited here.

[0092] In some possible embodiments, the data processing rule may include: processing the initial data according to the data structure. Figure 3 shown.

[0093] For example, see Figure 3 , Figure 3 Schematic diagram of the data structure provided by an embodiment of the present application. Figure 3As shown, the data structure includes: project, version, module, page, and textdatas, including text, position, ex_position, confidence, and timestamp.

[0094] It should be noted that the above-mentioned text specifically refers to the original text corresponding to the initial data, that is, the original content, position specifically includes the starting point and end point of the initial data, and confidence specifically refers to the data confidence of the initial data.

[0095] Based on this, the terminal device can Figure 3 The data structure shown processes the initial data accordingly to obtain the target data.

[0096] Specifically, since the above data structure includes project, version, module, page and timestamp, and combined with S201, the image name in the initial data is named "project-version-module-page-time (screenshot time)", the terminal device can determine the above project, version, module, page and timestamp according to the image name of the initial data.

[0097] For example, assuming that the image name is: Car01-x1x2x3x4xxx-loveCar-homePage-20240909102501.png, the corresponding project is "Car01", the version is "x1x2x3x4xxx", the module is "loveCar", the page is "homePage", and the timestamp is "20240909102501".

[0098] At the same time, the terminal device can obtain the upper left corner coordinate and the lower right corner coordinate of the initial data in real time, and determine the upper left corner coordinate as the starting point of the initial data, and determine the lower right corner coordinate as the end point of the initial data, thereby obtaining position. Afterwards, the terminal device can calculate the average of the horizontal coordinate of the upper left corner coordinate and the horizontal coordinate of the lower right corner coordinate to obtain the horizontal coordinate of the center point, and the terminal device can calculate the average of the vertical coordinate of the upper left corner coordinate and the vertical coordinate of the lower right corner coordinate to obtain the vertical coordinate of the center point, thereby obtaining ex_position.

[0099] For example, see Figure 4 , Figure 4 is a schematic diagram of target data provided by an embodiment of the present application. Figure 4 As shown, the target data includes two.

[0100] In some possible embodiments, the target data may further include an image name, data confidence, a data center point, a timestamp, and the like.

[0101] In S103, the target data is tested in different dimensions based on the data content and the data location to obtain test results of the user interface image in the different dimensions; the different dimensions include a text dimension and a translation dimension.

[0102] In the embodiment of the present application, after obtaining the target data, the terminal device can perform different dimensional tests on the target data according to the data content and data location in the target data to obtain test results of the user interface image in different dimensions, wherein the different dimensions include but are not limited to text dimension and translation dimension.

[0103] Among them, text dimensions include but are not limited to content dimension, position dimension, font dimension, and color dimension.

[0104] Specifically, the terminal device can input the target data including the data content and the data location into the test model for processing to obtain the test results of the user interface image in different dimensions.

[0105] The test model can be obtained by training a pre-built neural network model based on a preset sample set. Each sample data in the preset sample set includes sample information (the sample information includes the sample data location and sample data content) and a sample test result corresponding to the sample information (the sample test result includes test sub-results of different dimensions). When training the pre-built neural network model, the sample information in each sample data is used as the input of the neural network model, and the sample test result corresponding to the sample information in each sample data is used as the output of the neural network model. Through training, the neural network model can learn the correspondence between all possible sample information and sample test results, and the trained neural network model is used as the test model.

[0106] In some possible embodiments, in order to improve the test accuracy of different dimensions, the above test model may include multiple test sub-models, such as a text test sub-model and a translation test sub-model.

[0107] In some other possible embodiments, the text test sub-model may include a content test sub-model, a position test sub-model, a font test sub-model, a color test sub-model, and the like.

[0108] In the embodiment of the present application, the test results of the text dimension include but are not limited to test pass and test fail.

[0109] It should be noted that when the test result is that the test fails, in order to facilitate relevant personnel to promptly determine the area in the user interface image that fails the test, the test result may also include difference content.

[0110] The test results of the translation dimension include the translation content corresponding to the data content.

[0111] In one embodiment of the present application, the terminal device can specifically Figure 5 Steps S301 to S304 shown in the figure obtain test results of different dimensions, which are described in detail as follows:

[0112] In S301, with respect to the text dimension, historical version information corresponding to the target data is obtained; the historical version information includes historical content and historical position of the target data.

[0113] In S302, the target data is tested based on the historical version information to obtain a test result of the user interface image in the text dimension.

[0114] It should be noted that the target data may also include target version information.

[0115] In this embodiment, since the database corresponding to the terminal device stores a large amount of various data under historical versions, the terminal device can obtain the historical version information corresponding to the target data through the target version information in the target data, wherein the historical version information includes the historical content and historical location of the target data.

[0116] It should be noted that the historical version information specifically refers to: information included in the last version before the version included in the image name of the target data.

[0117] Afterwards, the terminal device may compare the target data with the historical version information in the text dimension to detect the similarity between the target data and the historical version information to obtain a test result of the user interface image in the text dimension.

[0118] In some possible embodiments, when the text dimension includes content dimension, position dimension, translation dimension, font dimension and color dimension, the terminal device can also compare the target data with the historical version information one by one in the content dimension, position dimension, translation dimension, font dimension and color dimension in turn, and detect the similarity between the target data and the historical version information to obtain the test results of the user interface image in the content dimension, position dimension, font dimension and color dimension respectively.

[0119] In one embodiment of the present application, when the target data includes target version information, and the target data is at least two, the text dimension includes the content dimension, and the content dimension includes the content quantity dimension and the actual content dimension, in order to improve the test accuracy and test quality under the content dimension, the terminal device can specifically obtain the test result under the content dimension according to the following steps, which are detailed as follows:

[0120] Determine the historical quantity of target data under the historical version information;

[0121] Compare the historical quantity with the quantity of target data under the target version information to obtain a test result of the user interface image under the content quantity dimension;

[0122] The data content of all target data under the target version information is compared with the data content of all target data under the historical version information to obtain a test result of the user interface image under the actual content dimension.

[0123] In this embodiment, the terminal device may count the historical quantity of target data under the historical version information, and count the quantity of target data under the target version information.

[0124] Afterwards, the terminal device can compare the historical quantity with the quantity of target data under the target version information to obtain the test result of the user interface image under the content quantity dimension. The test result under the content quantity dimension includes but is not limited to: test passed and test failed.

[0125] It should be noted that the above test is used to indicate that the amount of target data in the user interface image is equal to the historical amount of target data in the historical version information, that is, the user interface image has no change in the amount of target data.

[0126] The failure of the above test is used to indicate that the amount of target data in the user interface image is not equal to the historical amount of target data in the historical version information, that is, the user interface image has changed in the amount of target data.

[0127] In some possible embodiments, when the terminal device determines that the test result of the user interface image in the content quantity dimension is a test failure, it can use the Python set library to perform a difference set to record the target data and / or redundant target data in the user interface image that does not exist relative to the historical version information.

[0128] In this embodiment, the terminal device can integrate the data content of all target data under the target version information to obtain the overall data content under the target version information. At the same time, the terminal device can also integrate the data content of all target data under the historical version information to obtain the overall data content under the historical version information.

[0129] In some possible embodiments, the terminal device may splice the data contents of each target data under the target version information to integrate the data contents of all the target data under the target version information, thereby obtaining the overall data content under the target version information.

[0130] Exemplarily, assuming that the data contents of each target data under the target version information are X1, X2, X3, X4, X5 respectively, the overall data content under the target version information may be: [X1, X2, X3, X4, X5].

[0131] In some other possible embodiments, the terminal device may splice the data contents of each target data under the historical version information to integrate the data contents of all the target data under the historical version information, thereby obtaining the overall data content under the historical version information.

[0132] Exemplarily, assuming that the data contents of each target data under the historical version information are A1, A2, A3, A4, A5 respectively, the overall data content under the historical version information may be: [A1, A2, A3, A4, A5].

[0133] Afterwards, the terminal device may compare the overall data content under the target version information with the overall data content under the historical version information to obtain a test result of the user interface image under the actual content dimension.

[0134] It should be noted that the test results under the actual content dimension are used to describe whether the various data contents included in the overall data content under the target version information are the same as the various data contents included in the overall data content under the historical version information.

[0135] In this embodiment, the test results in the actual content dimension include but are not limited to: test passed and test failed.

[0136] It should be noted that the above test is used to characterize that the overall data content under the above target version information is the same as the overall data content under the historical version information, that is, the various data contents contained in the overall data content under the target version information correspond to the various data contents contained in the overall data content under the historical version information.

[0137] In some possible embodiments, the terminal device may compare each data content included in the overall data content under the target version information with each data content included in the overall data content under the historical version information one by one to detect whether there is a one-to-one correspondence between the each data content included in the overall data content under the target version information and the each data content included in the overall data content under the historical version information, that is, whether the each data content included in the overall data content under the target version information is the same as the each data content included in the overall data content under the historical version information, thereby obtaining a test result of the user interface image under the actual content dimension.

[0138] Exemplarily, assuming that the overall data content under the target version information is [X1, X2, X3, X4, X5], and the overall data content under the historical version information is also [X1, X2, X3, X4, X5], it means that there is a one-to-one correspondence between the various data contents included in the overall data content under the target version information and the various data contents included in the overall data content under the historical version information, that is, the various data contents included in the overall data content under the target version information are the same as the various data contents included in the overall data content under the historical version information. Therefore, the terminal device can determine that the test result of the user interface image under the actual content dimension is that the test passed.

[0139] It should be noted that the concatenation order of the data contents of the above-mentioned target data will not affect the test result obtained after comparing the data contents included in the overall data content under the target version information with the data contents included in the overall data content under the historical version information.

[0140] Exemplarily, assuming that the overall data content under the target version information is [X1, X2, X3, X4, X5], and the overall data content under the historical version information is [X1, X3, X2, X5, X4], since there is still a one-to-one correspondence between the various data contents included in the overall data content under the target version information and the various data contents included in the overall data content under the historical version information, that is, the various data contents included in the overall data content under the target version information and the various data contents included in the overall data content under the historical version information are still the same, therefore, the terminal device can still determine that the test result of the user interface image under the actual content dimension at this time is that the test passed.

[0141] For example, assuming that the overall data content under the target version information is [X1, X2, X3, X4, X5], and the overall data content under the historical version information is also [X1, X2, X3, X4, X6], it means that the data contents included in the overall data content under the target version information and the data contents included in the overall data content under the historical version information do not correspond one to one, that is, the data contents included in the overall data content under the target version information and the data contents included in the overall data content under the historical version information are not exactly the same. Therefore, the terminal device can determine that the test result of the user interface image in the actual content dimension is that the test failed.

[0142] Among them, the correspondence between the various data contents included in the overall data content under the target version information and the various data contents included in the overall data content under the historical version information specifically means that the actual meaning expressed by the various data contents included in the overall data content under the target version information is the same as the actual meaning expressed by the various data contents included in the overall data content under the historical version information.

[0143] The failure of the above test is used to indicate that the overall data content under the above target version information is different from the overall data content under the historical version information, that is, the individual data contents included in the overall data content under the target version information do not correspond to the individual data contents included in the overall data content under the historical version information.

[0144] It should be noted that the above non-correspondence may be: among the data contents included in the overall data content under the target version information, there is a certain data content that is not included in the overall data content under the historical version information.

[0145] The above-mentioned non-correspondence may also be: among the data contents included in the overall data content under the historical version information, there is a certain data content that is not included in the overall data content under the target version information.

[0146] In some other possible embodiments, when the terminal device determines that the test result of the user interface image in the actual content dimension is a test failure, the terminal device may record the portion of the target data that has a content difference.

[0147] In another embodiment of the present application, when the data position of the target data includes the starting point, the end point and the center point of the target data, the historical position includes the historical starting point, the historical end point and the historical center point of the target data, and the text dimension includes the position dimension, in order to improve the test accuracy and test quality in the position dimension, the terminal device can specifically obtain the test result of the user interface image in the position dimension according to the following steps, which are detailed as follows:

[0148] Compare the starting point, the end point and the center point of the target data with the historical starting point, the historical end point and the historical center point respectively, to obtain a first result of the user interface image in the position dimension;

[0149] Compare the font size of the data content with the font size of the historical content to obtain a second result of the user interface image in the position dimension;

[0150] Based on the first result and the second result, a test result of the user interface image in the position dimension is determined.

[0151] In this embodiment, when the target data includes multiple ones, the terminal device can compare the data content of each target data under the target version information with the data content of each target data under the historical version information one by one, so as to determine the data content of a target data under the historical version information that is the same as the data content of any target data under the target version information, and match the target data under the target version information with the same data content with the target data under the historical version information.

[0152] Afterwards, the terminal device can compare the target data under the successfully matched target version information with the target data under the historical version information in terms of location dimension.

[0153] It should be noted that the starting point of the target data specifically refers to the upper left corner coordinates of the first-order data in the target data, the end point of the target data specifically refers to the lower right corner coordinates of the last-order data in the target data, and the center point of the target data is calculated from the starting point and end point of the target data.

[0154] In this embodiment, in order to accurately test the data location of the target data, the terminal device may compare the starting point of the target data with its matching historical starting point, compare the end point of the target data with its matching historical end point, and compare the center point of the target data with its matching historical end point, so as to obtain the first result of the user interface image in the location dimension. The first result includes but is not limited to test pass and test fail.

[0155] It should be noted that the above test is used to characterize that the starting point, end point and center point of the target data have passed the test, that is, the starting point, end point and center point of the target data are exactly the same as their corresponding historical starting point, historical end point and historical center point respectively.

[0156] The above test failure is used to indicate that at least one of the starting point, end point and center point of the target data has failed the test, that is, at least one of the starting point, end point and center point of the target data is different from the corresponding historical position.

[0157] In practical applications, since the font size will also affect the data position of the target data, that is, cause the data position to change, in this embodiment, in order to further improve the test accuracy of the target data position, the terminal device can compare the font size of the data content of the target data with the font size of its corresponding historical content to obtain the second result of the user interface image under the position dimension. The second result includes but is not limited to test pass and test fail.

[0158] It should be noted that the above test uses the same font size of the data content used to characterize the target data as the font size of the corresponding historical content.

[0159] The above test fails because the font size of the data content used to characterize the target data is not completely the same as the font size of its corresponding historical content.

[0160] Based on this, the terminal device can determine the test result of the user interface image in the said position dimension according to the said first result and the second result.

[0161] Specifically, when the first result is that the test fails, and / or the second result is that the test fails, the terminal device can determine that the test result of the user interface image in the position dimension is that the test fails; when the first result is that the test passes, and the second result is that the test passes, the terminal device can determine that the test result of the user interface image in the position dimension is that the test passes.

[0162] In some possible embodiments, when the terminal device determines that the test result of the user interface image in the position dimension is a test failure, the terminal device may record a portion of the target data that has a position difference.

[0163] In another embodiment of the present application, when the text dimension includes a font dimension, in order to improve the test accuracy and test quality in the font dimension, the terminal device may specifically perform the following steps: Figure 6 The steps S401 to S405 shown in the figure obtain the test results of the user interface image in the font dimension, which are described in detail as follows:

[0164] In S401, the user interface image is cropped based on the data position to obtain a first image containing only the target data.

[0165] In S402, the historical interface image corresponding to the historical version information is cropped based on the historical position to obtain a second image containing only the target data.

[0166] In this embodiment, when the target data includes multiple ones, the terminal device can compare the data content of each target data under the target version information with the data content of each target data under the historical version information one by one, so as to determine the data content of a target data under the historical version information that is the same as the data content of any target data under the target version information, and match the target data under the target version information with the same data content with the target data under the historical version information.

[0167] Afterwards, the terminal device can compare the target data under the successfully matched target version information with the target data under the historical version information in terms of font dimensions.

[0168] In this embodiment, in order to achieve accurate testing of the user interface image in the font dimension and avoid the influence of other data except the target data on the test, the terminal device can crop the user interface image based on the data position of the target data to obtain a first image containing only the target data. At the same time, the terminal device can crop the historical interface image corresponding to the historical version information based on the historical position matching the target data to obtain a second image containing only the target data.

[0169] In S403, grayscale processing is performed on the first image and the second image respectively to obtain a third image corresponding to the first image and a fourth image corresponding to the second image.

[0170] In this embodiment, in order to further improve the test quality in the font dimension, the terminal device can perform grayscale processing on the first image and the second image respectively to identify the classical Chinese outline and details of the target data, thereby obtaining a third image corresponding to the first image and a fourth image corresponding to the second image.

[0171] In some possible embodiments, the terminal device may use Python's PIL library to perform grayscale processing on the first image and the second image.

[0172] In S404, the similarity between the third image and the fourth image is calculated to obtain a first structural similarity index.

[0173] In this embodiment, after obtaining the third image and the fourth image, the terminal device may compare the third image with the fourth image to detect the similarity between the third image and the fourth image, thereby obtaining a first structural similarity index.

[0174] In practical applications, the structural similarity (SSIM) index is an indicator to measure the similarity between two images.

[0175] In some possible embodiments, the terminal device may use the OpenCV library of python to perform a similarity comparison on the third image and the fourth image to obtain the first structural similarity index.

[0176] In S405 , based on the first structural similarity index, a test result of the user interface image in the font dimension is determined.

[0177] In this embodiment, after obtaining the first structural similarity index, the terminal device may compare the first structural similarity index with a set threshold value, wherein the set threshold value may be determined according to actual needs and is not limited here.

[0178] In one embodiment of the present application, when the terminal device detects that the first structural similarity index is greater than or equal to a set threshold, it can determine that the similarity between the third image and the fourth image is extremely high. Therefore, the terminal device can determine that the test result of the user interface image in the font dimension is a passed test.

[0179] In another embodiment of the present application, when the terminal device detects that the first structural similarity index is less than a set threshold, it can determine that the similarity between the third image and the fourth image is low. Therefore, the terminal device can determine that the test result of the user interface image in the font dimension is that the test failed.

[0180] In some possible embodiments, when the terminal device determines that the test result of the user interface image in the font dimension is a test failure, the terminal device may record a portion of the target data that has a font difference.

[0181] In another embodiment of the present application, when the text dimension includes a color dimension, in order to improve the test accuracy and test quality under the color dimension, the terminal device may specifically perform the following steps: Figure 7 The steps S501 to S505 shown in the figure obtain the test results of the user interface image in the color dimension, which are described in detail as follows:

[0182] In S501, the user interface image is cropped based on the data position to obtain a first image containing only the target data.

[0183] In S502, the historical interface image corresponding to the historical version information is cropped based on the historical position to obtain a second image containing only the target data.

[0184] In this embodiment, when the target data includes multiple ones, the terminal device can compare the data content of each target data under the target version information with the data content of each target data under the historical version information one by one, so as to determine the data content of a target data under the historical version information that is the same as the data content of any target data under the target version information, and match the target data under the target version information with the same data content with the target data under the historical version information.

[0185] Afterwards, the terminal device can compare the target data under the successfully matched target version information and the target data under the historical version information in terms of color dimension.

[0186] In this embodiment, in order to achieve accurate testing of the user interface image in the color dimension and avoid the influence of other data except the target data on the test, the terminal device can crop the user interface image based on the data position of the target data to obtain a first image containing only the target data. At the same time, the terminal device can crop the historical interface image corresponding to the historical version information based on the historical position matching the target data to obtain a second image containing only the target data.

[0187] In S503, three-channel color separation processing is performed on the first image and the second image respectively to obtain a first sub-image of the first image in different channel colors and a second sub-image of the second image in the different channel colors.

[0188] In this embodiment, in order to further improve the test quality in the color dimension, the terminal device can perform three-channel color separation processing on the first image and the second image respectively to obtain a first sub-image of the first image in different channel colors and a second sub-image of the second image in different channel colors. The three-channel colors specifically refer to the three color channels of red (R), green (G), and blue (B).

[0189] In some possible embodiments, the terminal device may use the split function in OpenCV to separate the multi-channel arrays in the first image and the second image into three single-channel arrays, thereby obtaining a first sub-image of the first image in different channel colors and a second sub-image of the second image in different channel colors.

[0190] In S504, for any channel color, the similarity between the first sub-image and the second sub-image under the any channel color is calculated to obtain a second structural similarity index.

[0191] In this embodiment, for any channel color, the terminal device can compare the first sub-image and the second sub-image corresponding to the any channel color to detect the similarity between the first sub-image and the second sub-image, thereby obtaining the second structural similarity index under the any channel color.

[0192] In S505 , based on the second structural similarity index under the different channel colors, a test result of the user interface image under the color dimension is determined.

[0193] In this embodiment, after obtaining the second structural similarity indexes under different channel colors, the terminal device may compare each second structural similarity index with a set threshold.

[0194] In one embodiment of the present application, when the terminal device detects that each second structural similarity index is greater than or equal to a set threshold, it can determine that the similarity between the first image and the second image is extremely high. Therefore, the terminal device can determine that the test result of the user interface image in the color dimension is a passed test.

[0195] In another embodiment of the present application, when the terminal device detects that at least one second structural similarity index is less than a set threshold, it can determine that the similarity between the first image and the second image is low. Therefore, the terminal device can determine that the test result of the user interface image in the color dimension is that the test failed.

[0196] In some possible embodiments, when the terminal device determines that the test result of the user interface image in the color dimension is a test failure, the terminal device may record a portion of the target data that has a color difference.

[0197] In S303, the target data is input into a translation model for processing to obtain translation data corresponding to the target data.

[0198] In this embodiment, the terminal device may input the target data into the translation model for processing to obtain translation data corresponding to the target data.

[0199] It should be noted that the above translation model may be an existing multimodal machine translation (MMT), which will not be elaborated here.

[0200] In practical applications, MMT is a machine translation method for processing multimodal data. It can accept inputs in different forms such as images and audios and generate corresponding translated texts, or accept text inputs and generate corresponding image or audio outputs.

[0201] In S304, a target image including the translation data and the data content is output based on the data position, and the target image is determined as a test result of the user interface image in the translation dimension; the size of the target image is the same as that of the user interface image.

[0202] In this embodiment, after obtaining the translation data, in order to facilitate relevant personnel to conduct subsequent semantic checking tests of classical Chinese, the terminal device can draw a target image containing the translation data and data content on a blank canvas based on the data position of the target data, and output the target image.

[0203] For example, see Figure 8 , Figure 8 is a schematic diagram of a target image provided by an embodiment of the present application. Figure 8 As shown, Figure 8 (a) in the figure is the user interface image. Figure 8 (b) in the figure is the target image corresponding to (a) and containing the translation data.

[0204] Afterwards, the terminal device may determine the target image as the test result of the user interface image in the translation dimension, wherein the size of the target image is the same as the size of the user interface image.

[0205] From the above, it can be seen that a user interface testing method provided by an embodiment of the present application obtains a user interface image to be tested; performs text recognition on the user interface image to obtain target data; the target data includes data content and data position; the data position refers to the position of the target data in the user interface image; based on the data content and data position, the target data is tested in different dimensions to obtain test results of the user interface image in different dimensions; different dimensions include text dimensions and translation dimensions. Compared with the prior art, the present application can achieve accurate testing of the UI interface in dimensions such as text and translation by obtaining target data including data content and data position through text recognition, thereby improving the test quality. At the same time, it solves the problem of ordinary testers who do not have the function of small languages ​​to test small language UI pages, without the need for the intervention of professional language teams, expanding the scope of application and increasing practicality.

[0206] In one embodiment of the present application, after the terminal device obtains the test results of the user interface image in the different dimensions, in order to facilitate relevant personnel to promptly and clearly understand the areas where the user interface image has differences, the terminal device may specifically perform the following steps, which are described in detail as follows:

[0207] For any dimension, if the test result of any dimension is different from the target data under the historical version information, the target data is marked based on the data position, and the marked user interface image is output.

[0208] In this embodiment, for any dimension, when the terminal device detects that the test result of any dimension is different from the target data under the historical version information, that is, the test result of any dimension is that the test failed, the terminal device can mark the target data based on the data position of the target data and output the marked user interface image.

[0209] The marking processing methods include but are not limited to: highlighting the target data and marking the target data with a frame.

[0210] In some possible embodiments, in order to further clarify the specific dimension in which the target data failed the test, the terminal device can set a color corresponding to each dimension, and when it is detected that the test result of the target data in a certain dimension is that the test failed, the terminal device can mark the target data with the color corresponding to the dimension.

[0211] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0212] Corresponding to the user interface testing method described in the above embodiment, Fig. 9 The structure diagram of a user interface testing device provided by an embodiment of the present application is shown. For the convenience of description, only the part related to the embodiment of the present application is shown. Fig. 9 The user interface testing device 600 includes: a first acquisition unit 61, a first identification unit 62 and a first testing unit 63. Among them:

[0213] The first acquisition unit 61 is used to acquire a user interface image to be tested.

[0214] The first recognition unit 62 is used to perform text recognition on the user interface image to obtain target data; the target data includes data content and data position; the data position refers to the position of the target data in the user interface image.

[0215] The first testing unit 63 is used to perform tests on the target data in different dimensions based on the data content and the data location to obtain test results of the user interface image in the different dimensions; the different dimensions include a text dimension and a translation dimension.

[0216] In one embodiment of the present application, the first identification unit 62 specifically includes: a naming unit, a second identification unit and a first processing unit.

[0217] The naming unit is used to name the user interface image based on a naming rule to obtain an image file containing the image name.

[0218] The second recognition unit is used to input the image file into a text recognition model for recognition to obtain initial data; the initial data includes the image name, original content, data coordinates and data confidence.

[0219] The first processing unit is used to process the initial data based on the data processing rules to obtain the target data if the data confidence is greater than or equal to the set confidence.

[0220] In one embodiment of the present application, the first test unit 63 specifically includes: a second acquisition unit, a second test unit, a second processing unit and an output unit.

[0221] The second acquisition unit is used to acquire the historical version information corresponding to the target data with respect to the text dimension; the historical version information includes the historical content and historical position of the target data.

[0222] The second testing unit is used to test the target data based on the historical version information to obtain a test result of the user interface image in the text dimension.

[0223] The second processing unit is used for inputting the target data into the translation model for processing with respect to the translation dimension to obtain translation data corresponding to the target data.

[0224] The output unit is used to output a target image containing the translation data and the data content based on the data position, and determine the target image as a test result of the user interface image in the translation dimension; the size of the target image is the same as that of the user interface image.

[0225] In one embodiment of the present application, the target data includes target version information; the target data is at least two, the text dimension includes a content dimension, the content dimension includes a content quantity dimension and an actual content dimension; the second test unit specifically includes: a first determination unit, a first comparison unit and a second comparison unit. Wherein:

[0226] The first determining unit is used to determine the history quantity of the target data under the history version information.

[0227] The first comparison unit is used to compare the historical quantity with the quantity of target data under the target version information to obtain a test result of the user interface image under the content quantity dimension.

[0228] The second comparison unit is used to compare the data content of all target data under the target version information with the data content of all target data under the historical version information to obtain the test result of the user interface image under the actual content dimension.

[0229] In one embodiment of the present application, the data position includes the starting point, end point and center point of the target data, the historical position includes the historical starting point, historical end point and historical center point of the target data, the text dimension includes the position dimension, and the second test unit specifically includes: a third comparison unit, a fourth comparison unit and a second determination unit. Wherein:

[0230] The third comparison unit is used to compare the starting point, end point and center point of the target data with the historical starting point, the historical end point and the historical center point respectively to obtain a first result of the user interface image in the position dimension.

[0231] The fourth comparison unit is used to compare the font size of the data content with the font size of the historical content to obtain a second result of the user interface image in the position dimension;

[0232] The second determination unit is used to determine a test result of the user interface image in the position dimension based on the first result and the second result.

[0233] In one embodiment of the present application, the text dimension includes a font dimension, and the second testing unit specifically includes: a first cropping unit, a second cropping unit, a third processing unit, a first calculating unit, and a third determining unit.

[0234] The first cropping unit is used to crop the user interface image based on the data position to obtain a first image containing only the target data.

[0235] The second cropping unit is used to crop the historical interface image corresponding to the historical version information based on the historical position to obtain a second image containing only the target data.

[0236] The third processing unit is used to perform grayscale processing on the first image and the second image respectively to obtain a third image corresponding to the first image and a fourth image corresponding to the second image.

[0237] The first calculation unit is used to calculate the similarity between the third image and the fourth image to obtain a first structural similarity index.

[0238] The third determination unit is used to determine a test result of the user interface image in the font dimension based on the first structural similarity index.

[0239] In one embodiment of the present application, the text dimension includes a color dimension, and the second testing unit specifically includes: a third cropping unit, a fourth cropping unit, a fourth processing unit, a second calculating unit, and a fourth determining unit.

[0240] The third cropping unit is used to crop the user interface image based on the data position to obtain a first image containing only the target data.

[0241] The fourth cropping unit is used to crop the historical interface image corresponding to the historical version information based on the historical position to obtain a second image containing only the target data.

[0242] The fourth processing unit is used to perform three-channel color separation processing on the first image and the second image respectively to obtain a first sub-image of the first image in different channel colors and a second sub-image of the second image in the different channel colors.

[0243] The second calculation unit is used to calculate the similarity between the first sub-image and the second sub-image under any channel color to obtain a second structural similarity index.

[0244] The fourth determination unit is used to determine the test result of the user interface image in the color dimension based on the second structural similarity index under the different channel colors.

[0245] In one embodiment of the present application, the user interface testing device 600 further includes: a marking unit.

[0246] The marking unit is used to mark the target data based on the data position for any dimension, and output the marked user interface image if the test result of any dimension is different from the target data under the historical version information.

[0247] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0248] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0249] Fig.10 This is a schematic diagram of the structure of a terminal device provided in an embodiment of the present application. Fig.10 As shown, the terminal device 7 of this embodiment includes: at least one processor 70 ( Fig.10 Only one is shown in the figure), a memory 71, and a computer program 72 stored in the memory 71 and executable on the at least one processor 70, wherein the processor 70 implements the steps of any of the above-mentioned user interface testing method embodiments when executing the computer program 72.

[0250] The terminal device 7 may include, but is not limited to, a processor 70 and a memory 71. Those skilled in the art will appreciate that Fig.10 It is only an example of the terminal device 7 and does not constitute a limitation on the terminal device 7. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.

[0251] The processor 70 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0252] In some embodiments, the memory 71 may be an internal storage unit of the terminal device 7, such as the memory of the terminal device 7. In other embodiments, the memory 71 may also be an external storage device of the terminal device 7, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 7. Further, the memory 71 may also include both the internal storage unit of the terminal device 7 and an external storage device. The memory 71 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program. The memory 71 may also be used to temporarily store data that has been output or is to be output.

[0253] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0254] An embodiment of the present application provides a computer program product. When the computer program product runs on a terminal device, the terminal device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0255] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device capable of carrying the computer program code to the terminal device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0256] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0257] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A user interface testing method, characterized in that: include: Get the user interface image to be tested; Performing text recognition on the user interface image to obtain target data; The target data includes data content and data location; The data position refers to the position of the target data in the user interface image; The target data is tested in different dimensions based on the data content and the data location to obtain test results of the user interface image in the different dimensions; the different dimensions include a text dimension and a translation dimension.

2. The user interface testing method according to claim 1, characterized in that: The performing text recognition on the user interface image to obtain target data includes: Naming the user interface image based on a naming rule to obtain an image file containing the image name; Inputting the image file into a text recognition model for recognition to obtain initial data; the initial data includes the image name, original content, data coordinates, and data confidence; If the data confidence is greater than or equal to the set confidence, the initial data is processed based on the data processing rules to obtain the target data.

3. The user interface testing method according to claim 1, characterized in that: The testing of the target data in different dimensions based on the data content and the data position to obtain the test results of the user interface image in the different dimensions includes: For the text dimension, obtaining historical version information corresponding to the target data; the historical version information includes historical content and historical location of the target data; Testing the target data based on the historical version information to obtain a test result of the user interface image in the text dimension; With respect to the translation dimension, the target data is input into a translation model for processing to obtain translation data corresponding to the target data; A target image including the translation data and the data content is output based on the data position, and the target image is determined as a test result of the user interface image in the translation dimension; and a size of the target image is the same as a size of the user interface image.

4. The user interface testing method according to claim 3, characterized in that: The target data includes target version information; the target data is at least two, the text dimension includes a content dimension, and the content dimension includes a content quantity dimension and an actual content dimension; the test result of the user interface image under the content dimension is obtained according to the following method: Determine the historical quantity of target data under the historical version information; Compare the historical quantity with the quantity of target data under the target version information to obtain a test result of the user interface image under the content quantity dimension; The data content of all target data under the target version information is compared with the data content of all target data under the historical version information to obtain a test result of the user interface image under the actual content dimension.

5. The user interface testing method according to claim 3, characterized in that: The data position includes the starting point, the end point and the center point of the target data, the historical position includes the historical starting point, the historical end point and the historical center point of the target data, the text dimension includes the position dimension, and the test result of the user interface image under the position dimension is obtained according to the following method: Compare the starting point, the end point and the center point of the target data with the historical starting point, the historical end point and the historical center point respectively, to obtain a first result of the user interface image in the position dimension; Compare the font size of the data content with the font size of the historical content to obtain a second result of the user interface image in the position dimension; Based on the first result and the second result, a test result of the user interface image in the position dimension is determined.

6. The user interface testing method according to claim 3, characterized in that: The text dimension includes a font dimension, and the test result of the user interface image under the font dimension is obtained according to the following method: Performing cropping processing on the user interface image based on the data position to obtain a first image containing only the target data; Based on the historical position, a historical interface image corresponding to the historical version information is cropped to obtain a second image containing only the target data; Performing grayscale processing on the first image and the second image respectively to obtain a third image corresponding to the first image and a fourth image corresponding to the second image; Calculating the similarity between the third image and the fourth image to obtain a first structural similarity index; Based on the first structural similarity index, a test result of the user interface image in the font dimension is determined.

7. The user interface testing method according to claim 3, characterized in that: The text dimension includes a color dimension, and the test result of the user interface image under the color dimension is obtained according to the following method: Performing cropping processing on the user interface image based on the data position to obtain a first image containing only the target data; Based on the historical position, a historical interface image corresponding to the historical version information is cropped to obtain a second image containing only the target data; Performing three-channel color separation processing on the first image and the second image respectively to obtain a first sub-image of the first image in different channel colors and a second sub-image of the second image in the different channel colors; For any channel color, calculate the similarity between the first sub-image and the second sub-image under the any channel color to obtain a second structural similarity index; Based on the second structural similarity index under the different channel colors, a test result of the user interface image under the color dimension is determined.

8. The user interface testing method according to any one of claims 1 to 7, characterized in that: After the target data is tested in different dimensions based on the data content and the data position to obtain the test results of the user interface image in the different dimensions, the method further includes: For any dimension, if the test result of any dimension is different from the target data under the historical version information, the target data is marked based on the data position, and the marked user interface image is output.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the user interface testing method according to any one of claims 1 to 8 is implemented.

10. A computer program product, characterized in that It comprises a computer program, which implements the user interface testing method according to any one of claims 1 to 8 when being executed.

Images