Dial test method, device, electronic equipment and storage medium
By comparing pixel pigment on multiple sample page pictures of the application, determining the background color of the navigation bar and the focus coordinates of the functional blocks, the problem of difficulty in positioning the navigation bar in the existing technology is solved, and an efficient dialing method is realized.
Patent Information
- Application Number
- CN202311342356.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-16
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2043-10-16
AI Technical Summary
The prior art is difficult to accurately identify and locate the bottom navigation bar positions and functional blocks in applications, especially at different applications and resolutions, resulting in inefficient dialing.
By obtaining multiple sample page pictures, performing pixel pigment comparison, determining the background color of the navigation bar, and calculating the focus coordinates of the function block based on the color, thereby achieving accurate navigation bar and function block positioning.
It improves dialing efficiency, is suitable for massive applications and electronic devices with different resolutions, reducing the possibility of manual maintenance and error identification.
Smart Images

Figure CN117573508B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, and in particular to a dialing method, device, electronic device and storage medium. Background Art
[0002] Currently, the number of dial tests for the top applications in operation is large in each period, and the application coverage is wide, including not only e-commerce, music, video, finance, government and enterprise, communications, tourism and other industries. The full process of automatic dial test requires a quick switch to the required function interface through the bottom navigation bar. Among the related technologies, manual dial test, automatic element recognition, image recognition, relative coordinate recognition and other dial test technologies are mainly adopted. Due to the fact that some applications cannot obtain the UI properties of the navigation bar through conventional methods to prevent brushing, and there are differences in the block areas of the bottom navigation bar of each application, traditional dial test technology is difficult to identify the position of the bottom navigation bar, and it is difficult to locate the coordinates of the navigation bar function block. Therefore, how to accurately and quickly locate the navigation bar and function blocks at the bottom of the page has become a problem that needs to be solved urgently. Summary of the invention
[0003] In order to overcome the problems existing in the related art, the present disclosure provides a dialing method, device, electronic device and storage medium.
[0004] According to a first aspect of an embodiment of the present disclosure, a dialing method is provided, including:
[0005] Obtain multiple sample page images of the page to be tested on the first application;
[0006] Perform pixel color comparison on the plurality of sample page images to obtain a navigation bar image of the page to be tested;
[0007] Determine the background color of the navigation bar image;
[0008] Determine the focus coordinates of the function block in the navigation bar image according to the background color;
[0009] The function block is dialed based on the focus coordinates of the function block.
[0010] According to a second aspect of an embodiment of the present disclosure, a dialing device is provided, including:
[0011] A first acquisition module is used to acquire multiple sample page images of a page to be tested of a first application;
[0012] A second acquisition module is used to perform pixel color comparison on the plurality of sample page images to obtain a navigation bar image of the page to be tested;
[0013] A first determining module, used to determine the background color of the navigation bar image;
[0014] A second determination module, used to determine the focus coordinates of the function block in the navigation bar image according to the background color;
[0015] The dialing test module is used to perform a dialing test on the functional block based on the focus coordinates of the functional block.
[0016] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, characterized in that it includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the program, the method described in the first aspect above is implemented.
[0017] According to a fourth aspect of an embodiment of the present disclosure, there is provided a computer-readable storage medium on which a computer program is stored, wherein the computer program implements the method described in the first aspect when executed by a processor.
[0018] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: using the relatively fixed area unchanged features in multiple sample page images to accurately determine the navigation bar image of the page to be tested. Based on the background color combined with computer graphics processing technology, the function blocks in the navigation bar are flexibly, accurately and quickly located for dialing test. This method is suitable for dialing test of massive applications and can be applied to electronic devices with different resolutions to improve the efficiency of dialing test.
[0019] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.
[0021] Figure 1 The figure is a flow chart of a dialing method according to an exemplary embodiment.
[0022] Figure 2 The figure is a schematic diagram showing a page to be tested according to an exemplary embodiment.
[0023] Figure 3 The figure is a schematic diagram showing a static area at the bottom of a page to be tested according to an exemplary embodiment.
[0024] Figure 4 The figure is a flow chart of another dialing method according to an exemplary embodiment.
[0025] Figure 5 The flowchart of another dialing method is shown according to an exemplary embodiment.
[0026] Figure 6 The figure is a schematic diagram of a navigation bar image according to an exemplary embodiment.
[0027] Figure 7 It is a schematic diagram of a function block image according to an exemplary embodiment.
[0028] Figure 8 The figure is a structural block diagram of a dialing device according to an exemplary embodiment.
[0029] Fig. 9 is a block diagram of an electronic device 900 according to an exemplary embodiment. DETAILED DESCRIPTION
[0030] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.
[0031] It should be noted that since some of the application UI attributes of the bottom navigation bar of the application interface lack common attributes such as name and text, and the application attribute naming rules are set by each application, such as bottomNav, new_btns, etc., there is no universality, and the layout attributes LinearLayout and View are not unique. In order to prevent brushing, some applications cannot obtain the navigation bar UI attributes through conventional methods, so the UI control identification method cannot be used for matching. In addition, since the style, height, resolution, etc. of the bottom navigation bar of different application interfaces are different, and the upper boundary of some navigation bars is unclear, the specific rectangular range cannot be determined, so the conventional relative coordinate method cannot be used to determine the navigation bar position.
[0032] Regarding the problem of difficulty in locating the coordinates of the function blocks in the navigation bar, due to the common name and text attributes such as mine, me, not logged in, personal center, blank, etc., and like the navigation bar, some applications have anti-brushing UI elements that cannot be recognized. Because the number of applications is large and they change every issue, the maintenance workload of using traditional element recognition is large and the probability of abnormalities is high. The application icon style of the function blocks is maintained by each application, the image style is not uniform, the size is not uniform, the position is not uniform, and version updates will also cause the icons to be different. Therefore, the use of image positioning still has the problem of difficulty in locating the function blocks. In addition, because the number of block areas of the bottom navigation bar in different applications is different, the height and width are different, and the focus of the click area is different, relative positioning cannot accurately locate each function block.
[0033] To this end, the present disclosure proposes a dialing test method, device, electronic device and storage medium, which combine computer graphics processing technology to flexibly, accurately and quickly locate the function blocks in the navigation bar and then perform dialing test. Figure 1 is a flow chart of a dialing method according to an exemplary embodiment. It should be noted that the dialing method of the embodiment of the present disclosure can be used in the dialing device in the embodiment of the present disclosure, and the dialing device in the embodiment of the present disclosure can be configured in an electronic device. Figure 1 As shown, the dialing method may include but is not limited to the following steps.
[0034] Step 101: obtain multiple sample page images of pages to be tested on a first application.
[0035] It should be noted that the multiple sample page images can be different sample page images before and after the page to be tested slides. For example, based on the slidable nodes in the page to be tested, the page to be tested can be slid one or more times in a certain direction, and at least two images before and after the sliding are used as sample page images.
[0036] Since there may be sliding obstacles such as jump links in the dynamic area of the page to be tested, in order to avoid jumping from the page to be tested to other pages in the process of sliding the page to obtain the sample page image, in some embodiments of the present disclosure, the slidable nodes in the page to be tested can be accurately determined based on the number of bytes corresponding to the images and texts in the page to be tested and the number of bytes of the a tag link.
[0037] Figure 2 2 is a schematic diagram of a page to be tested according to an exemplary embodiment, wherein the page to be tested includes a dynamic area 201 and a static area (a status bar 202 located above the page and a navigation bar 203 located below the page). Figure 2 As shown, the dynamic area 201 in the page to be tested can be divided into multiple content blocks #1-1 to #3-3. Among them, dividing the dynamic area into #1-1 to #3-3 is only exemplary and does not constitute a limitation on the number of content blocks in the present disclosure. According to the number of bytes B1 corresponding to the pictures and texts in the multiple content blocks and the number of bytes of the a tag link A1, the label ratio S1 = A1 / (B1+A1) of each content block is obtained. If the label ratio S1 exceeds the preset label ratio threshold, it can be determined that there is a sliding obstacle within the content block range. Traverse #1-1 to #3-3, select the slidable content blocks whose label ratio S1 does not exceed the preset label ratio threshold, and prioritize the slidable content blocks according to the principle that the closer the content is to the center point of the screen, the higher the priority.
[0038] Then, the screen resolution length W1 and height H1, as well as the touchable range length W2 and height H2 in the actual user content block are obtained (some electronic devices may have untouchable areas such as black edges), thereby obtaining the touchable range of the dynamic area (W1-W2 ± R1)*(H1-H2 ± R2), where R1 and R2 are adjustment coefficients based on the byte concentration of the information content block. According to the priority ranking of the slidable content blocks and the touchable range, the slidable nodes in the page to be tested are determined. The page to be tested is slid based on the slidable nodes to obtain multiple sample page images before and after the page to be tested slides.
[0039] As an example, based on the slidable node, the page to be tested can be slid along the x-axis direction to obtain the first picture before the page to be tested slides and the second picture after the page to be tested slides. Based on the slidable node, the page to be tested can be slid along the y-axis direction to obtain the third picture before the page to be tested slides and the fourth picture after the page to be tested slides. Determine the similarity between the first picture, the second picture, the third picture, and the fourth picture, and use the two pictures with the smallest similarity as multiple sample page pictures. Using the two pictures with the largest difference as sample page pictures is conducive to determining a relatively fixed navigation bar image in subsequent steps.
[0040] Step 102 , performing pixel color comparison on a plurality of sample page images to obtain a navigation bar image of the page to be tested.
[0041] In one implementation, multiple sample page images can be scanned row by row from bottom to top to perform pixel color comparison, and a relatively fixed navigation bar image in the page to be tested is determined based on the color difference between the pixels at the same position of the multiple sample page images. Taking two sample page images as an example, a first color difference threshold is set, and it is determined whether the R channel difference, G channel difference, or B channel difference in RGB or RGBA format at the same coordinate position of the two images is greater than the first color difference threshold, abs(piexl[0]-bgPixel[0])>threshold or abs(piexl[1]-bgPixel[1])>threshold or abs(piexl[2]-bgPixel[2])>threshold, and the number of pixels in each row whose color difference is greater than or equal to the first color difference threshold is counted. If the number of pixels in a row scanned exceeds the first pixel ratio of the row, the navigation bar calculation is terminated, and the navigation bar image at the bottom of the page to be tested is generated based on the calculated navigation bar height and image width.
[0042] Step 103: determine the background color of the navigation bar image.
[0043] In some embodiments of the present disclosure, all pixels in the navigation bar image may be counted and stored, and the top N pigments therein may be found out as the background color of the navigation bar image.
[0044] Step 104: determining the focus coordinates of the function blocks in the navigation bar image according to the background color.
[0045] like Figure 2 As shown, the navigation bar 203 includes multiple function blocks ("Home", "Community", "Search", "Information" and "Personal"). Among them, the pigment of the pixel points of the text part in the function block is significantly different from the background color of the navigation bar image. Therefore, in some embodiments of the present disclosure, the navigation bar image can be scanned by column based on the background color. The blank area in the navigation bar image and the text area of each function block are divided according to the proportion of the background color in each column of pixels. The center point coordinates of the text area of each function block are used as the focus coordinates of the function block in the navigation bar image.
[0046] It should be noted that the navigation bar at the bottom of some application pages may have a border of a non-background color. To avoid interference caused by the non-background color border when scanning the navigation bar image by column based on the background color, the non-background color border in the navigation bar image can be removed in advance based on the background color. After removing the non-background color border, the navigation bar image is scanned by column based on the background color to determine the focus coordinates of the function block.
[0047] As an example, the pigment of each column in the navigation bar image can be error matched with the background color bgPixelN. If it is less than or equal to the second pigment difference threshold threshold2, the temporary count tempCount of each column is recorded. When the count of the background color in a single column is greater than the second pixel ratio of the navigation bar image height Iy, it means that the column does not belong to the non-background color border, and the x-axis starting position startX and the x-axis ending position endX of the border secondary border position are recorded. Scan the image from top to bottom and from left to right for each row, and error match the pigment of each row with the background color bgPixelN. If it is less than or equal to the second pigment difference threshold threshold2, the temporary count tempCount of each row is recorded. When the count of the background color in a single row is greater than the third pixel ratio of the navigation bar width Iw, it means that the row does not belong to the non-background color border, and the y-axis starting position startY and the y-axis ending position endY of the border secondary border position are recorded. According to the background color and the non-border area in the navigation bar image, scan from left to right range(startX,endX) and from top to bottom range(startY,endY), column by column, divide the functional blocks in the navigation bar image and determine the focus coordinates of the functional blocks.
[0048] Step 105: Perform dialing test on the function block based on the focus coordinates of the function block.
[0049] Optionally, in some embodiments of the present disclosure, the function block image in the navigation bar image can also be captured according to the background color. Compared with the image positioning in the traditional dialing technology, this embodiment does not require manual maintenance of the screenshot image, is suitable for massive applications, and saves manpower. The focus coordinates and function block images of the function block are stored in the screenshot sample library for subsequent massive application reuse, and can also provide a reference for the positioning of subsequent similar scenes.
[0050] As an example, the operating data of the first application, the mobile phone resolution, the focus coordinates of the function block and other data can be stored in the screenshot sample library, and the corresponding function block images captured can be automatically uploaded to the FTP server after classification. For the application that has been stored in the library, the focus coordinates of the function block can be directly used for positioning when it is subsequently accessed. When a new second application is accessed for dialing, it can be first determined whether there is a function block image in the screenshot sample library in the page to be tested of the second application (which can be determined by comparing the overlap). If the page to be tested on the second application includes the function block image in the screenshot sample library, the corresponding function block in the page to be tested on the second application can be clicked and dialed directly based on the focus coordinates of the corresponding function block in the screenshot sample library. If the dialing fails, the image recognition positioning method can be used for clicking. If the page to be tested on the second application does not include the function block image in the screenshot sample library, the steps in this embodiment can be used to locate the navigation bar function block.
[0051] It should also be noted that the dialing method proposed in the embodiment of the present disclosure is more applicable to scenarios where there are many elements and no regularity, and relative paths such as xpath and css are also prone to failure when the application is updated. This method can be identified without identifying the UI tree. In addition, compared with the relative position positioning in the traditional dialing technology, the dialing method of this embodiment is different from the relative position positioning in the traditional dialing technology. For mobile phone models with different resolutions, the styles of massive application designs are different, resulting in the need for manual maintenance of the identifiable area, which is a large workload. This method can be identified without relative positions.
[0052] By implementing the disclosed embodiments, the navigation bar image of the page to be tested is accurately determined by utilizing the relatively fixed area unchanged features in multiple sample page images. Based on the background color combined with computer graphics processing technology, the function blocks in the navigation bar are flexibly, accurately and quickly located for dialing test. This method is suitable for dialing test of massive applications and can be applied to electronic devices with different resolutions to improve the efficiency of dialing test.
[0053] It should be noted that the above embodiment determines the navigation bar image by comparing the bottom fixed area of multiple sample page images. Figure 3As shown, there are also cases where the area above the navigation bar does not belong to the navigation bar but belongs to the bottom fixed area, such as the player area above the navigation bar of music applications. Determining the navigation bar image by comparing the bottom fixed area in multiple sample page images may misjudge the fixed area above the navigation bar as the navigation bar. To this end, the present disclosure proposes another dialing method to accurately extract the navigation bar image in the page to be tested.
[0054] Figure 4 FIG. 1 is a flow chart of another dialing method according to an exemplary embodiment. Figure 4 As shown, the dialing method may include but is not limited to the following steps.
[0055] Step 401: Obtain a first sample page image and a second sample page image of a page to be tested on a first application.
[0056] Step 402: Obtain a first pixel value matrix of the first sample page image and a second pixel value matrix of the second sample page image.
[0057] Step 403 , performing pixel color comparison on the first pixel value matrix and the second pixel value matrix row by row from bottom to top, and obtaining the color difference of the pixel points at the same position in the first pixel value matrix and the second pixel value matrix.
[0058] Step 404 : Obtain the number of first pixel points whose color difference is greater than or equal to a first color difference threshold in each row of the first pixel value matrix and the second pixel value matrix.
[0059] Step 405: determine the static area at the bottom of the page to be tested according to the first number of pixels.
[0060] The number of first pixel points in each row whose color difference is greater than or equal to the first color difference threshold is counted. If the number of first pixel points in a row scanned exceeds the first pixel ratio of the row, the static area calculation ends.
[0061] Step 406, obtaining a third pixel value matrix of the static area.
[0062] Step 407 , counting the number of pixels of the same pigment in the bottom preset rectangular frame in the third pixel value matrix, and determining the pigment with the largest number of pixels in the preset rectangular frame as the first pigment.
[0063] As an example, a preset rectangular frame (such as Figure 3 The dotted box in the figure can be a rectangular box with a height of less than the preset pixel at the bottom of the page to be tested and a width of the page to be tested Iw. The pigment with the largest number of pixels in the preset rectangular box is determined as the first pigment, that is, the background color of the bottom area (navigation bar) of the page to be tested.
[0064] Step 408: Divide the static area into a navigation bar area and a non-navigation bar area based on the first pigment, and determine a navigation bar image according to the navigation bar area.
[0065] In one implementation, the static area can be scanned row by row from bottom to top according to the pixels to obtain the color difference between the pixels and the first pigment. A third color difference threshold threshold3 is set to determine whether the color difference between the pixels in the static area and the first pigment is less than or equal to the third color difference threshold threshold3, abs(piexl1[0]-bgPixelN[0])<=threshold3 and abs(piexl1[1]-bgPixelN[1])<=threshold3 and abs(piexl1[2]-bgPixelN[2])<=threshold3. The number of pixels in each row whose color difference between the pixels in the static area and the first pigment is greater than the third color difference threshold threshold3 is counted (i.e., the number of pixels in each row that are inconsistent with the first pigment is counted). If a row is scanned and there are continuous pixels that are inconsistent with the first pigment, and the number of pixels in the row that are inconsistent with the first pigment exceeds the third pixel ratio of the row, and the height of the row is greater than the navigation bar judgment height BH, the calculation of the navigation bar area ends, and the other areas in the static area are non-navigation bar areas.
[0066] by Figure 3 As an example of the embodiment shown in the figure, scan to y m If there are continuous pixels that are inconsistent with the first color in the row, and the number of inconsistent pixels exceeds the third pixel ratio of the row, the navigation bar area calculation is terminated. m-1 The row part is the navigation bar area, y m row-y M The row portion is the non-navigation bar area.
[0067] Step 409: determine the background color of the navigation bar image.
[0068] Optionally, in the embodiment of the present disclosure, the first pigment determined in step 407 may be used as the background color of the navigation bar image.
[0069] Step 410: determining the focus coordinates of the function block in the navigation bar image according to the background color.
[0070] Step 411, based on the focus coordinates of the function block, perform dialing test on the function block.
[0071] In the embodiment of the present disclosure, step 410 to step 411 may be implemented in any manner in the embodiments of the present disclosure, and the embodiment of the present disclosure does not limit this and will not be described in detail.
[0072] By implementing the disclosed embodiment, the static area at the bottom of the page to be tested is determined by utilizing the features of relatively fixed areas in a plurality of sample page images. The static area is scanned twice to determine the navigation bar image of the page to be tested, which can adapt to a variety of application scenarios and reduce the probability of misjudgment of the navigation bar. Based on the background color combined with computer graphics processing technology, the function blocks in the navigation bar are accurately located and then dialed. This method is suitable for dialing of massive applications and electronic devices with different resolutions, which improves the efficiency of dialing.
[0073] Figure 5 FIG. 1 is a flow chart of another dialing method according to an exemplary embodiment. Figure 5 As shown, the dialing method may include but is not limited to the following steps.
[0074] Step 501: obtain multiple sample page images of pages to be tested on a first application.
[0075] Step 502: Perform pixel color comparison on multiple sample page images to obtain a navigation bar image of the page to be tested.
[0076] Step 503: determine the background color of the navigation bar image.
[0077] Step 504: Obtain a fourth pixel value matrix of the navigation bar image.
[0078] Step 505 , scanning the fourth pixel value matrix column by column to determine whether the color of the pixel point in each column is consistent with the background color.
[0079] Step 506 , recording a first cumulative number of columns where the pixel pigment is consistent with the background color and a second cumulative number of columns where the pixel pigment is inconsistent with the background color.
[0080] Step 507, in response to scanning a column where the pixel pigment is inconsistent with the background color, clearing the first accumulated column number.
[0081] Step 508, in response to scanning a column in which the pixel pigment is consistent with the background color, clearing the second accumulated column number.
[0082] As an example, Figure 6This is a schematic diagram of a navigation bar image according to an exemplary embodiment. When scanning to the x0 position, the color of the pixel point in this column is consistent with the background color, and the first cumulative column number sameCount==1 is recorded. Continue to scan rightward column by column, and record the first cumulative column number sameCount++. Until scanning to the x1 position, the color of the pixel point in this column is inconsistent with the background color, and record the second cumulative column number diffCount==1. At this time, initialize the first cumulative column number, and clear the first cumulative column number sameCount==0 to ensure that it can be recorded normally when it is consistent with the background color next time. Continue to scan rightward column by column, and record the second cumulative column number diffCount++. When scanning to the x2 position, the color of the pixel point in this column is consistent with the background color, re-record the first cumulative column number sameCount==1, and clear the second cumulative column number diffCount==0. Similarly, the fourth pixel value matrix is scanned column by column through the polling mechanism until all pixels are scanned.
[0083] Step 509: taking the columns corresponding to when the first cumulative column number and the second cumulative column number are 1 as the first division points of the functional blocks.
[0084] That is to say, the number of columns where the pixel color is consistent with the background color for the first time and the number of columns where the pixel color is inconsistent with the background color for the first time in each round of scanning of the fourth pixel value matrix are used as the first segmentation point of the functional block. Figure 6 For example, x0, x1, x2, ..., x 12 As the first split point of the functional block.
[0085] In order to reduce the noise impact of color blocks and avoid occasional jitter, optionally, in some embodiments of the present disclosure, when the pigments of consecutive pixels are inconsistent with the background color for the first time in each round of scanning (such as 3 consecutive pixels are inconsistent with the background color) and / or the height of the inconsistent pixel positions is less than the preset pixel ratio of the navigation bar height, the corresponding column number is determined as the first segmentation point.
[0086] Step 510: Determine the focus coordinates of the function block in the navigation bar image based on the first segmentation point.
[0087] It should be noted that in order to avoid the situation where the same functional block is mistakenly divided into two functional blocks, in some embodiments of the present disclosure, the text spacing areas with too small spacing can be filtered out according to the spacing between the first segmentation points, thereby dividing the functional blocks in the navigation bar image.
[0088] by Figure 6For example, if the distance between the first segmentation point x2 and the first segmentation point x3 is less than the preset interval threshold, it can be determined that the area between x2 and x3 is a text interval area, and the first segmentation point x2 and the first segmentation point x3 corresponding to the interval are deleted. 12 , divide the functional blocks in the navigation bar image. Determine the functional block area according to the column pixels corresponding to the first segmentation point after filtering. If the column pixel color corresponding to the first segmentation point x0 is consistent with the background color, the area between the first segmentation point x0 and the next first segmentation point x1 is a blank area. If the column pixel color corresponding to the first segmentation point x1 is inconsistent with the background color, the area between the first segmentation point x1 and the next first segmentation point x4 is a functional block area. Thus, the left and right boundary arrays [x1~x4], [x5~x8] and [x9~x 12 ]. The focus coordinates of each function block can be confirmed based on the boundary array and the height of the navigation bar image.
[0089] As an example, the y-axis coordinates of the focus coordinates of each functional block are H1-Iy+(endY-startY) / 2, where H1 is the screen resolution, Iy is the navigation bar image height, startY-endY is the y-axis position range except the border, and the x-axis coordinate is the midpoint of the left and right boundary arrays of each functional block.
[0090] In some embodiments of the present disclosure, each functional block region may also be intercepted based on the left and right boundary arrays of each functional block, such as Figure 7 As shown, the function block images region1 to region3 are obtained. The function block images region1 to region3 and the focus coordinates of the corresponding function blocks are stored in the screenshot sample library for subsequent application dialing and positioning.
[0091] Step 511, based on the focus coordinates of the function block, perform dialing test on the function block.
[0092] In the embodiments of the present disclosure, steps 501 to 503 may be implemented in any manner in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.
[0093] By implementing the disclosed embodiment, the navigation bar image of the page to be tested is accurately determined by utilizing the relatively fixed area unchanged features in multiple sample page images. Based on the background color combined with computer graphics, the pixels in the navigation bar image are scanned through a polling mechanism, which can more accurately divide the functional blocks in the navigation bar, accurately locate the focal coordinates of the functional blocks, and then perform dialing. This method is suitable for dialing of massive applications and electronic devices with different resolutions, and improves the efficiency of dialing.
[0094] Figure 8 FIG. 1 is a structural block diagram of a dialing device according to an exemplary embodiment. Figure 8 As shown, the dialing detection device includes: a first acquisition module 801 , a second acquisition module 802 , a first determination module 803 , a second determination module 804 and a dialing detection module 805 .
[0095] The first acquisition module 801 is used to acquire multiple sample page images of a page to be tested of a first application.
[0096] The second acquisition module 802 is used to perform pixel color comparison on multiple sample page images to obtain the navigation bar image of the page to be tested.
[0097] The first determining module 803 is used to determine the background color of the navigation bar image.
[0098] The second determining module 804 is used to determine the focus coordinates of the function block in the navigation bar image according to the background color.
[0099] The dialing test module 805 is used to perform dialing test on the function block based on the focus coordinates of the function block.
[0100] In some embodiments of the present disclosure, the first acquisition module 801 is specifically used to: determine a slidable node in the page to be tested; slide the page to be tested based on the slidable node, and obtain multiple sample page images before and after the page to be tested slides.
[0101] In some embodiments of the present disclosure, the first acquisition module 801 is specifically used to: divide the dynamic area in the page to be tested into multiple content blocks; obtain the label ratio of each content block according to the number of bytes corresponding to the pictures and texts in the multiple content blocks and the number of bytes of the a tag link; obtain the touchable range of the dynamic area; determine the slidable node according to the label ratio and touchable range of each content block.
[0102] In some embodiments of the present disclosure, the first acquisition module 801 is specifically used to: based on a slidable node, slide the page to be tested along the x-axis direction to obtain a first image before the page to be tested slides and a second image after the page to be tested slides; based on the slidable node, slide the page to be tested along the y-axis direction to obtain a third image before the page to be tested slides and a fourth image after the page to be tested slides; determine the similarity between any two images in the first image, the second image, the third image, and the fourth image, and use the two images with the smallest similarity as multiple sample page images.
[0103] In some embodiments of the present disclosure, the plurality of sample page images include a first sample page image and a second sample page image. The second acquisition module 802 is specifically used to: acquire a first pixel value matrix of the first sample page image and a second pixel value matrix of the second sample page image; perform pixel color comparison on the first pixel value matrix and the second pixel value matrix row by row from bottom to top to acquire the color difference of pixels at the same position in the first pixel value matrix and the second pixel value matrix; acquire the number of first pixels in each row of the first pixel value matrix and the second pixel value matrix whose color difference is greater than a first color difference threshold; and determine the navigation bar image of the page to be tested according to the number of first pixels.
[0104] In some embodiments of the present disclosure, the second acquisition module 802 is specifically used to: determine the static area at the bottom of the page to be tested based on the first number of pixel points; obtain a third pixel value matrix of the static area; count the number of pixels of the same pigment in a preset rectangular box at the bottom of the third pixel value matrix, and determine the pigment with the largest number of pixels in the preset rectangular box as the first pigment; divide the static area into a navigation bar area and a non-navigation bar area based on the first pigment; and determine the navigation bar image based on the navigation bar area.
[0105] In some embodiments of the present disclosure, the second determination module 804 is specifically used to: obtain a fourth pixel value matrix of the navigation bar image; scan the fourth pixel value matrix column by column to determine whether the pixel color in each column is consistent with the background color; record a first cumulative column number in which the pixel color is consistent with the background color and a second cumulative column number in which the pixel color is inconsistent with the background color; in response to scanning a column in which the pixel color is inconsistent with the background color, clear the first cumulative column number; in response to scanning a column in which the pixel color is consistent with the background color, clear the second cumulative column number; use the column corresponding to when the first cumulative column number and the second cumulative column number are 1 as the first segmentation point of the functional block; and determine the focus coordinates of the functional block in the navigation bar image based on the first segmentation point.
[0106] In some embodiments of the present disclosure, the second determining module 804 is further configured to: remove the border of non-background color in the navigation bar image based on the background color.
[0107] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.
[0108] Fig. 9 is a block diagram of an electronic device 900 according to an exemplary embodiment.
[0109] like Fig. 9 As shown, the electronic device 900 includes:
[0110] The memory 901 and the processor 902 , and the bus 903 connecting different components (including the memory 901 and the processor 902 ), the memory 901 stores a computer program, and when the processor 902 executes the program, the dialing method of the embodiment of the present disclosure is implemented.
[0111] Bus 903 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor or a local bus using any of a variety of bus structures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnect (PCI) bus.
[0112] The electronic device 900 typically includes a variety of electronic device readable media. These media can be any available media that can be accessed by the electronic device 900, including volatile and non-volatile media, removable and non-removable media.
[0113] The memory 901 may also include computer system readable media in the form of volatile memory, such as random access memory (RAM) 904 and / or cache memory 905. The electronic device 900 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 906 may be used to read and write non-removable, non-volatile magnetic media ( Fig. 9 not shown, usually called a "hard drive"). Although Fig. 9 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 903 via one or more data medium interfaces. Memory 901 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present disclosure.
[0114] A program / utility 908 having a set (at least one) of program modules 907 may be stored, for example, in the memory 901, such program modules 907 including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination thereof may include an implementation of a network environment. The program modules 907 generally perform the functions and / or methods of the embodiments described in the present disclosure.
[0115] The electronic device 900 may also communicate with one or more external devices 909 (e.g., keyboard, pointing device, display 911, etc.), one or more devices that enable a user to interact with the electronic device 900, and / or any device that enables the electronic device 900 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication may be performed through an input / output (I / O) interface 912. Furthermore, the electronic device 900 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 913. Fig. 9 As shown, the network adapter 913 communicates with other modules of the electronic device 900 via the bus 903. It should be understood that although Fig. 9 Not shown, other hardware and / or software modules may be used in conjunction with electronic device 900, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0116] The processor 902 executes various functional applications and data processing by running the programs stored in the memory 901 .
[0117] It should be noted that the implementation process and technical principles of the electronic device of this embodiment refer to the aforementioned explanation of the dialing method of the embodiment of the present disclosure, which will not be repeated here.
[0118] In order to implement the above embodiments, the present disclosure also proposes a computer-readable storage medium.
[0119] When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device can perform the above-mentioned dialing method. Optionally, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.
[0120] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.
[0121] In the technical solution of the present disclosure, the acquisition, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the relevant laws and regulations and do not violate public order and good morals.
[0122] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims
1. A dialing method, characterized in that: include: Obtain multiple sample page images of the page to be tested on the first application; Perform pixel color comparison on the plurality of sample page images to obtain a navigation bar image of the page to be tested; Determine the background color of the navigation bar image; Determine the focus coordinates of the function block in the navigation bar image according to the background color; Based on the focal coordinates of the functional block, dialing the functional block; The step of determining the focus coordinates of the function block in the navigation bar image according to the background color includes: Obtaining a fourth pixel value matrix of the navigation bar image; Scanning the fourth pixel value matrix column by column to determine whether the color of the pixel points in each column is consistent with the background color; Recording a first cumulative number of columns where the pixel pigment is consistent with the background color and a second cumulative number of columns where the pixel pigment is inconsistent with the background color; In response to scanning a column in which the pixel pigment is inconsistent with the background color, clearing the first accumulated column number; In response to scanning a column whose pixel pigments are consistent with the background color, clearing the second accumulated column number; using the columns corresponding to when the first cumulative column number and the second cumulative column number are 1 as the first segmentation points of the functional block; Based on the first segmentation point, determining the focus coordinates of the function block in the navigation bar image; According to the spacing between the first segmentation points, the text spacing areas with too small spacing are filtered out.
2. The method according to claim 1, characterized in that The step of obtaining a plurality of sample page images of the first application to be tested page includes: Determine the slidable nodes in the page to be tested; The page to be tested is slid based on the slidable node to obtain a plurality of sample page images before and after the page to be tested slides.
3. The method according to claim 2, characterized in that The determining of the slidable nodes in the page to be tested includes: Dividing the dynamic area in the page to be tested into multiple content blocks; According to the number of bytes corresponding to the pictures and texts in the multiple content blocks and the number of bytes of the a-label links, obtaining the label ratio of each of the content blocks; Acquire the touchable range of the dynamic area; The slidable nodes are determined according to the label ratio of each content block and the touchable range.
4. The method according to claim 2, characterized in that: The step of sliding the page to be tested based on the slidable node to obtain a plurality of sample page images before and after the page to be tested slides includes: Based on the slidable node, slide the page to be tested along the x-axis direction to obtain a first picture before the page to be tested slides and a second picture after the page to be tested slides; Based on the slidable node, slide the page to be tested along the y-axis direction to obtain a third picture before the page to be tested slides and a fourth picture after the page to be tested slides; Determine the similarity between any two of the first picture, the second picture, the third picture, and the fourth picture, and use the two pictures with the smallest similarity as the multiple sample page pictures.
5. The method according to claim 1, characterized in that The plurality of sample page images include a first sample page image and a second sample page image; and performing pixel color matching on the plurality of sample page images to obtain the navigation bar image of the page to be tested includes: Obtaining a first pixel value matrix of the first sample page image and a second pixel value matrix of the second sample page image; Performing pixel color comparison on the first pixel value matrix and the second pixel value matrix row by row from bottom to top to obtain the color difference of the pixel points at the same position in the first pixel value matrix and the second pixel value matrix; Obtaining the number of first pixel points whose color difference is greater than a first color difference threshold in each row of the first pixel value matrix and the second pixel value matrix; The navigation bar image of the page to be tested is determined according to the first number of pixels.
6. The method according to claim 5, characterized in that The determining of the navigation bar image of the page to be tested according to the first number of pixels includes: Determine a static area at the bottom of the page to be tested according to the first number of pixels; Acquire a third pixel value matrix of the static area; Counting the number of pixel points of the same pigment in a preset rectangular frame at the bottom of the third pixel value matrix, and determining the pigment with the largest number of pixel points in the preset rectangular frame as the first pigment; Dividing the static area into a navigation bar area and a non-navigation bar area based on the first pigment; The navigation bar image is determined according to the navigation bar area.
7. The method according to claim 1, characterized in that Before determining the focus coordinates of the function block in the navigation bar image according to the background color, the method further includes: Based on the background color, a border of a non-background color in the navigation bar image is removed.
8. The method according to claim 1, characterized in that Also include at least one of: intercepting a function block image in the navigation bar image according to the background color; Storing the focus coordinates of the function block and the image of the function block in a screenshot sample library; In response to the page to be tested on the second application including the function block image in the screenshot sample library, the corresponding function block in the page to be tested on the second application is tested based on the focus coordinates of the corresponding function block in the screenshot sample library.
9. A dialing device, characterized in that: include: A first acquisition module is used to acquire multiple sample page images of a page to be tested of a first application; A second acquisition module is used to perform pixel color comparison on the plurality of sample page images to obtain a navigation bar image of the page to be tested; A first determining module, used to determine the background color of the navigation bar image; A second determination module, used to determine the focus coordinates of the function block in the navigation bar image according to the background color; A dialing test module, used for performing a dialing test on the functional block based on the focus coordinates of the functional block; The second determining module is specifically used for: Obtaining a fourth pixel value matrix of the navigation bar image; Scanning the fourth pixel value matrix column by column to determine whether the color of the pixel points in each column is consistent with the background color; Recording a first cumulative number of columns where the pixel pigment is consistent with the background color and a second cumulative number of columns where the pixel pigment is inconsistent with the background color; In response to scanning a column in which the pixel pigment is inconsistent with the background color, clearing the first accumulated column number; In response to scanning a column whose pixel pigments are consistent with the background color, clearing the second accumulated column number; using the columns corresponding to when the first cumulative column number and the second cumulative column number are 1 as the first segmentation points of the functional block; Based on the first segmentation point, determining the focus coordinates of the function block in the navigation bar image; According to the spacing between the first segmentation points, the text spacing areas with too small spacing are filtered out.
10. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 8 is implemented.
11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.
