Webpage picture splicing method and device

Abstract

The invention discloses a webpage picture splicing method and device, and relates to the technical field of software testing. According to one specific embodiment, the method comprises the steps that in response to a received picture splicing request, a first-screen picture and a non-first-screen picture of a webpage to be spliced are obtained, and the first-screen picture and the non-first-screen picture are partially overlapped; respectively identifying first screen position information of each picture composition element in the first screen picture and non-first screen position information of each picture composition element in the non-first screen picture; according to the first-screen position information of each picture composition element in the first-screen picture and the non-first-screen position information of each picture composition element in the non-first-screen picture, searching overlapped picture composition elements where the first-screen picture and the non-first-screen picture are overlapped, and further determining a splicing point of the first-screen picture and the non-first-screen picture; and according to the splicing points, performing de-duplication splicing on the first-screen picture and the non-first-screen picture to obtain a webpage picture. According to the embodiment, accurate picture deduplication splicing is achieved, and a complete and reliable webpage picture is provided for subsequent compatibility testing.

Description

Technical Field

[0001] The present invention relates to the technical field of software testing, and in particular to a method and device for splicing web page images. Background Art

[0002] Software compatibility testing aims to verify the compatibility and stability of software or systems in different environments, platforms, devices, or configurations. For example, it can compare whether web page images display properly on mobile phones with different resolutions. Existing compatibility testing typically directly obtains the image displayed on the first screen of the software on the device and performs compatibility testing by configuring different resolutions and other indicators.

[0003] In the process of implementing the present invention, the inventors found that the prior art has the following problems:

[0004] In actual testing, the first screen of each device is often unable to display the complete web page image. In addition, due to the influence of screen size and resolution, the first screen of different devices will have different content when displaying the same web page. It is unreasonable to fail the compatibility test simply because of the difference in screen size and resolution. Summary of the Invention

[0005] In view of this, an embodiment of the present invention provides a method and device for splicing web page images, which realizes a method for accurately deduplicating and splicing web page images, avoids the situation where compatibility testing fails due to differences in screen size and resolution, and provides complete and reliable web page comparison images for subsequent software compatibility testing.

[0006] To achieve the above object, according to a first aspect of an embodiment of the present invention, a method for splicing web page images is provided, comprising:

[0007] In response to receiving the image stitching request, obtaining a home screen image and a non-home screen image of the web page to be stitched, wherein the home screen image and the non-home screen image partially overlap;

[0008] Respectively identifying the first-screen position information of each image component element in the first-screen image and the non-first-screen position information of each image component element in the non-first-screen image;

[0009] According to the first-screen position information of each image component element in the first-screen image and the non-first-screen position information of each image component element in the non-first-screen image, searching for overlapping image component elements where the first-screen image and the non-first-screen image overlap, and determining the splicing point between the first-screen image and the non-first-screen image according to the overlapping image component elements;

[0010] According to the splicing points, the first screen image and the non-first screen images are deduplicated and spliced ​​to obtain a web page image.

[0011] Optionally, before respectively identifying the first-screen position information of each picture component element in the first-screen image and the non-first-screen position information of each picture component element in the non-first-screen image, the method further includes: performing grayscale processing, one-dimensional flattening processing and sparse processing on the first-screen image and the non-first-screen image, respectively, to obtain a one-dimensional first-screen image and a one-dimensional non-first-screen image; the respectively identifying the first-screen position information of each picture component element in the first-screen image and the non-first-screen position information of each picture component element in the non-first-screen image includes: calculating the first-screen position information of each picture component element in the first-screen image and the non-first-screen position information of each picture component element in the non-first-screen image based on the one-dimensional first-screen image and the one-dimensional non-first-screen image.

[0012] Optionally, based on the one-dimensional first screen image and the one-dimensional non-first screen image, the first screen position information of each picture component element in the first screen image and the non-first screen position information of each picture component element in the non-first screen image are calculated, including: performing averaging processing on the one-dimensional first screen image and the one-dimensional non-first screen image respectively to obtain a one-dimensional mean first screen image and a one-dimensional mean non-first screen image; based on the one-dimensional mean first screen image and the size of the first screen image, calculating the position starting point and position end point of each picture component element in the first screen image in the non-first screen; based on the one-dimensional mean non-first screen image and the size of the non-first screen image, calculating the position starting point and position end point of each picture component element in the non-first screen image in the non-first screen.

[0013] Optionally, based on the first-screen position information of each picture component element in the first-screen image and the non-first-screen position information of each picture component element in the non-first-screen image, the overlapping picture component elements where the first-screen image and the non-first-screen image overlap are searched, including: extracting the picture component elements of the first-screen image based on the position starting point and position ending point of each picture component element in the first-screen image on the first screen; extracting the picture component elements of the non-first-screen image based on the position starting point and position ending point of each picture component element in the non-first-screen image on the non-first-screen; using a matching algorithm to compare and match the picture component elements of the first-screen image with the picture component elements of the non-first-screen image; and determining the overlapping picture component elements where the first-screen image and the non-first-screen image overlap based on the comparison and matching results.

[0014] Optionally, based on the overlapping picture component elements, the splicing point of the first screen image and the non-first screen image is determined, including: based on the position information corresponding to the overlapping picture component elements, searching for the target overlapping picture component element located at the last position of the first screen image from the overlapping picture component elements, and using the target overlapping picture component element as the splicing point of the first screen image and the non-first screen image.

[0015] Optionally, according to the splicing point, the first screen image and the non-first screen image are deduplicated and spliced, including: taking the image above the position of the splicing point in the first screen image as the upper splicing image, and taking the image below the position of the splicing point in the non-first screen image as the lower splicing image; and deduplicating and splicing the upper splicing image, the splicing point and the lower splicing image using an image splicing algorithm.

[0016] Optionally, the image elements are laid out in the form of web page floors.

[0017] Optionally, when there are multiple non-first screen images, the method further includes: according to the order of each non-first screen image, using any method of the first aspect of the above-mentioned embodiment of the present invention, iteratively deduplicating and splicing each non-first screen image and the first screen image in turn.

[0018] According to a second aspect of an embodiment of the present invention, there is provided an apparatus for splicing web page images, comprising:

[0019] A module for obtaining images to be spliced, configured to obtain, in response to receiving an image splicing request, a home screen image and non-home screen images of the web page to be spliced, wherein the home screen image and the non-home screen images partially overlap;

[0020] A position information identification module, configured to respectively identify the first-screen position information of each image component element in the first-screen image and the non-first-screen position information of each image component element in the non-first-screen image;

[0021] a splicing point determination module, configured to search for overlapping image components where the first screen image and the non-first screen image overlap based on the first screen position information of each image component element in the first screen image and the non-first screen position information of each image component element in the non-first screen image, and determine the splicing point between the first screen image and the non-first screen image based on the overlapping image component elements;

[0022] The deduplication and splicing module is used to deduplication and splice the first screen image and the non-first screen image according to the splicing points to obtain a web page image.

[0023] According to a third aspect of an embodiment of the present invention, there is provided an electronic device for splicing web page images, comprising:

[0024] one or more processors;

[0025] a storage device for storing one or more programs,

[0026] When the one or more programs are executed by the one or more processors, the one or more processors implement the method provided in the first aspect of the embodiment of the present invention.

[0027] According to a fourth aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored. When the program is executed by a processor, the method provided by the first aspect of the embodiment of the present invention is implemented.

[0028] One embodiment of the invention has the following advantages or beneficial effects: by responding to receiving a request for image splicing, obtaining the first screen image and the non-first screen image of the web page to be spliced, the first screen image and the non-first screen image partially overlap; respectively identifying the first screen position information of each image component element in the first screen image and the non-first screen position information of each image component element in the non-first screen image; according to the first screen position information of each image component element in the first screen image and the non-first screen position information of each image component element in the non-first screen image, finding the overlapping image component elements where the first screen image and the non-first screen image overlap, and determining the splicing points of the first screen image and the non-first screen image according to the overlapping image component elements; according to the splicing points, deduplicating and splicing the first screen image and the non-first screen image to obtain a technical solution for web page images, realizing a precise method for deduplicating and splicing web page images, avoiding the situation where the compatibility test fails due to differences in screen size and resolution, and providing complete and reliable web page comparison images for subsequent software compatibility testing. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings are provided for a better understanding of the present invention and are not intended to limit the present invention.

[0030] Figure 1 is a schematic diagram of the main process of the method for splicing web page images according to an embodiment of the present invention;

[0031] FIG2( a ) is a schematic diagram of the first screen of an embodiment of the present invention;

[0032] FIG2( b ) is a schematic diagram of a one-dimensional first screen image according to an embodiment of the present invention;

[0033] FIG2( c ) is a schematic diagram of a one-dimensional mean first screen image according to an embodiment of the present invention;

[0034] Figure 3 1 is a schematic diagram of a comparison and matching process of image components according to an embodiment of the present invention;

[0035] Figure 4 is a schematic diagram of splicing points according to an embodiment of the present invention;

[0036] Figure 5 Detailed flowchart of the method for splicing web page images according to an embodiment of the present invention;

[0037] Figure 6 2 is a schematic diagram of main modules of a device for splicing web page images according to an embodiment of the present invention;

[0038] Figure 7is an exemplary system architecture diagram in which embodiments of the present invention may be applied;

[0039] Figure 8 It is a schematic diagram of the structure of a computer system of a terminal device or server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0040] It should be noted that the acquisition, storage and application of user personal information involved in the technical solution of this disclosure are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0041] The following description of exemplary embodiments of the present invention is made in conjunction with the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding. These details should be considered as merely exemplary. Therefore, it should be appreciated by those skilled in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0042] In existing software compatibility tests, the home screen of each device is often unable to display the complete web page image. In addition, due to the influence of screen size and resolution, the home screens of different devices may display different content when displaying the same web page. It is unreasonable to fail the compatibility test due to differences in screen size and resolution.

[0043] In order to solve the above-mentioned problems existing in the prior art, the present invention proposes a method for splicing web page images. Before performing software compatibility testing, based on the first-screen position information of each image component element in the identified first-screen image and the non-first-screen position information of each image component element in the non-first-screen image, the overlapping image component elements of the first-screen image and the non-first-screen image are searched, and then the splicing points of the first-screen image and the non-first-screen image are determined based on the overlapping image component elements, so as to de-duplicate and splice the first-screen image and the non-first-screen image according to the splicing points to obtain a complete web page image. A precise method for de-duplicate and splicing web page images is implemented, which avoids the situation where the compatibility test fails due to differences in screen size and resolution, and provides complete and reliable web page comparison images for subsequent software compatibility testing.

[0044] Figure 1 FIG. 1 is a schematic diagram of the main process of the method for splicing web page images according to an embodiment of the present invention. Figure 1 As shown, the web page image splicing method according to the embodiment of the present invention includes the following steps S101 to S103.

[0045] Step S101: in response to receiving a request for image splicing, obtaining a home screen image and a non-home screen image of a web page to be spliced, wherein the home screen image and the non-home screen image partially overlap.

[0046] Specifically, the embodiment of the present invention is based on a cloud-based testing platform for real cloud devices, taking mobile phone devices as an example. According to the test needs, the software compatibility tester uses a cross-platform UI automation testing tool based on image recognition through the cloud-based testing platform to perform screen sliding and screenshot operations on the real cloud device provided by the platform, and obtains the first screen image and non-first screen image of the web page to be spliced. It can be understood that the first screen image and the non-first screen image are generated by the sliding action of simulated browsing, and the first screen image and the non-first screen image will partially overlap in display content. For example, for shopping software, its first screen image often includes image components such as navigation bar, category button, special offer area, and transaction item list, among which the transaction item list may only display the first few transaction items. Accordingly, the non-first screen after the sliding operation may include a special offer area and a transaction item list, wherein the transaction item list not only includes the transaction items displayed in the first screen, but also displays several subsequent transaction items.

[0047] Step S102: Identify the first-screen position information of each image component in the first-screen image and the non-first-screen position information of each image component in the non-first-screen image.

[0048] Specifically, different web page images have different layout rules. For the first-screen image and non-first-screen images to be spliced ​​obtained by the embodiment of the present invention, it is first necessary to clarify the layout characteristics of each image component element in the first-screen image, that is, the position information, as well as the position information of each image component element in the non-first-screen image. Through a pre-determined position information recognition algorithm, the first-screen image and the non-first-screen image are identified by image text, etc., to obtain the first-screen position information of each image component element in the first-screen image, as well as the non-first-screen position information of each image component element in the non-first-screen image.

[0049] According to one embodiment of the present invention, before respectively identifying the first-screen position information of each picture component element in the first-screen image and the non-first-screen position information of each picture component element in the non-first-screen image, the method further includes: performing grayscale processing, one-dimensional flattening processing and sparse processing on the first-screen image and the non-first-screen image respectively to obtain a one-dimensional first-screen image and a one-dimensional non-first-screen image; the respectively identifying the first-screen position information of each picture component element in the first-screen image and the non-first-screen position information of each picture component element in the non-first-screen image includes: calculating the first-screen position information of each picture component element in the first-screen image and the non-first-screen position information of each picture component element in the non-first-screen image based on the one-dimensional first-screen image and the one-dimensional non-first-screen image.

[0050] Specifically, in order to improve the efficiency of recognizing the position information of each image component element, before respectively identifying the first-screen position information of each image component element in the first-screen image and the non-first-screen position information of each image component element in the non-first-screen image, the first-screen image and the non-first-screen image can be preprocessed. For example, the first-screen image and the non-first-screen image can be grayscale processed respectively by the color space conversion method in the open source computer vision library to obtain a two-dimensional first-screen image and a two-dimensional non-first-screen image, but it is not limited to this; then, the two-dimensional first-screen image and the two-dimensional non-first-screen image can be one-dimensionally flattened by the one-dimensional conversion method in the specified programming language library, but it is not limited to this; finally, considering that there are still relatively many pixels in the first-screen image and the non-first-screen image after the one-dimensional flattening processing, in order to further reduce unnecessary pixels, the first-screen image and the non-first-screen image after the one-dimensional flattening processing are subjected to sparse processing in the image processing technology to obtain a sparsed one-dimensional first-screen image and a one-dimensional non-first-screen image. Based on the one-dimensional first screen image and the one-dimensional non-first screen image obtained after preprocessing, the first screen position information of each image component element in the subsequent first screen image and the non-first screen position information of each image component element in the non-first screen image are calculated.

[0051] According to another embodiment of the present invention, based on the one-dimensional first screen image and the one-dimensional non-first screen image, the first screen position information of each picture component element in the first screen image and the non-first screen position information of each picture component element in the non-first screen image are calculated, including: respectively performing averaging processing on the one-dimensional first screen image and the one-dimensional non-first screen image to obtain a one-dimensional average first screen image and a one-dimensional average non-first screen image; based on the one-dimensional average first screen image and the size of the first screen image, calculating the position starting point and position end point of each picture component element in the first screen image in the non-first screen according to the one-dimensional average non-first screen image and the size of the non-first screen image.

[0052] Specifically, according to the distribution characteristics of web page images, the above-mentioned pre-processed one-dimensional first screen image and one-dimensional non-first screen image are averaged, and the pixel mean of the first screen image and the pixel mean of the non-first screen image are calculated. Then, each pixel in the first screen image is compared with the corresponding pixel mean, and the pixels less than the mean are regarded as 0, and the pixels not less than the mean are regarded as 1, to obtain a one-dimensional mean first screen image of the first screen image; similarly, each pixel in the non-first screen is also compared with the pixel mean of the non-first screen image to obtain a one-dimensional mean non-first screen image. The one-dimensional mean first screen image and the one-dimensional mean non-first screen image that have been averaged can intuitively present the layout rules of the first screen image and the non-first screen image. It can be understood that the area with continuous 1s is usually a picture component element, and the area with continuous 0s is usually the transition area between two picture components. Record the starting point and ending point of each continuous area of ​​1 in the one-dimensional mean first screen image, and the starting point and ending point of each continuous area of ​​0, combined with the total number of picture pixels of the current device, the picture size (horizontal width and vertical height), and then calculate the starting point and ending point of each picture component element in the first screen image based on the distribution characteristics of the web page pictures. Similarly, the starting point and ending point of each picture component element in the non-first screen image are also calculated through the starting point and ending point of each continuous area of ​​1 in the one-dimensional mean non-first screen image, and the starting point and ending point of each continuous area of ​​0, combined with the total number of picture pixels of the current device and the size of the non-first screen image. It can be understood that the size of the non-first screen image and the size of the first screen image of the same device are the same.

[0053] According to yet another embodiment of the present invention, the image components are arranged in the form of web page floors.

[0054] Specifically, the embodiment of the present invention uses web page floors as an example to perform deduplication and splicing of web page floors. The image components of the first-screen image or non-first-screen image are laid out in the form of web page floors. Accordingly, the first-screen position information of the image components of the first-screen image is equivalent to the first-screen position information of the floor in the first-screen image, and the non-first-screen position information of the image components of the non-first-screen image is equivalent to the non-first-screen position information of the floor in the non-first-screen image. Each image component is equivalent to the content in each floor. A web page floor is a specific layout unit for each visual component element in a web page image. Since web page images are browsed vertically, different components are located at different positions in the web page, which can be vividly described by floors. For example, in a web page image that displays a list of trading items, from the perspective of the vertical layout, the layout of each visual component element of the image in the web page is neatly and regularly arranged from top to bottom. For a specific example, see Figure 2(a).

[0055] Figure 2(a) is a schematic diagram of the first screen image of an embodiment of the present invention. The various image components in the first screen image are arranged according to the structure of the web page floors. It can be seen that the first screen is divided into 6 floors, as shown in the white oval box in the figure. The list of gold list trading items starts from the 4th floor from the top to the bottom. The 6th floor is truncated due to the screen size limitation and only a part is retained. Figure 2(b) is a schematic diagram of a one-dimensional first screen image of an embodiment of the present invention. The one-dimensional first screen image corresponds to the first screen image of Figure 2(a), and is a one-dimensional representation obtained after preprocessing Figure 2(a). Figure 2(c) is a schematic diagram of a one-dimensional mean first screen image of an embodiment of the present invention, and is obtained after averaging Figure 2(b). It can be seen from Figure 2(c) that the areas with continuous 1 correspond to the floors from top to bottom in Figure 2(a). Figure 2(c) intuitively presents the layout rules of the first screen image.

[0056] For example, based on the first screen image of Figure 2(a) above, the total number of pixels of the image is set to X, the horizontal width of the first screen image is W, and the vertical height is H. By default, the width of the entire floor is the width of the image. Then, the core of the position start point and the position end point is to determine the starting height and the ending height of the floor. For example, at the upper left cross mark and the lower right cross mark of the fourth floor in Figure 2(a), the specific calculation formula is:

[0057]

[0058]

[0059] Where ImgH1 is the starting height of the floor, ImgH2 is the ending height of the floor, Xbegin represents the pixel value corresponding to the starting point of the peak (mean value is 1) of the floor to be calculated in the total pixels of the one-dimensional mean first screen image, and correspondingly, Xfinal represents the pixel value corresponding to the ending point of the peak (mean value is 1) of the floor to be calculated in the total pixels of the one-dimensional mean first screen image.

[0060] Step S103: According to the first-screen position information of each picture component element in the first-screen image and the non-first-screen position information of each picture component element in the non-first-screen image, find the overlapping picture component elements where the first-screen image and the non-first-screen image overlap, and determine the splicing point of the first-screen image and the non-first-screen image according to the overlapping picture component elements.

[0061] Specifically, considering that the first-screen image and the non-first-screen image have overlapping parts, in order to accurately splice the first-screen image and the non-first-screen image, it is necessary to find the overlapping picture components where the first-screen image and the non-first-screen image overlap. Therefore, based on the first-screen position information of each picture component element in the first-screen image and the non-first-screen position information of each picture component element in the non-first-screen image determined above, the distribution of each picture component element in the first-screen image and the non-first-screen image can be determined, and then the overlapping overlapping picture component elements can be found, and the splicing points of the first-screen image and the non-first-screen image can be determined from the overlapping picture component elements.

[0062] According to one embodiment of the present invention, based on the first-screen position information of each picture component element in the first-screen image and the non-first-screen position information of each picture component element in the non-first-screen image, the overlapping picture component elements of the first-screen image and the non-first-screen image are searched, including: extracting the picture component elements of the first-screen image according to the position starting point and position ending point of each picture component element in the first-screen image; extracting the picture component elements of the non-first-screen image according to the position starting point and position ending point of each picture component element in the non-first-screen image; using a matching algorithm to compare and match the picture component elements of the first-screen image and the picture component elements of the non-first-screen image; and determining the overlapping picture component elements of the first-screen image and the non-first-screen image according to the comparison and matching results.

[0063] Specifically, an extraction algorithm is used to extract the image elements contained in the first-screen image based on the starting and ending positions of each image component element in the first-screen image determined by the above calculation. Similarly, an extraction algorithm is used to extract the image elements contained in the non-first-screen image based on the starting and ending positions of each image component element in the non-first-screen image determined by the above calculation. A structural similarity algorithm, a histogram, and a hash algorithm are then used to compare and match the image components of the first-screen image and the image components of the non-first-screen image for content similarity. Finally, the similarity result is compared with a preset matching threshold. When the similarity is greater than the preset matching threshold, it indicates that the content of the image components of the compared first-screen image and the image components of the non-first-screen image are the same, and the image components of the first-screen image and the image components of the non-first-screen image overlap, which is an overlapping image component element. For example, when the similarity between image component A in the first screen and image component C in the non-first-screen image is greater than the preset matching threshold, it indicates that A and C are the same, that is, A and C are overlapping image components that overlap in the first-screen image and the non-first-screen image.

[0064] Figure 3It is a schematic diagram of the comparison and matching process of the image constituent elements of the embodiment of the present invention. The image constituent elements of the first screen image and the non-first screen image are arranged according to the structure of the web page floor, so the first screen position information of the image constituent elements of the first screen image is equivalent to the first screen position information of the floor in the first screen image, and the non-first screen position information of the image constituent elements of the non-first screen image is equivalent to the non-first screen position information of the floor in the non-first screen image, and each image constituent element is equivalent to the content in each floor. The embodiment of the present invention uses the floor as the extraction unit, and extracts the content of each floor in the first screen image and the content of each floor in the non-first screen according to the calculated first screen position information of each floor in the first screen image and the non-first screen position information of each floor in the non-first screen image. The structural similarity algorithm, histogram, and hash algorithm are used to compare and match the content of each floor in the first screen and the content of each floor in the non-first screen, calculate the similarity, and determine whether the content of the two floors subjected to similarity comparison and matching is the same by judging whether the similarity is greater than the matching threshold.

[0065] According to another embodiment of the present invention, the splicing point of the first screen image and the non-first screen image is determined based on the overlapping picture component elements, including: according to the position information corresponding to the overlapping picture component elements, searching for the target overlapping picture component element located at the last position of the first screen image from the overlapping picture component elements, and using the target overlapping picture component element as the splicing point of the first screen image and the non-first screen image.

[0066] Specifically, based on the overlapping picture constituent elements obtained by the above-mentioned similarity comparison and matching, the position information of the overlapping picture constituent elements on the first screen is obtained. The overlapping picture constituent elements may be one or more. If there is one, then this overlapping picture constituent element is the target overlapping picture constituent element located at the last position of the first screen image. If there are multiple overlapping picture constituent elements, then the overlapping picture constituent element at the back position of the first screen image, that is, the bottommost overlapping picture constituent element, is searched from the multiple overlapping picture constituent elements, and the overlapping picture constituent element found is used as the splicing point between the first screen image and the non-first screen image.

[0067] Figure 4 : This is a schematic diagram of the splicing points of an embodiment of the present invention. The left side is the first screen image, and the right side is the non-first screen image. According to the first screen position information of each image component element in the first screen image and the non-first screen position information of each image component element in the non-first screen image, the overlapping image component elements where the first screen image and the non-first screen image overlap are found. In this figure, the overlapping image component elements are the third and fourth gold list trading items from the top to the bottom of the first screen (see the circular frame selection part in the figure for details). From them, the last position in the first screen image, that is, the fourth gold list trading item (see the triangle mark in the first screen image on the left for details) is selected as the target overlapping image component element, that is, the splicing point.

[0068] Step S104: De-duplicate and splice the first screen image and the non-first screen image according to the splicing points to obtain a web page image.

[0069] Specifically, according to the position of the splicing point in the first screen image and the position of the splicing point in the non-first screen image, the splicing range of the first screen image and the splicing range of the non-first screen image are determined, and the splicing range of the first screen image and the splicing range of the non-first screen image are spliced ​​using the relevant methods in the specified programming language library to obtain a web page image.

[0070] According to one embodiment of the present invention, the first screen image and the non-first screen image are deduplicated and spliced ​​according to the splicing point, including: taking the image above the position of the splicing point in the first screen image as the upper splicing image, and taking the image below the position of the splicing point in the non-first screen image as the lower splicing image; and deduplicating and splicing the upper splicing image, the splicing point and the lower splicing image using an image splicing algorithm.

[0071] Specifically, the splicing point is used as the splicing reference for the first screen image and the non-first screen image. The image content above the splicing point in the first screen image is the upper splicing image, and correspondingly, the image content below the splicing point in the non-first screen image is the lower splicing image. Finally, the upper splicing image, the splicing point, and the lower splicing image are deduplicated and spliced ​​using a splicing algorithm. For example, Figure 4 , the 4th gold list trading item on the first screen is the splicing point. Correspondingly, the pictures above the 4th gold list trading item on the first screen are the upper splicing pictures. The 4th gold list trading item on the first screen is at the position of the 2nd gold list trading item in the non-first screen picture, so the pictures below the 2nd gold list trading item in the non-first screen picture are the lower splicing pictures. Finally, the upper splicing picture, splicing point and lower splicing picture are spliced ​​to get the web page picture.

[0072] Figure 5It is a detailed flow chart of the method for splicing web page images according to an embodiment of the present invention. The cloud real machine is swiped and screenshotted through the cloud test platform to obtain the first screen image and the non-first screen image of the web page to be spliced. The first screen image and the non-first screen image are subjected to grayscale processing, one-dimensional flattening processing and sparse processing to obtain a one-dimensional first screen image and a one-dimensional non-first screen image. The image component elements of the first screen image and the non-first screen image of the embodiment of the present invention are laid out according to the structure of the web page floor, that is, each image component element is laid out in the form of a floor, and accordingly, the first screen position information of the image component elements of the first screen image is equivalent to the first screen position information of the floor in the first screen image, and the non-first screen position information of the image component elements of the non-first screen image is equivalent to the non-first screen position information of the floor in the non-first screen image. According to the one-dimensional first screen image and the one-dimensional non-first screen image, the first screen position information of each floor in the first screen image and the non-first screen position information of each floor in the non-first screen image are calculated, and then according to the first screen position information of each floor in the first screen image and the non-first screen position information of each floor in the non-first screen image, the content of each floor included in the first screen image and the content of each floor included in the non-first screen image are extracted. A matching algorithm is then used to compare and match the content of each floor above the first screen with the content of each floor below the first screen. Similarity is calculated and overlapping floors are found where the first and second screen images overlap, i.e., the overlapping image components mentioned above. The joining points between the first and second screen images are then determined from these overlapping floors. The upper and lower spliced ​​images are then deduplicated using an image splicing algorithm. The upper spliced ​​image, the splicing points, and the lower spliced ​​image are then spliced ​​together to create the webpage image.

[0073] According to another embodiment of the present invention, when there are multiple non-first screen images, the method further includes: according to the order of each non-first screen image, using any one of the above-mentioned web page image splicing methods, iteratively de-duplicating and splicing each non-first screen image and the first screen image in turn.

[0074] Specifically, the web page image splicing method of the embodiment of the present invention is not limited to the splicing of one non-first screen image and the first screen image, but also supports the splicing of multiple non-first screen images and the first screen image. In the case where there are multiple non-first screen images, each non-first screen image is ordered. According to the order of each non-first screen image, the above-mentioned web page image splicing method is used to de-duplicate and splice the first non-first screen image, and then use the de-duplicated and spliced ​​web page image as the first screen image, and de-duplicate and splice it with the second non-first screen image to obtain a secondary spliced ​​image. If there is a third non-first screen image, then use the secondary spliced ​​image as the first screen image, and de-duplicate and splice it with the third non-first screen image, and so on, until the last non-first screen image is de-duplicated and spliced ​​to obtain the required web page image.

[0075] The embodiment of the present invention provides a reliable and accurate web page image splicing method for software compatibility testing, so that compatibility testing is no longer limited to the comparison of first screen images, and also avoids the situation where the compatibility test fails due to differences in screen size and resolution. It also supports deduplication and splicing of multiple non-first screen images.

[0076] Figure 6 Schematic diagram of the main modules of the apparatus for splicing web page images according to an embodiment of the present invention. Figure 6 As shown, the apparatus 600 for splicing webpage images mainly includes a module for obtaining images to be spliced ​​601 , a position information identification module 602 , a splicing point determination module 603 and a deduplication splicing module 604 .

[0077] The image acquisition module 601 is configured to acquire, in response to receiving an image splicing request, a home screen image and a non-home screen image of the web page to be spliced, wherein the home screen image and the non-home screen image partially overlap;

[0078] A position information identification module 602 is configured to identify the first-screen position information of each image component in the first-screen image and the non-first-screen position information of each image component in the non-first-screen image;

[0079] A splicing point determination module 603 is configured to search for overlapping image components where the first-screen image and the non-first-screen image overlap based on the first-screen position information of each image component element in the first-screen image and the non-first-screen position information of each image component element in the non-first-screen image, and determine a splicing point between the first-screen image and the non-first-screen image based on the overlapping image component elements;

[0080] The deduplication and splicing module 604 is used to deduplication and splice the first screen image and the non-first screen image according to the splicing points to obtain a web page image.

[0081] According to one embodiment of the present invention, the device 600 for splicing web page images also includes a preprocessing module (not shown in the figure), which is used to: before respectively identifying the first screen position information of each image component element in the first screen image and the non-first screen position information of each image component element in the non-first screen image, perform grayscale processing, one-dimensional flattening processing and sparse processing on the first screen image and the non-first screen image to obtain a one-dimensional first screen image and a one-dimensional non-first screen image; the position information identification module 602 is also used to: calculate the first screen position information of each image component element in the first screen image and the non-first screen position information of each image component element in the non-first screen image based on the one-dimensional first screen image and the one-dimensional non-first screen image.

[0082] According to another embodiment of the present invention, the position information identification module 602 is further used to: respectively perform averaging processing on the one-dimensional first screen image and the one-dimensional non-first screen image to obtain a one-dimensional average first screen image and a one-dimensional average non-first screen image; calculate the position starting point and position end point of each picture component element in the first screen image on the first screen based on the one-dimensional average first screen image and the size of the first screen image; calculate the position starting point and position end point of each picture component element in the non-first screen image on the non-first screen based on the one-dimensional average non-first screen image and the size of the non-first screen image.

[0083] According to another embodiment of the present invention, the splicing point determination module 603 is further used to: extract the picture component elements of the first screen image according to the position starting point and position end point of each picture component element in the first screen image; extract the picture component elements of the non-first screen image according to the position starting point and position end point of each picture component element in the non-first screen; use a matching algorithm to compare and match the picture component elements of the first screen image and the picture component elements of the non-first screen image; and determine the overlapping picture component elements of the first screen image and the non-first screen image based on the comparison and matching results.

[0084] According to another embodiment of the present invention, the splicing point determination module 603 is further used to: search for a target overlapping picture component element located at the last position of the first screen image from the overlapping picture component elements according to the position information corresponding to the overlapping picture component elements, and use the target overlapping picture component element as the splicing point of the first screen image and the non-first screen image.

[0085] According to another embodiment of the present invention, the deduplication and splicing module 604 is further used to: use the picture above the position of the splicing point in the first screen image as the upper splicing picture, and use the picture below the position of the splicing point in the non-first screen image as the lower splicing picture; and use the picture splicing algorithm to deduplicate and splice the upper splicing picture, the splicing point and the lower splicing picture.

[0086] According to yet another embodiment of the present invention, the image components are arranged in the form of web page floors.

[0087] According to another embodiment of the present invention, when there are multiple non-first screen images, the web page image splicing device 600 also includes an iterative deduplication splicing module (not shown in the figure), which is used to: in accordance with the order of each non-first screen image, use any method of web page image splicing of the embodiment of the present invention to iteratively dedupe and splice each non-first screen image and the first screen image in turn.

[0088] Figure 7 is an exemplary system architecture diagram to which embodiments of the present invention can be applied.

[0089] like Figure 7 As shown, system architecture 700 may include terminal devices 701, 702, 703, a network 704, and a server 705. Network 704 is used to provide a medium for communication links between terminal devices 701, 702, 703 and server 705. Network 704 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0090] Users can use terminal devices 701, 702, 703 to interact with server 705 via network 704 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 701, 702, 703, such as web page image splicing applications, etc. (only as an example).

[0091] The terminal devices 701 , 702 , and 703 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0092] Server 705 can be a server that provides various services, such as a background management server that supports the web page image splicing performed by users using terminal devices 701, 702, and 703 (only as an example). In response to receiving the image splicing request, the background management server can obtain the first screen image and non-first screen image of the web page to be spliced, and the first screen image and the non-first screen image partially overlap; respectively identify the first screen position information of each image component element in the first screen image and the non-first screen position information of each image component element in the non-first screen image; based on the first screen position information of each image component element in the first screen image and the non-first screen position information of each image component element in the non-first screen image, find the overlapping image component elements where the first screen image and the non-first screen image overlap, and determine the splicing point of the first screen image and the non-first screen image based on the overlapping image component elements; based on the splicing point, deduplicate the first screen image and the non-first screen image to obtain the web page image, etc., and feed back the processing results (such as web page images, etc. - only as an example) to the terminal device.

[0093] It should be noted that the web page image splicing method provided by the embodiment of the present invention is generally executed by the server 705 , and accordingly, the web page image splicing device is generally set in the server 705 .

[0094] It should be understood that Figure 7 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0095] Reference below Figure 8, which shows a schematic structural diagram of a computer system of a terminal device or server suitable for implementing an embodiment of the present invention. Figure 8 The terminal device or server shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0096] like Figure 8 As shown, the computer system 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage unit 808 into a random access memory (RAM) 803. Various programs and data required for the operation of the system 800 are also stored in the RAM 803. The CPU 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0097] The following components are connected to the I / O interface 805: an input section 806 including a keyboard, a mouse, and the like; an output section 807 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 808 including a hard disk; and a communication section 809 including a network interface card such as a LAN card or a modem. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 810 as needed, so that computer programs read therefrom can be installed into the storage section 808 as needed.

[0098] In particular, according to the embodiments disclosed in the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 809, and / or installed from a removable medium 811. When the computer program is executed by the central processing unit (CPU) 801, the above-mentioned functions defined in the system of the present invention are performed.

[0099] It should be noted that the computer-readable medium described in the present invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media can include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical cable, RF, or any suitable combination thereof.

[0100] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of the boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0101] The units involved in the embodiments of the present invention may be implemented in software or hardware. The units described may also be provided in a processor. For example, the processor may include: a module for acquiring images to be spliced, a module for identifying position information, a module for determining splicing points, and a module for removing duplicates and splicing.

[0102] Among them, the names of these modules do not constitute a limitation of the module itself in some cases. For example, the deduplication and splicing module can also be described as "a module used to deduplicate and splice the first screen image and the non-first screen image according to the splicing points to obtain a web page image."

[0103] On the other hand, the present invention also provides a computer-readable medium, which may be included in the device described in the embodiment; or it may exist independently and not be assembled into the device. The computer-readable medium carries one or more programs, and when the one or more programs are executed by a device, the device includes: in response to receiving a request for image splicing, obtaining the first screen image and the non-first screen image of the web page to be spliced, wherein the first screen image and the non-first screen image partially overlap; respectively identifying the first screen position information of each image component element in the first screen image and the non-first screen position information of each image component element in the non-first screen image; according to the first screen position information of each image component element in the first screen image and the non-first screen position information of each image component element in the non-first screen image, searching for the overlapping image component elements where the first screen image and the non-first screen image overlap, and determining the splicing point of the first screen image and the non-first screen image according to the overlapping image component elements; according to the splicing point, de-duplicating and splicing the first screen image and the non-first screen image to obtain a web page image.

[0104] According to the technical solution of the embodiment of the present invention, it has the following advantages or beneficial effects: by responding to receiving a picture splicing request, the first screen image and the non-first screen image of the web page to be spliced ​​are obtained, and the first screen image and the non-first screen image have some overlap; the first screen position information of each picture component element in the first screen image and the non-first screen position information of each picture component element in the non-first screen image are respectively identified; according to the first screen position information of each picture component element in the first screen image and the non-first screen position information of each picture component element in the non-first screen image, the overlapping picture component elements where the first screen image and the non-first screen image overlap are found, and according to the overlapping picture component elements, the splicing points of the first screen image and the non-first screen image are determined; according to the splicing points, the first screen image and the non-first screen image are deduplicated and spliced ​​to obtain a technical solution for web page images, thereby realizing a precise method for deduplicating and splicing web page images, avoiding the situation where the compatibility test fails due to differences in screen size and resolution, and providing complete and reliable web page comparison images for subsequent software compatibility testing.

[0105] The specific embodiments described herein do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A method for splicing web page images, characterized in that: include: In response to receiving the image stitching request, obtaining a home screen image and a non-home screen image of the web page to be stitched, wherein the home screen image and the non-home screen image partially overlap; Respectively identifying the first-screen position information of each image component element in the first-screen image and the non-first-screen position information of each image component element in the non-first-screen image; According to the first-screen position information of each image component element in the first-screen image and the non-first-screen position information of each image component element in the non-first-screen image, searching for overlapping image component elements where the first-screen image and the non-first-screen image overlap, and determining the splicing point between the first-screen image and the non-first-screen image according to the overlapping image component elements; According to the splicing points, the first screen image and the non-first screen images are deduplicated and spliced ​​to obtain a web page image.

2. The method according to claim 1, characterized in that Before respectively identifying the first-screen position information of each image component element in the first-screen image and the non-first-screen position information of each image component element in the non-first-screen image, the method further includes: Performing grayscale processing, one-dimensional flattening processing, and thinning processing on the first screen image and the non-first screen image respectively to obtain a one-dimensional first screen image and a one-dimensional non-first screen image; The step of respectively identifying the first-screen position information of each image component element in the first-screen image and the non-first-screen position information of each image component element in the non-first-screen image includes: According to the one-dimensional first-screen image and the one-dimensional non-first-screen image, first-screen position information of each picture component element in the first-screen image and non-first-screen position information of each picture component element in the non-first-screen image are calculated.

3. The method according to claim 2, characterized in that Calculating, according to the one-dimensional first-screen image and the one-dimensional non-first-screen image, first-screen position information of each picture component element in the first-screen image and non-first-screen position information of each picture component element in the non-first-screen image, including: Performing averaging processing on the one-dimensional first screen image and the one-dimensional non-first screen image respectively to obtain a one-dimensional average first screen image and a one-dimensional average non-first screen image; Calculate the starting point and ending point of each image component in the first screen image based on the one-dimensional average first screen image and the size of the first screen image; According to the one-dimensional mean non-first screen image and the size of the non-first screen image, the position starting point and the position ending point of each picture component element in the non-first screen image are calculated.

4. The method according to claim 3, characterized in that Searching for overlapping picture elements where the first screen image and the non-first screen image overlap according to the first screen position information of each picture component element in the first screen image and the non-first screen position information of each picture component element in the non-first screen image, including: Extracting the image components of the first screen image according to the starting point and end point of each image component element in the first screen image; Extracting the image components of the non-first screen image according to the starting point and ending point of each image component element in the non-first screen image; Using a matching algorithm, content comparison and matching are performed between the image components of the first screen image and the image components of the non-first screen images; Based on the comparison and matching results, the overlapping picture component elements where the first screen image and the non-first screen image overlap are determined.

5. The method according to claim 1, wherein Determining a splicing point between the first screen image and the non-first screen image according to the overlapping image component elements includes: According to the position information corresponding to the overlapping picture component elements, the target overlapping picture component element located at the last position of the first screen image is searched from the overlapping picture component elements, and the target overlapping picture component element is used as the splicing point of the first screen image and the non-first screen image.

6. The method according to claim 1, characterized in that Deduplication and splicing of the first screen image and the non-first screen image are performed according to the splicing points, including: The picture above the position of the splicing point in the first screen picture is the upper splicing picture, and the picture below the position of the splicing point in the non-first screen picture is the lower splicing picture; The upper spliced ​​image, the splicing points and the lower spliced ​​image are deduplicated and spliced ​​using an image splicing algorithm.

7. The method according to claim 1, characterized in that The image components are arranged in the form of web page floors.

8. The method according to claim 1, characterized in that In the case where there are multiple non-first screen images, the method further includes: According to the order of each of the non-first screen images, use any one of the methods described in claims 1 to 7 to iteratively remove duplicates and splice each of the non-first screen images and the first screen image in turn.

9. A device for splicing web page images, characterized in that: include: A module for obtaining images to be spliced, configured to obtain, in response to receiving an image splicing request, a home screen image and non-home screen images of the web page to be spliced, wherein the home screen image and the non-home screen images partially overlap; A position information identification module, configured to respectively identify the first-screen position information of each image component element in the first-screen image and the non-first-screen position information of each image component element in the non-first-screen image; a splicing point determination module, configured to search for overlapping image components where the first screen image and the non-first screen image overlap based on the first screen position information of each image component element in the first screen image and the non-first screen position information of each image component element in the non-first screen image, and determine the splicing point between the first screen image and the non-first screen image based on the overlapping image component elements; The deduplication and splicing module is used to deduplication and splice the first screen image and the non-first screen image according to the splicing points to obtain a web page image.

10. A mobile electronic device terminal, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 8.

11. A computer-readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images