Advertisement page intercepting processing method and device and electronic equipment

Abstract

The application discloses an advertisement page interception processing method and device and electronic equipment. The advertisement page interception processing method intercepts a screenshot image of a visible area in a target window after each update by scrolling a scroll bar containing the target window, and then splices the screenshot images intercepted after each update to obtain advertisement page data.

Description

Technical Field

[0001] This application relates to the field of internet advertising monitoring, and more specifically, to a method, apparatus, and electronic device for capturing and processing advertising pages. Background Technology

[0002] In recent years, with the development of internet technology, various advertising alliances and e-commerce platforms have emerged, bringing with them the prominence of internet advertising problems. Compared to traditional advertising, internet advertising is convenient to publish, diverse in form, and allows for precise targeting. Illegal advertisements can easily hide within it, making them highly concealed and difficult to detect, track, and prosecute using traditional methods. Therefore, regulating internet advertising is essential.

[0003] The first step in regulation is to collect data on internet advertisements. These advertisements exist on website webpages, and some are long ad pages that are difficult to display completely within a single window. Therefore, a solution for capturing and processing ad pages is needed, one that can accurately and completely capture these long ad pages. Summary of the Invention

[0004] To address the aforementioned technical problems, this application is proposed. Embodiments of this application provide an advertising page capture processing method, apparatus, and electronic device, which captures a screenshot of the visible area within a target window by scrolling a scroll bar containing the target window, and finally stitches the screenshots captured after each update together to obtain advertising page data.

[0005] According to one aspect of this application, an advertising page capture processing method is provided, comprising:

[0006] Receive a screenshot command for the target window containing the advertising page displayed on the terminal's screen;

[0007] Starting from the initial value, the value of the scroll bar used to scroll the target window is updated multiple times according to a preset update method;

[0008] Capture a screenshot of the visible area of ​​the target window after each update; and

[0009] The screenshots captured after each update are stitched together to obtain a scrolling screenshot of the complete interface of the target window.

[0010] In the above-mentioned advertising page capture processing method, the value of the scroll bar used to scroll the target window is updated multiple times according to a preset update method starting from the initial value. This includes: in response to detecting that the value of the scroll bar is allowed to be modified through the system layer call interface provided by the operating system of the terminal, the value of the scroll bar is updated multiple times according to the preset update method starting from the initial value through the system layer call interface.

[0011] In the above-mentioned advertising page capture processing method, the value of the scroll bar used to scroll the target window is updated multiple times according to a preset update method starting from an initial value. This includes: detecting whether the type of the object displayed in the target window is a specified type; in response to the detection result that the type of the object displayed in the target window is the specified type, determining that the target window contains the scroll bar; detecting whether it is allowed to modify the value of the scroll bar through a preset scheduling interface, including: obtaining the class name of the scroll bar, the startup information of the scheduling interface, and the setting information of the scroll bar, wherein the startup information of the calling interface is used to indicate whether the calling interface is enabled, and the setting information of the scroll bar is used to indicate whether the value of the scroll bar is allowed to be modified; and, in response to the class name of the scroll bar being a specified class name, the startup information of the calling interface indicating that the calling interface has been started, and the setting information of the scroll bar indicating that the value of the scroll bar is allowed to be modified, the value of the scroll bar used to scroll the target window is updated multiple times according to a preset update method starting from an initial value through the calling interface.

[0012] In the above-mentioned advertising page capture processing method, the screenshot images captured after each update are stitched together to obtain a scrolling screenshot image of the complete interface of the target window, including:

[0013] The training phase includes:

[0014] The screenshot images of adjacent frames to be stitched are set as the first image and the second image;

[0015] The first image and the second image are respectively passed through a first convolutional neural network as a feature extractor to obtain a first feature map and a second feature map;

[0016] Based on predetermined location information, the regions to be spliced ​​in the first feature map and the second feature map are determined to be the third feature map and the fourth feature map, respectively. The portion of the first feature map other than the third feature map is set as the fifth feature map, and the portion of the second feature map other than the fourth feature map is set as the sixth feature map.

[0017] Calculate the directed edge feature graph between the third feature graph and the fourth feature graph;

[0018] The real stitched image corresponding to the directed edge feature map is input into the first convolutional neural network to obtain the stitched feature map;

[0019] The concatenated feature map and the directed edge feature map are respectively input into the second convolutional neural network used as an adversarial generator to obtain the discriminator loss function value;

[0020] The second convolutional neural network, acting as an adversarial generator, is trained based on the discriminator loss function value; and

[0021] The difference feature map between the directed edge feature map and the concatenated feature map is calculated as the motion information feature map;

[0022] The fifth feature map and the motion information feature map are fused to obtain a first fused feature map, and the sixth feature map and the motion information feature map are fused to obtain a second fused feature map;

[0023] The first fused feature map and the second fused feature map are respectively input into the third convolutional neural network used as a generator to obtain the generated first image part and the generated second image part;

[0024] Calculate a first difference between the generated first image portion and the real first image portion, and a second difference between the generated second image portion and the real second image portion, and train the third convolutional neural network as the generator using the first difference and the second difference; and

[0025] The usage phase includes:

[0026] The screenshot images of adjacent frames to be stitched are set as the first image and the second image;

[0027] The first image and the second image are respectively passed through the first convolutional neural network, which has been trained in the training phase, to obtain a first feature map and a second feature map.

[0028] Based on predetermined location information, the regions to be spliced ​​in the first feature map and the second feature map are determined to be the third feature map and the fourth feature map, respectively. The portion of the first feature map other than the third feature map is set as the fifth feature map, and the portion of the second feature map other than the fourth feature map is set as the sixth feature map.

[0029] Calculate the directed edge feature graph between the third feature graph and the fourth feature graph;

[0030] The directed edge feature map is input into the second convolutional neural network, which has been trained during the training phase and serves as an adversarial generator, to obtain a third image portion corresponding to the stitching region.

[0031] The difference feature map between the directed edge feature map and the third image portion is calculated as the motion information feature map;

[0032] The fifth feature map and the motion information feature map are fused to obtain a first fused feature map, and the sixth feature map and the motion information feature map are fused to obtain a second fused feature map;

[0033] The first fused feature map and the second fused feature map are respectively input into the third convolutional neural network, which has been trained during the training phase, to obtain a first image portion corresponding to the fifth feature map and a second image portion corresponding to the sixth feature map; and

[0034] The first image portion, the third image portion, and the second image portion are stitched together.

[0035] In the above-mentioned advertising page capture processing method, determining the areas to be spliced ​​in the first feature map and the second feature map as the third feature map and the fourth feature map based on predetermined position information includes: determining the areas to be spliced ​​in the first feature map and the second feature map based on the position of the scroll bar.

[0036] In the above-mentioned method for capturing and processing advertisement pages, calculating the directed edge feature graph between the third feature graph and the fourth feature graph includes:

[0037] The directed edge feature map between the third feature map and the fourth feature map is calculated using the following formula, wherein the formula is:

[0038]

[0039] Wherein, n is a hyperparameter, which is positively correlated with the scrolling speed of the target window.

[0040] In the above-mentioned method for capturing and processing advertisement pages, inputting the spliced ​​feature map and the directed edge feature map into a second convolutional neural network (CNN) serving as an adversarial generator to obtain discriminator loss function values ​​includes: inputting the spliced ​​feature map into the second CNN to obtain a seventh feature map; inputting the directed edge feature map into the second CNN to obtain an eighth feature map; determining whether the values ​​at predetermined positions in the seventh feature map and the eighth feature map are the same; in response to the seventh feature map and the eighth feature map having the same value at predetermined positions, calculating the negative value of the base-2 logarithm of the value at the predetermined position as a first value; in response to the seventh feature map and the eighth feature map having different values ​​at predetermined positions, calculating the base-2 logarithm of the value at the predetermined position as a second value; and calculating the sum of the average of the first value at positions with the same value and the average of the second value at positions with different values ​​as the discriminator loss function value.

[0041] According to another aspect of this application, an advertising page capture processing apparatus is provided, comprising:

[0042] The instruction receiving module is used to receive screenshot instructions for the target window containing the advertisement page displayed on the terminal's screen.

[0043] The update module is used to update the value of the scroll bar used to scroll the target window multiple times according to a preset update method, starting from the initial value;

[0044] The screenshot module is used to capture a screenshot image of the visible area of ​​the target window after each update; and

[0045] The stitching module is used to stitch together the screenshots captured after each update to obtain a scrolling screenshot of the complete interface of the target window.

[0046] According to another aspect of this application, an electronic device is provided, comprising: a processor; and a memory storing computer program instructions, which, when executed by the processor, cause the processor to perform the advertising page interception processing method as described above.

[0047] According to another aspect of this application, a computer-readable medium is provided that stores computer program instructions thereon, which, when executed by a processor, cause the processor to perform the advertising page interception processing method as described above.

[0048] Compared with the prior art, the advertising page capture processing method, apparatus and electronic device provided in this application obtain advertising page data by scrolling a scroll bar containing a target window, capturing a screenshot of the visible area of ​​the target window after each update, and finally stitching the screenshots captured after each update together. Attached Figure Description

[0049] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0050] Figure 1 The illustration shows an application scenario of the advertising page cropping processing method according to an embodiment of this application.

[0051] Figure 2 The illustration shows a flowchart of an advertising page cropping processing method according to an embodiment of this application.

[0052] Figure 3 The illustration shows a flowchart of the advertising page capture processing method according to an embodiment of this application, in which the value of the scroll bar used to scroll the target window is updated multiple times starting from an initial value according to a preset update method.

[0053] Figure 4 The illustration shows one of the flowcharts in the advertising page capture processing method according to an embodiment of this application, in which the screenshot images captured after each update are stitched together to obtain a scrolling screenshot image of the complete interface of the target window.

[0054] Figure 5 The illustration shows a second flowchart of the advertising page capture processing method according to an embodiment of this application, in which the screenshot images captured after each update are stitched together to obtain a scrolling screenshot image of the complete interface of the target window.

[0055] Figure 6 The illustration shows a schematic diagram of the training phase of the advertising page capture processing method according to an embodiment of this application, in which the screenshot images captured after each update are stitched together to obtain a scrolling screenshot image of the complete interface of the target window.

[0056] Figure 7 The illustration shows a schematic diagram of the usage stage of the advertising page capture processing method according to an embodiment of this application, in which the screenshot images captured after each update are stitched together to obtain a scrolling screenshot image of the complete interface of the target window.

[0057] Figure 8 The illustration shows a block diagram of an advertising page cropping processing apparatus according to an embodiment of this application.

[0058] Figure 9 The illustration shows a block diagram of the update module of the advertising page capture processing apparatus according to an embodiment of this application.

[0059] Figure 10 The figure shows a block diagram of the splicing module of the advertising page cropping processing device according to an embodiment of the present application.

[0060] Figure 11 A block diagram of an electronic device according to an embodiment of this application is illustrated. Detailed Implementation

[0061] Hereinafter, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0062] Application Overview

[0063] As mentioned above, with the development of internet technology, various advertising alliances and e-commerce platforms have emerged, accompanied by the prominence of problems related to internet advertising. Compared to traditional advertising, internet advertising is convenient to publish, diverse in form, and allows for precise targeting. Illegal advertisements can easily hide within it, making them highly concealed and difficult to detect, track, and prosecute using traditional methods. Therefore, regulating internet advertising is essential.

[0064] The first step in regulation is to collect data on internet advertisements, which exist on website web pages. Some internet advertisements are long ad pages, which are difficult to display completely in one window.

[0065] Therefore, we look forward to an advertising page capture solution that can accurately and completely capture long advertising pages.

[0066] It should be understood that for long advertisement pages, the target window can be updated by scrolling a scrollbar containing the target window. For example, the target window can be scrolled using a scrollbar provided by the operating system or a scrollbar contained within the target window itself. Furthermore, after each update, a screenshot of the visible area of ​​the target window is captured, and these screenshots are then stitched together to obtain the long advertisement page.

[0067] Based on this, this application proposes an advertising page capture processing method, which includes: receiving a screenshot instruction for a target window containing an advertising page displayed on the screen of a terminal; updating the value of a scroll bar used to scroll the target window multiple times according to a preset update method starting from an initial value; capturing a screenshot image of the visible area of ​​the target window after each update; and stitching the screenshot images captured after each update together to obtain a scrolling screenshot image of the complete interface of the target window.

[0068] Figure 1 The illustration shows an application scenario of the advertising page cropping processing method according to an embodiment of this application.

[0069] like Figure 1 As shown, in this application scenario, by controlling the scrolling of the target window (e.g., as...) Figure 1 The scroll bar of W shown (e.g., as shown) Figure 1 The target window containing the advertisement page is scrolled using R (as shown in the diagram), and a screenshot of the target window is obtained after each scroll. Then, the screenshot captured after each update is input into a server deployed with a stitching algorithm (e.g., such as...). Figure 1 As shown in the diagram (S), the server is able to stitch the screenshot images together based on the stitching algorithm to obtain a scrolling screenshot image, i.e., the long advertisement page.

[0070] After introducing the basic principles of this application, various non-limiting embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0071] Exemplary methods

[0072] Figure 2 The illustration shows a flowchart of an advertising page cropping processing method according to an embodiment of this application. Figure 2 As shown, the advertising page capture processing method according to an embodiment of this application includes: S110, receiving a screenshot instruction for a target window containing an advertising page displayed on the screen of a terminal; S120, updating the value of a scroll bar used to scroll the target window multiple times according to a preset update method, starting from an initial value; S130, capturing a screenshot image of the visible area of ​​the target window after each update; and S140, stitching the screenshot images captured after each update together to obtain a scrolling screenshot image of the complete interface of the target window.

[0073] In step S110, a screenshot command is received for the target window containing the advertising page displayed on the terminal's screen. That is, a screenshot operation is received for the windows containing the target advertising start page and landing page. It should be understood that in the technical solution of this application, the target advertisement is a "long" advertisement, which cannot be fully displayed within a single target window. Therefore, the target window needs to be scrolled to display the complete portion of the target advertisement.

[0074] In step S120, the value of the scroll bar used to scroll the target window is updated multiple times according to a preset update method, starting from the initial value. Accordingly, in one example of this application, if it is detected that modification of the scroll bar value is permitted through a system call interface provided by the operating system of a terminal or virtual terminal, the system call interface can be invoked, and the scroll bar value is updated multiple times according to the preset update method, starting from the initial value. Here, the initial value of the scroll bar is typically 0, corresponding to the beginning portion of the target window; the scroll bar value is updated multiple times according to the preset update method, for example, by adding a predetermined value each time to update the scroll bar value multiple times to obtain the other portions of the target window up to the end portion. Furthermore, the predetermined value added each time can be adaptively adjusted based on the size of the target window, which is not limited to this application.

[0075] It is worth mentioning that, in this embodiment, since the operating system of the terminal or virtual terminal provides a scroll bar for scrolling the target window, in this specific example, the application corresponding to the target window does not need to provide an interface for user control of the scroll bar. The scroll bar of the target window can be controlled through the system layer interface, thereby enabling scrolling screenshots of target windows of third-party applications that do not provide an interface for controlling the scroll bar, thus expanding the application scenarios of scrolling screenshots.

[0076] Furthermore, in this specific embodiment, in response to detecting that the value of the scroll bar is allowed to be modified through the system-level call interface provided by the operating system of the terminal, the process of updating the value of the scroll bar multiple times from the initial value according to a preset update method through the system-level call interface includes: updating the value of the scroll bar multiple times from the initial value according to a preset update method through the user interface object attribute acquisition interface.

[0077] In another specific example of this application, when the target window contains a scrollbar, it is determined whether modification of the scrollbar value is permitted through a preset API call. This includes obtaining the class name of the scrollbar, the activation information of the API call, and the setting information of the scrollbar. The activation information of the API call indicates whether the API call is enabled, and the setting information of the scrollbar indicates whether modification of the scrollbar value is permitted. When the class name of the scrollbar is a specified class name, the activation information of the API call indicates that the API call is enabled, and the setting information of the scrollbar indicates that modification of the scrollbar value is permitted, it is determined that modification of the scrollbar value through the API call is permitted. Before determining whether modification of the scrollbar value through the preset API call is permitted, the method further includes: detecting whether the type of the object displayed in the target window is a specified object class; if the detection result is that the type of the object displayed in the target window is the specified object class, it is determined that the target window contains the scrollbar.

[0078] That is, in the other specific example, such as Figure 3As shown, the process of updating the value of the scroll bar used to scroll the target window multiple times from an initial value according to a preset update method includes the following steps: S210, detecting whether the type of the object displayed in the target window is a specified type; S220, in response to the detection result that the type of the object displayed in the target window is the specified type, determining that the target window contains the scroll bar; S230, detecting whether it is allowed to modify the value of the scroll bar through a preset scheduling interface, including: obtaining the class name of the scroll bar, the startup information of the scheduling interface, and the setting information of the scroll bar, wherein the startup information of the calling interface is used to indicate whether the calling interface is enabled, and the setting information of the scroll bar is used to indicate whether the value of the scroll bar is allowed to be modified; and S240, in response to the class name of the scroll bar being a specified class name, the startup information of the calling interface indicating that the calling interface has been started, and the setting information of the scroll bar indicating that the value of the scroll bar is allowed to be modified, updating the value of the scroll bar used to scroll the target window multiple times from an initial value according to a preset update method through the calling interface.

[0079] In step S130, a screenshot of the visible area of ​​the target window is captured after each update. That is, a screenshot of the visible area within the target window is captured after each scroll bar is scrolled; specifically, screenshots of different parts of the target advertisement in the target window are captured after each update of the scroll bar value.

[0080] In step S140, the screenshots captured after each update are stitched together to obtain a scrolling screenshot of the complete interface of the target window. That is, the screenshots containing different parts of the target advertising page are stitched together to form the scrolling screenshot, i.e., the target long advertising page.

[0081] When stitching screenshots together, two main aspects of image processing need to be considered. One is the handling of overlapping pixels at the stitching location, and the other is that since the image contains motion information when scrolling, the motion information needs to be removed during stitching to obtain a static and complete image.

[0082] Here, considering that both overlapping pixels and motion information are related to the differential features of the image, in this example, the above two functions can be achieved by training a single end-to-end generator model, thereby realizing image stitching processing.

[0083] Specifically, for the screenshot images of adjacent frames to be stitched together, for example, referred to as the first image and the second image, the first feature map and the second feature map are first obtained by the first convolutional neural network for feature extraction, and the areas to be stitched in the first feature map and the second feature map are determined based on predetermined position information, such as the position of the scroll bar, for example, referred to as the third feature map and the fourth feature map, while the parts of the first feature map and the second feature map other than the third feature map and the fourth feature map are referred to as the fifth feature map and the sixth feature map.

[0084] Here, considering the motion relationship between the third and fourth feature maps, instead of directly calculating the difference map, we calculate the directed edge feature map between the third and fourth feature maps, that is:

[0085]

[0086] Here, the hyperparameter n depends on the scrolling speed. The faster the scrolling speed, the larger the value of n. This can be intuitively understood as the faster the scrolling speed, the more overlapping areas and motion information need to be processed.

[0087] Then, the directed edge feature map is input into the second convolutional neural network, which serves as the adversarial generator, to obtain the third image portion corresponding to the region to be stitched. The training process of the second convolutional neural network is as follows: it receives the real stitched image corresponding to the directed edge feature map, inputs the real stitched image into the first convolutional neural network to obtain the stitched feature map, then inputs the stitched feature map and the directed edge feature map into the second convolutional neural network respectively, and updates the second convolutional neural network through the discriminator loss function.

[0088] Furthermore, for the first image portion and the second image portion excluding the area to be stitched, i.e., the fifth feature map and the sixth feature map, the difference feature map between the directed edge feature map and the stitching feature map can be used as a motion information feature map. This motion information feature map is then used as a motion information mask to train the third convolutional neural network as the generator. That is, the first fused feature map of the fifth feature map and the motion information feature map, and the second fused feature map of the sixth feature map and the motion information feature map, are alternately input into the third convolutional neural network to train it to remove motion information from the fifth and sixth feature maps to generate the first image portion and the second image portion.

[0089] Specifically, the splicing process of the advertising page cropping processing method according to the embodiments of this application includes a training phase and a usage phase, such as... Figure 4 and Figure 5 As shown.

[0090] Figure 4The illustration shows one of the flowcharts in the advertising page capture processing method according to an embodiment of this application, in which the screenshot images captured after each update are stitched together to obtain a scrolling screenshot image of the complete interface of the target window.

[0091] like Figure 4 As shown, the training phase of the stitching process includes the following steps: S410, setting the screenshot images of adjacent frames to be stitched as a first image and a second image; S420, passing the first image and the second image through a first convolutional neural network as a feature extractor to obtain a first feature map and a second feature map; S430, based on predetermined position information, determining the regions to be stitched in the first feature map and the second feature map as a third feature map and a fourth feature map, and setting the portion of the first feature map excluding the third feature map as a fifth feature map and setting the portion of the second feature map excluding the fourth feature map as a sixth feature map; S440, calculating the directed edge feature map between the third feature map and the fourth feature map; S450, inputting the real stitched image corresponding to the directed edge feature map into the first convolutional neural network to obtain a stitched feature map; S460, inputting the stitched feature map and the directed edge feature map into an adversarial generator. S470: The second convolutional neural network obtains the discriminator loss function value; S480: The second convolutional neural network, acting as an adversarial generator, is trained based on the discriminator loss function value; S490: The difference feature map between the directed edge feature map and the concatenated feature map is calculated as a motion information feature map; S490: The fifth feature map and the motion information feature map are fused to obtain a first fused feature map, and the sixth feature map and the motion information feature map are fused to obtain a second fused feature map; S500: The first fused feature map and the second fused feature map are respectively input into the third convolutional neural network, acting as a generator, to obtain a generated first image portion and a generated second image portion; S510: The first difference between the generated first image portion and the real first image portion and the second difference between the generated second image portion and the real second image portion are calculated, and the third convolutional neural network, acting as a generator, is trained using the first difference and the second difference.

[0092] Figure 6 The illustration shows a schematic diagram of the training phase of the advertising page capture processing method according to an embodiment of this application, in which the screenshots captured after each update are stitched together to obtain a scrolling screenshot of the complete interface of the target window. Figure 6 As shown, in the architecture of the training phase of the stitching operation according to the embodiments of this application, the first image to be stitched (e.g., as shown in the diagram) is first... Figure 6 The IN1 shown in the image and the second image (e.g., as shown in the image) Figure 6The IN2 input shown is used as the first convolutional neural network (e.g., as shown) for feature extraction. Figure 6 The CNN1 shown is used to obtain the first feature map (e.g., as shown in the diagram). Figure 6 The F1 shown is the second feature map (e.g., as shown in the figure). Figure 6 (as shown in F2). Next, based on predetermined position information, such as the value of the scroll bar, the region to be stitched between the first feature map and the second feature map is determined as the third feature map (e.g., as shown in F2). Figure 6 The F3 shown in the figure) and the fourth feature map (e.g., as shown in the figure) Figure 6 The F4 shown in the diagram is used to set the portion of the first feature map other than the third feature map as the fifth feature map (e.g., as shown in the diagram). Figure 6 As shown in F5), and setting the portion of the second feature map other than the fourth feature map as the sixth feature map (e.g., as shown in F5). Figure 6 (as shown in F6). Next, the directed edge feature graph between the third feature graph and the fourth feature graph is calculated (e.g., as shown in F6). Figure 6 The intended Fd); then, the ground stitched image corresponding to the directed edge feature map (e.g., as shown in the figure); Figure 6 The IN0 (as shown) is input into the first convolutional neural network to obtain the concatenated feature map (e.g., as shown). Figure 6 The concatenated feature map and the directed edge feature map are then input into a second convolutional neural network (e.g., as shown in the diagram) that serves as an adversarial generator. Figure 6 The CNN2 shown is used to obtain the discriminator loss function value, and the second convolutional neural network, which acts as an adversarial generator, is trained based on the discriminator loss function value. Furthermore, the difference feature map between the directed edge feature map and the concatenated feature map is calculated as a motion information feature map (e.g., such as...). Figure 6 The fifth feature map and the motion information feature map are fused to obtain a first fused feature map (e.g., as shown in the figure). Figure 6 The intended Ff1), and the sixth feature map and the motion information feature map are fused to obtain a second fused feature map (e.g., as shown in the figure). Figure 6 (as shown in Ff2). Next, the first fused feature map and the second fused feature map are respectively input into a third convolutional neural network (e.g., as shown in Ff2) which serves as the generator. Figure 6 The CNN3 shown is used to obtain the first image portion generated (e.g., as shown in the diagram). Figure 6 The intended Im1) and the generated second image portion (e.g., as shown) Figure 6 As shown in Im2); finally, calculate the generated first image portion and the real first image portion (e.g., as shown in Im2); Figure 6The first difference between the intended Imr1 and the generated second image portion and the real second image portion (e.g., as shown) Figure 6 The second difference between Imr2 (as shown in the diagram) is used to train the third convolutional neural network as a generator, using the first difference and the second difference.

[0093] Figure 5 The illustration shows a second flowchart of the advertising page capture processing method according to an embodiment of this application, which involves stitching together the screenshots captured after each update to obtain a scrolling screenshot of the complete interface of the target window. Figure 5 As shown, the usage phase of the stitching process includes the following steps: S610, setting the screenshot images of adjacent frames to be stitched as a first image and a second image; S620, passing the first image and the second image through the first convolutional neural network, which has been trained in the training phase as a feature extractor, to obtain a first feature map and a second feature map; S630, based on predetermined position information, determining the regions to be stitched in the first feature map and the second feature map as a third feature map and a fourth feature map, and setting the portion of the first feature map other than the third feature map as a fifth feature map and setting the portion of the second feature map other than the fourth feature map as a sixth feature map; S640, calculating the directed edge feature map between the third feature map and the fourth feature map; S650, inputting the directed edge feature map into the trained convolutional neural network. The second convolutional neural network, which serves as an adversarial generator and has been trained during the training phase, obtains a third image portion corresponding to the splicing region; S660, the difference feature map between the directed edge feature map and the third image portion is calculated as a motion information feature map; S670, the fifth feature map and the motion information feature map are fused to obtain a first fused feature map, and the sixth feature map and the motion information feature map are fused to obtain a second fused feature map; S680, the first fused feature map and the second fused feature map are respectively input into the third convolutional neural network, which serves as a generator and has been trained during the training phase, to obtain a first image portion corresponding to the fifth feature map and a second image portion corresponding to the sixth feature map; and S690, the first image portion, the third image portion, and the second image portion are spliced.

[0094] Figure 7 The illustration shows a schematic diagram of the architecture of the advertising page capture processing method according to an embodiment of this application, in which the screenshots captured after each update are stitched together to obtain a scrolling screenshot of the complete interface of the target window. Figure 7 As shown, in the architecture of the use phase of the stitching operation according to the embodiments of this application, the first image to be stitched (e.g., as shown) is first... Figure 7The IN1 shown in the image and the second image (e.g., as shown in the image) Figure 7 The IN2 input shown is the first convolutional neural network (e.g., as shown) trained during the training phase, which acts as a feature extractor. Figure 7 The CNN1 shown is used to obtain the first feature map (e.g., as shown in the diagram). Figure 7 The F1 shown is the second feature map (e.g., as shown in the figure). Figure 7 (as shown in F2). Next, based on predetermined position information, such as the value of the scroll bar, the region to be stitched between the first feature map and the second feature map is determined as the third feature map (e.g., as shown in F2). Figure 7 The F3 shown in the figure) and the fourth feature map (e.g., as shown in the figure) Figure 7 The F4 shown in the diagram is used to set the portion of the first feature map other than the third feature map as the fifth feature map (e.g., as shown in the diagram). Figure 7 As shown in F5), and setting the portion of the second feature map other than the fourth feature map as the sixth feature map (e.g., as shown in F5). Figure 7 (as shown in F6). Next, the directed edge feature graph between the third feature graph and the fourth feature graph is calculated (e.g., as shown in F6). Figure 7 The directed edge feature map is then input into the second convolutional neural network (e.g., as shown in the diagram) that has been trained during the training phase and serves as an adversarial generator. Figure 7 The CNN2 shown is used to obtain a third image portion corresponding to the stitched region (e.g., as shown in the diagram). Figure 7 As shown in Im3). Furthermore, the difference feature map between the directed edge feature map and the third image portion is calculated as a motion information feature map (e.g., as shown in Im3). Figure 7 The fifth feature map and the motion information feature map are fused to obtain a first fused feature map (e.g., as shown in the figure). Figure 7 The intended Ff1), and the sixth feature map and the motion information feature map are fused to obtain a second fused feature map (e.g., as shown in the figure). Figure 7 (as shown in Ff2). Next, the first fused feature map and the second fused feature map are respectively input into the third convolutional neural network (e.g., as shown in the training phase) which serves as the generator. Figure 7 The CNN3 shown herein) obtains the first image portion corresponding to the fifth feature map (e.g., as shown in the diagram). Figure 7 The image shown is Im1) and the second image portion corresponding to the sixth feature map (e.g., as shown in the image). Figure 7 As shown in Im2); finally, the first image portion, the third image portion, and the second image portion are stitched together to obtain a scrolling screenshot of the complete interface of the target window (e.g., as shown in Im2). Figure 7 The implied Im0).

[0095] More specifically, in step S410 of the training phase, the screenshot images of adjacent frames to be stitched are set as the first image and the second image. For ease of explanation, two adjacent frame screenshot images are used as an example to illustrate the training phase of the stitching operation.

[0096] More specifically, in step S420 of the training phase, the first image and the second image are respectively passed through a first convolutional neural network as a feature extractor to obtain a first feature map and a second feature map. That is, the first convolutional neural network as a feature extractor is used to explicitly spatially encode the first image and the second image to extract high-dimensional bank feature representations of the first image and the second image to obtain the first feature map and the second feature map.

[0097] As those skilled in the art will know, convolutional neural networks (CNNs) outperform conventional statistical feature extraction methods in extracting local features. For ease of understanding, the process of CNNs extracting local image features is explained mathematically, where the formula is:

[0098] f i =active(N i ×f i-1 +B i )

[0099] Among them, f i-1 f is the input to the first convolutional neural network in the i-th layer. i N is the output of the first convolutional neural network in the i-th layer. i Let B be the filter of the first convolutional neural network in the i-th layer, and B i Let be the bias vector of the first convolutional neural network in the i-th layer, and active represent the non-linear activation function, such as Si gmoi d non-linear activation function, tanh non-linear activation function, etc.

[0100] More specifically, in step S430 of the training phase, based on predetermined position information, the regions to be stitched in the first and second feature maps are determined as the third and fourth feature maps, respectively. The portion of the first feature map excluding the third feature map is set as the fifth feature map, and the portion of the second feature map excluding the fourth feature map is set as the sixth feature map. In a specific example of this application, the regions to be stitched in the first and second feature maps are determined based on the position of the scroll bar (i.e., the scroll bar's setting value). Here, the spatial mapping invariance of the convolutional neural network during feature extraction is actually applied. That is, if the regions to be stitched in the first and second images can be determined, then after mapping by the convolutional neural network, the positions of the regions to be stitched in the first and second images are also mapped to the first and second feature maps to obtain the third and fourth feature maps. Further, the portion of the first feature map excluding the third feature map is set as the fifth feature map, and the portion of the second feature map excluding the fourth feature map is set as the sixth feature map.

[0101] More specifically, in step S440 of the training phase, a directed edge feature map is calculated between the third feature map and the fourth feature map. Here, considering the motion relationship between the third feature map and the fourth feature map, instead of directly calculating the difference map, in this embodiment, the directed edge feature map between the third feature map and the fourth feature map is calculated to preserve and characterize the motion information between the third feature map and the fourth feature map.

[0102] Specifically, in this embodiment of the application, the directed edge feature map between the third feature map and the fourth feature map is calculated using the following formula, wherein the formula is:

[0103]

[0104] Here, n is a hyperparameter that is positively correlated with the scrolling speed of the target window. It can be understood that the hyperparameter n depends on the scrolling speed; the faster the scrolling speed, the larger the value of n. Intuitively, a faster scrolling speed requires processing more overlapping areas and motion information.

[0105] More specifically, in step S450 of the training phase, the real stitched image corresponding to the directed edge feature map is input into the first convolutional neural network to obtain the stitched feature map. It should be understood that, because the directed edge feature map retains and represents the motion information between the third and fourth feature maps, in order to fully utilize the beneficial parts of the motion information and remove the adverse effects brought about by the motion information, this embodiment employs an adversarial generative approach to achieve the above-mentioned technical objective.

[0106] Specifically, firstly, the actual stitched image corresponding to the directed edge feature map is obtained, that is, the actual image corresponding to the region to be stitched. Then, the actual stitched image is input into the first convolutional neural network to obtain the stitched feature map.

[0107] More specifically, in step S460 of the training phase, the concatenated feature map and the directed edge feature map are respectively input into the second convolutional neural network used as an adversarial generator to obtain discriminator loss function values. That is, the concatenated feature map and the directed edge feature map are input into the second convolutional neural network used as an adversarial generator to obtain discriminator loss function values, wherein the discriminator loss function values ​​are used to train the second convolutional neural network to deceive the second convolutional neural network into fully learning the features in the directed edge feature map that are consistent with the concatenated feature map.

[0108] Specifically, in this embodiment, the process of inputting the concatenated feature map and the directed edge feature map into a second convolutional neural network (CNN) serving as an adversarial generator to obtain the discriminator loss function value includes: first, inputting the concatenated feature map into the second CNN to obtain a seventh feature map; then, inputting the directed edge feature map into the second CNN to obtain an eighth feature map; next, determining whether the values ​​at predetermined positions in the seventh feature map and the eighth feature map are the same; then, in response to the seventh feature map and the eighth feature map having the same value at predetermined positions, calculating the negative value of the base-2 logarithm of the value at the predetermined position as a first value; next, in response to the seventh feature map and the eighth feature map having different values ​​at predetermined positions, calculating the base-2 logarithm of the value at the predetermined position as a second value; and then, calculating the sum of the average of the first value at positions with the same value and the average of the second value at positions with different values ​​as the discriminator loss function value.

[0109] More specifically, in step S470 of the training phase, the second convolutional neural network, acting as an adversarial generator, is trained based on the discriminator loss function value. It should be understood that training the second convolutional neural network using the discriminator loss function value is intended to trick the second convolutional neural network into fully learning features in the directed edge feature map that are consistent with the stitched feature map, so that the output of the second convolutional neural network is closer to the real stitched image.

[0110] More specifically, in step S480 of the training phase, a difference feature map between the directed edge feature map and the stitched feature map is calculated as a motion information feature map. As mentioned earlier, the directed edge feature map includes motion information, while the stitched feature map, generated from a real stitched image, does not contain motion information. Therefore, the difference feature map between the two (e.g., calculating the positional difference between the two) is calculated to obtain a motion information feature map representing motion information.

[0111] Next, using the motion information feature map as a motion information mask, a third convolutional neural network, which serves as the generator, is trained.

[0112] More specifically, in step S490 of the training phase, the fifth feature map and the motion information feature map are fused to obtain a first fused feature map, and the sixth feature map and the motion information feature map are fused to obtain a second fused feature map. In this embodiment, the first fused feature map can be obtained by calculating a position-weighted sum between the fifth feature map and the motion information feature map, and the second fused feature map can be obtained by calculating a position-weighted sum between the sixth feature map and the motion information feature map.

[0113] More specifically, in step S500 of the training phase, the first fused feature map and the second fused feature map are respectively input into a third convolutional neural network serving as a generator to obtain a generated first image portion and a generated second image portion.

[0114] More specifically, in step S510 of the training phase, a first difference between the generated first image portion and the real first image portion and a second difference between the generated second image portion and the real second image portion are calculated, and the third convolutional neural network, acting as the generator, is trained using the first difference and the second difference. Thus, training the third convolutional neural network as the generator with the first difference and the second difference trains the third convolutional neural network to remove motion information from the fifth and sixth feature maps to generate the first and second image portions.

[0115] In this embodiment of the application, the first difference between the generated first image portion and the real first image portion can be set as the cross-entropy function value between the two, and the second difference between the generated second image portion and the real second image portion can be set as the cross-entropy function value between the two.

[0116] More specifically, in step S610 of the usage phase, the screenshot images of adjacent frames to be stitched are set as the first image and the second image. This step is consistent with step S410 of the training phase, so it will not be described again here.

[0117] More specifically, in step S620 of the usage phase, the first image and the second image are respectively processed by the first convolutional neural network, which has been trained in the training phase and serves as a feature extractor, to obtain a first feature map and a second feature map. This step is consistent with step S420 of the training phase, and therefore will not be described again here.

[0118] More specifically, in step S630 of the usage phase, based on predetermined location information, the regions to be stitched in the first feature map and the second feature map are determined to be the third feature map and the fourth feature map, respectively. The portion of the first feature map excluding the third feature map is designated as the fifth feature map, and the portion of the second feature map excluding the fourth feature map is designated as the sixth feature map. This step is consistent with step S430 of the training phase, and therefore will not be described again here.

[0119] More specifically, in step S640 of the usage phase, the directed edge feature map between the third feature map and the fourth feature map is calculated. This step is consistent with step S440 of the training phase, and therefore will not be described again here.

[0120] More specifically, in step S650 of the usage phase, the directed edge feature map is input into the second convolutional neural network, which has been trained in the training phase and serves as an adversarial generator, to obtain a third image portion corresponding to the stitching region. It should be understood that the trained second convolutional neural network can remove motion information from the directed edge feature map while retaining features in the directed edge feature map that are consistent with the actual image to be stitched, so that the third image portion is closer to the actual image to be stitched.

[0121] More specifically, in step S660 of the usage phase, the difference feature map between the directed edge feature map and the third image portion is calculated as a motion information feature map. It should be understood that motion information is removed from the third image portion, while the directed edge feature map contains motion information; therefore, the difference feature map between the two can represent a distributed representation of motion features, i.e., the motion information feature map.

[0122] More specifically, in step S670 of the usage phase, the fifth feature map and the motion information feature map are fused to obtain a first fused feature map, and the sixth feature map and the motion information feature map are fused to obtain a second fused feature map. This step is consistent with step S490 of the training phase, and therefore will not be described again here.

[0123] More specifically, in step S680 of the usage phase, the first fused feature map and the second fused feature map are respectively input into the third convolutional neural network, which has been trained in the training phase, to obtain a first image portion corresponding to the fifth feature map and a second image portion corresponding to the sixth feature map.

[0124] More specifically, in step S690 of the usage phase, the first image portion, the third image portion, and the second image portion are stitched together. That is, the first image portion, the second image portion, and the third image portion are stitched together with the first image portion on top, the third image portion in the middle, and the second image portion at the bottom to obtain a scrolling screenshot of the complete interface of the target window.

[0125] In summary, the advertising page capture processing method based on the embodiments of this application is explained. It involves scrolling a scroll bar containing a target window and capturing a screenshot of the visible area of ​​the target window after each update. Then, the screenshots captured after each update are stitched together to obtain advertising page data.

[0126] Exemplary device

[0127] Figure 8 The illustration shows a block diagram of an advertising page cropping processing apparatus according to an embodiment of this application.

[0128] like Figure 8 As shown, the advertising page capture processing apparatus 800 according to an embodiment of this application includes: an instruction receiving module 810, configured to receive a screenshot instruction for a target window containing an advertising page displayed on the screen of a terminal; an update module 820, configured to update the value of a scroll bar used to scroll the target window multiple times according to a preset update method, starting from an initial value; a screenshot module 830, configured to capture a screenshot image of the visible area of ​​the target window after each update; and a stitching module 840, configured to stitch the screenshot images captured after each update together to obtain a scrolling screenshot image of the complete interface of the target window.

[0129] In one example, in the aforementioned advertising page capture processing device 800, the update module 820 is further configured to: in response to detecting that the value of the scroll bar is allowed to be modified through the system-level call interface provided by the operating system of the terminal, update the value of the scroll bar multiple times from the initial value according to a preset update method through the system-level call interface.

[0130] In one example, in the aforementioned advertisement page capture processing device 800, such as Figure 9 As shown, the update module 820 includes: an object detection unit 821, used to detect whether the type of the object displayed in the target window is a specified type; an object determination unit 822, used to determine that the target window contains the scroll bar in response to the detection result that the type of the object displayed in the target window is the specified type; a scroll bar detection unit 823, used to detect whether it is allowed to modify the value of the scroll bar through a preset scheduling interface, including: obtaining the class name of the scroll bar, the startup information of the scheduling interface and the setting information of the scroll bar, wherein the startup information of the scheduling interface is used to indicate whether the scheduling interface is enabled, and the setting information of the scroll bar is used to indicate whether the value of the scroll bar is allowed to be modified; and an update unit 824, used to update the value of the scroll bar used to scroll the target window multiple times from the initial value according to a preset update method through the scheduling interface in response to the class name of the scroll bar being a specified class name, the startup information of the scheduling interface indicating that the scheduling interface has been started, and the setting information of the scroll bar indicating that the value of the scroll bar is allowed to be modified.

[0131] In one example, in the aforementioned advertisement page capture processing device 800, such as Figure 10 As shown, the splicing module 840 includes a training unit 850 and a usage unit 870.

[0132] like Figure 10As shown, the training unit 850 includes: a first image setting subunit 851, used to set the screenshot images of adjacent frames to be stitched as a first image and a second image; a first convolutional neural network unit 852, used to pass the first image and the second image through a first convolutional neural network as a feature extractor to obtain a first feature map and a second feature map; a first region determination subunit 853, used to determine the regions to be stitched in the first feature map and the second feature map as a third feature map and a fourth feature map based on predetermined position information, and set the part of the first feature map other than the third feature map as a fifth feature map and the part of the second feature map other than the fourth feature map as a sixth feature map; a first directed edge feature map generation subunit 854, used to calculate the directed edge feature map between the third feature map and the fourth feature map; a first stitching feature map generation subunit 855, used to input the real stitched image corresponding to the directed edge feature map into the first convolutional neural network to obtain a stitched feature map; and a discriminator loss function value calculation subunit 856, used to input the stitched feature map and the directed edge feature map into the discriminator loss function value calculation subunit 856 respectively. A second convolutional neural network serving as an adversarial generator is configured to obtain a discriminator loss function value; a first update subunit 857 is configured to train the second convolutional neural network serving as an adversarial generator based on the discriminator loss function value; a first motion information feature map generation unit 858 is configured to calculate the difference feature map between the directed edge feature map and the spliced ​​feature map as a motion information feature map; a first feature map fusion subunit 859 is configured to fuse the fifth feature map and the motion information feature map to obtain a first fused feature map, and fuse the sixth feature map and the motion information feature map to obtain a second fused feature map; a first image generation subunit 860 is configured to input the first fused feature map and the second fused feature map into a third convolutional neural network serving as a generator to obtain a generated first image portion and a generated second image portion; and a second update subunit 861 is configured to calculate a first difference between the generated first image portion and the real first image portion and a second difference between the generated second image portion and the real second image portion, and train the third convolutional neural network serving as a generator using the first difference and the second difference.

[0133] like Figure 10As shown, the usage unit 870 includes: a second image setting subunit 871, used to set the screenshot images of adjacent frames to be stitched as a first image and a second image; a second convolutional neural network subunit 872, used to pass the first image and the second image through the first convolutional neural network, which is a feature extractor trained during the training phase, to obtain a first feature map and a second feature map; a second region determination subunit 873, used to determine the regions to be stitched in the first feature map and the second feature map as a third feature map and a fourth feature map based on predetermined position information, and set the part of the first feature map other than the third feature map as a fifth feature map and the part of the second feature map other than the fourth feature map as a sixth feature map; a second directed edge feature map generation subunit 874, used to calculate the directed edge feature map between the third feature map and the fourth feature map; and an adversarial generation subunit 875, used to output the directed edge feature map. The system employs a second convolutional neural network, trained during the training phase, to generate an adversarial generator, which is used to obtain a third image portion corresponding to the stitching region. A second motion information feature map generation subunit 876 is used to calculate the difference feature map between the directed edge feature map and the third image portion as a motion information feature map. A second feature map fusion subunit 877 is used to fuse the fifth feature map and the motion information feature map to obtain a first fused feature map, and to fuse the sixth feature map and the motion information feature map to obtain a second fused feature map. A second image generation subunit 878 is used to input the first fused feature map and the second fused feature map into the third convolutional neural network, trained during the training phase, to generate a first image portion corresponding to the fifth feature map and a second image portion corresponding to the sixth feature map. Finally, an image stitching subunit 879 is used to stitch the first image portion, the third image portion, and the second image portion together.

[0134] In one example, in the aforementioned advertising page capture processing device 800, the first region determination subunit 853 is further configured to: determine the region to be spliced ​​in the first feature map and the second feature map based on the position of the scroll bar.

[0135] In one example, in the aforementioned advertisement page capture processing device 800, the first directed edge feature map generation subunit 854 is further configured to calculate the directed edge feature map between the third feature map and the fourth feature map using the following formula:

[0136]

[0137] Wherein, n is a hyperparameter, which is positively correlated with the scrolling speed of the target window.

[0138] In one example, in the aforementioned advertisement page capture processing apparatus 800, the discriminator loss function value calculation subunit 856 is further configured to input the spliced ​​feature map into the second convolutional neural network to obtain a seventh feature map; input the directed edge feature map into the second convolutional neural network to obtain an eighth feature map; determine whether the values ​​at predetermined positions in the seventh feature map and the eighth feature map are the same; in response to the seventh feature map and the eighth feature map having the same value at predetermined positions, calculate the negative value of the base-2 logarithm of the value at the predetermined positions as a first value; in response to the seventh feature map and the eighth feature map having different values ​​at predetermined positions, calculate the base-2 logarithm of the value at the predetermined positions as a second value; and calculate the sum of the average value of the first value at positions with the same value and the average value of the second value at positions with different values ​​as the discriminator loss function value.

[0139] Here, those skilled in the art will understand that the specific functions and operations of each unit and module in the aforementioned advertising page capture and processing device 800 have been referenced above. Figures 1 to 7 The method for capturing and processing advertising pages is described in detail in the description, and therefore, its repeated description will be omitted.

[0140] As described above, the advertising page capture processing device 800 according to the embodiments of this application can be implemented in various terminal devices, such as internet advertising monitoring servers. In one example, the advertising page capture processing device 800 according to the embodiments of this application can be integrated into the terminal device as a software module and / or hardware module. For example, the advertising page capture processing device 800 can be a software module in the operating device of the terminal device, or it can be an application developed for the terminal device; of course, the advertising page capture processing device 800 can also be one of many hardware modules of the terminal device.

[0141] Alternatively, in another example, the advertisement page capture processing device 800 and the terminal device can also be separate devices, and the advertisement page capture processing device 800 can be connected to the terminal device via wired and / or wireless networks, and transmit interactive information in accordance with an agreed data format.

[0142] Exemplary electronic devices

[0143] Below, for reference Figure 11 This describes an electronic device according to embodiments of the present application.

[0144] Figure 11 A block diagram of an electronic device according to an embodiment of this application is illustrated.

[0145] like Figure 11As shown, the electronic device 10 includes one or more processors 11 and memory 12.

[0146] The processor 11 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0147] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may execute the program instructions to implement the functions in the advertising page capture processing methods of the various embodiments of this application described above, and / or other desired functions. Various content such as screenshot images, first image portions, second image portions, and third image portions may also be stored in the computer-readable storage medium.

[0148] In one example, the electronic device 10 may also include an input device 13 and an output device 14, which are interconnected via a bus device and / or other forms of connection mechanism (not shown).

[0149] The input device 13 may include, for example, a keyboard, a mouse, etc.

[0150] The output device 14 can output various information to the outside, including a first image portion, a second image portion, a third image portion, and a scrolling screenshot image, etc. The output device 14 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0151] Of course, for the sake of simplicity, Figure 11 Only some of the components of the electronic device 10 relevant to this application are shown in this illustration; components such as buses, input / output interfaces, etc., are omitted. In addition, the electronic device 10 may include any other suitable components depending on the specific application.

[0152] Exemplary computer program products and computer-readable storage media

[0153] In addition to the methods and devices described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps of the functions in the advertising page capture processing methods according to various embodiments of this application described in the "Exemplary Methods" section of this specification.

[0154] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0155] Furthermore, embodiments of this application may also be computer-readable storage media storing computer program instructions that, when executed by a processor, cause the processor to perform the steps of the advertising page interception processing method according to various embodiments of this application as described in the "Exemplary Methods" section above.

[0156] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor device, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

Claims

1. A method for extracting and processing advertising pages, characterized in that, include: Receive a screenshot command for the target window containing the advertising page displayed on the terminal's screen; Starting from the initial value, the value of the scroll bar used to scroll the target window is updated multiple times according to a preset update method; Capture a screenshot of the visible area of ​​the target window after each update; as well as The screenshots captured after each update are stitched together to obtain a scrolling screenshot of the complete interface of the target window. Specifically, the screenshots captured after each update are stitched together to obtain a scrolling screenshot of the complete interface of the target window, including: The usage phase includes: The screenshot images of adjacent frames to be stitched are set as the first image and the second image; The first image and the second image are respectively passed through a first convolutional neural network, which has been trained in the training phase, to obtain a first feature map and a second feature map. Based on predetermined location information, the regions to be spliced ​​in the first feature map and the second feature map are determined to be the third feature map and the fourth feature map, respectively. The portion of the first feature map other than the third feature map is set as the fifth feature map, and the portion of the second feature map other than the fourth feature map is set as the sixth feature map. Calculate the directed edge feature graph between the third feature graph and the fourth feature graph; The directed edge feature map is input into the second convolutional neural network, which has been trained during the training phase and serves as an adversarial generator, to obtain the third image portion corresponding to the splicing region. The difference feature map between the directed edge feature map and the third image portion is calculated as the motion information feature map; The fifth feature map and the motion information feature map are fused to obtain a first fused feature map, and the sixth feature map and the motion information feature map are fused to obtain a second fused feature map; The first fused feature map and the second fused feature map are respectively input into the third convolutional neural network, which is a generator and has been trained in the training phase, to obtain a first image part corresponding to the fifth feature map and a second image part corresponding to the sixth feature map; The first image portion, the third image portion, and the second image portion are stitched together; The calculation of the directed edge feature map between the third feature map and the fourth feature map includes: The directed edge feature map between the third feature map and the fourth feature map is calculated using the following formula, wherein the formula is: ； Wherein, n is a hyperparameter, which is positively correlated with the scrolling speed of the target window.

2. The advertising page cropping method according to claim 1, wherein, Starting from the initial value, the value of the scroll bar used to scroll the target window is updated multiple times according to a preset update method, including: In response to detecting that the value of the scroll bar is allowed to be modified through the system-level call interface provided by the terminal's operating system, the value of the scroll bar is updated multiple times from the initial value according to a preset update method through the system-level call interface.

3. The advertising page capture and processing method according to claim 1, wherein, Starting from the initial value, the value of the scroll bar used to scroll the target window is updated multiple times according to a preset update method, including: Detect whether the type of the object displayed in the target window is the specified type; In response to the detection result indicating that the type of the object displayed in the target window is the specified type, it is determined that the target window contains the scroll bar; Detecting whether modification of the scroll bar value is allowed through a preset API call includes: obtaining the scroll bar's class name, the API call's startup information, and the scroll bar's setting information. The API call's startup information indicates whether the API call is enabled, and the scroll bar's setting information indicates whether the scroll bar's value is allowed to be modified. In response to the class name of the scroll bar being a specified class name, the activation information of the calling interface indicating that the calling interface has been started, and the setting information of the scroll bar indicating that the value of the scroll bar can be modified, the value of the scroll bar used to scroll the target window is updated multiple times from the initial value according to a preset update method through the calling interface.

4. The advertising page capture and processing method according to claim 1, wherein, The screenshots captured after each update are stitched together to obtain a scrolling screenshot of the complete interface of the target window, including: The training phase includes: The screenshot images of adjacent frames to be stitched are set as the first image and the second image; The first image and the second image are respectively passed through a first convolutional neural network as a feature extractor to obtain a first feature map and a second feature map; Based on predetermined location information, the regions to be spliced ​​in the first feature map and the second feature map are determined to be the third feature map and the fourth feature map, respectively. The portion of the first feature map other than the third feature map is set as the fifth feature map, and the portion of the second feature map other than the fourth feature map is set as the sixth feature map. Calculate the directed edge feature graph between the third feature graph and the fourth feature graph; The real stitched image corresponding to the directed edge feature map is input into the first convolutional neural network to obtain the stitched feature map; The concatenated feature map and the directed edge feature map are respectively input into the second convolutional neural network used as an adversarial generator to obtain the discriminator loss function value; The second convolutional neural network, which serves as an adversarial generator, is trained based on the discriminator loss function value. The difference feature map between the directed edge feature map and the concatenated feature map is calculated as the motion information feature map; The fifth feature map and the motion information feature map are fused to obtain a first fused feature map, and the sixth feature map and the motion information feature map are fused to obtain a second fused feature map; The first fused feature map and the second fused feature map are respectively input into the third convolutional neural network used as a generator to obtain the generated first image part and the generated second image part; The first difference between the generated first image portion and the real first image portion and the second difference between the generated second image portion and the real second image portion are calculated, and the third convolutional neural network, which serves as the generator, is trained using the first difference and the second difference.

5. The advertising page cropping method according to claim 4, wherein, Based on predetermined location information, the regions to be spliced ​​in the first and second feature maps are determined as the third and fourth feature maps, including: Based on the position of the scroll bar, the regions to be spliced ​​in the first feature map and the second feature map are determined.

6. The advertising page capture and processing method according to claim 5, wherein, The concatenated feature map and the directed edge feature map are respectively input into the second convolutional neural network used as an adversarial generator to obtain the discriminator loss function value, including: The spliced ​​feature map is input into the second convolutional neural network to obtain the seventh feature map; The directed edge feature map is input into the second convolutional neural network to obtain the eighth feature map; Determine whether the values ​​at predetermined positions in the seventh feature map and the eighth feature map are the same; In response to the fact that the values ​​at predetermined positions in the seventh feature map and the eighth feature map are the same, the negative value of the base-2 logarithm of the value at the predetermined position is calculated as the first value. In response to the difference between the values ​​at a predetermined position in the seventh feature map and the eighth feature map, the base-2 logarithm of the value at the predetermined position is calculated as the second value; The sum of the average value of the first value at positions with the same value and the average value of the second value at positions with different values ​​is calculated as the discriminator loss function value.

7. An advertising page cropping and processing apparatus, used to execute the advertising page cropping and processing method according to claim 1, characterized in that, include: The instruction receiving module is used to receive screenshot instructions for the target window containing the advertisement page displayed on the terminal's screen. The update module is used to update the value of the scroll bar used to scroll the target window multiple times according to a preset update method, starting from the initial value; The screenshot module is used to capture a screenshot image of the visible area of ​​the target window after each update; as well as The stitching module is used to stitch together the screenshots captured after each update to obtain a scrolling screenshot of the complete interface of the target window.

8. The advertising page capture and processing device according to claim 7, wherein, The update module is further configured to: in response to detecting that the value of the scroll bar is allowed to be modified through the system-level call interface provided by the operating system of the terminal, update the value of the scroll bar multiple times from the initial value according to a preset update method through the system-level call interface.

9. An electronic device, comprising: processor; as well as A memory storing computer program instructions that, when executed by the processor, cause the processor to perform the advertising page interception processing method as described in any one of claims 1-6.

Images