Information aggregation display method and device, storage medium and processor

Abstract

The invention discloses an information aggregation display method and device, a storage medium and a processor. In the scheme, M target images to be displayed are obtained, and each target image is an image intercepted when the picture content of an original layer of a display interface of an application program is updated once; an effective area image in each target image is extracted, N effective area images are obtained, and one effective area image corresponds to one complete content display area; aggregating the N effective area images to obtain an aggregated image; updating information in a target layer based on the aggregated image, wherein the target layer is a layer which is created in advance and is arranged on the original layer; and displaying the updated information in the target layer in the display interface. According to the technical scheme, the image content in the display interface of the application program can be aggregated and displayed without depending on an SDK provided by a manufacturer, so that the universality of an information aggregation display technology is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an information aggregation display method, device, storage medium and processor. Background Art

[0002] Information aggregation display refers to the integration and display of information from multiple sources or different types (such as video, text, pictures, etc.) on a unified interface or platform, providing users with a comprehensive and efficient information overview, allowing users to view a large amount of information at one time.

[0003] The existing information aggregation display method usually integrates the acquired third-party software development kit (Software Development Kit, SDK) into the application to realize the aggregate display of the screen content in the display interface of the application. However, the above-mentioned information aggregation display method relies on the third-party SDK, and it is necessary to obtain the third-party SDK from the third-party manufacturer before the information aggregation display can be realized. If the third-party manufacturer refuses to provide the relevant SDK or problems arise during the use of the SDK, it is difficult to achieve the purpose of information aggregation display. It can be seen that the current information aggregation display technology has the problem of poor versatility because it relies on the SDK implementation provided by third-party manufacturers. Summary of the invention

[0004] Based on the above problems, the present application provides an information aggregation display method, device, storage medium and processor, the purpose of which is to improve the versatility of information aggregation display.

[0005] The embodiments of the present application disclose the following technical solutions:

[0006] In a first aspect of the present application, a method for displaying information aggregation is provided, the method comprising:

[0007] Obtain M target images to be displayed, wherein each target image is an image captured when the original layer updates the screen content once, each target image includes multiple content display areas, the original layer is the layer corresponding to the display interface of the application, and M is a positive integer;

[0008] Extracting a valid area image from each target image to obtain a total of N valid area images, wherein one valid area image corresponds to a complete content display area, and N is an integer greater than or equal to M;

[0009] Aggregate N valid area images to obtain an aggregated image;

[0010] Update information in a target layer based on the aggregated image, wherein the target layer is a layer created in advance and set on the original layer;

[0011] The updated information in the target layer is displayed in the display interface.

[0012] In an optional implementation, a valid area image in each target image is extracted to obtain a total of N valid area images, including:

[0013] Identify boundary coordinate information of multiple content display areas in each target image, and obtain multiple boundary coordinate information corresponding to each target image;

[0014] Performing image segmentation on the target image based on multiple boundary coordinate information corresponding to each target image to obtain a total of L regional images, where L is a positive integer and L is greater than N;

[0015] The L region images are screened for completeness of the content display region to obtain N valid region images.

[0016] In an optional implementation, obtaining M target images to be displayed includes:

[0017] In response to the content update instruction, a first touch operation is performed on the original layer, wherein the content update instruction is an instruction generated when the user clicks a content update button in the target layer, and the first touch operation is used to simulate a sliding operation of the user in the original layer to update the screen content in the original layer;

[0018] Perform an image capture operation to capture the updated screen content in the original layer to obtain a target image;

[0019] The first touch operation and the image capture operation are performed multiple times until M target images are obtained.

[0020] In an optional implementation, after displaying the updated information in the target layer in the display interface, the information aggregation display method further includes:

[0021] In response to the information interaction instruction, a unique identifier corresponding to the target effective area image is obtained, wherein the target effective area image is an updated effective area image in the target layer clicked by the user;

[0022] A second touch operation is performed on the original layer according to the unique identifier, wherein the second touch operation is used to simulate the user's touch operation in the original layer, and the associated content corresponding to the target effective area image is displayed in the display interface.

[0023] In an optional implementation, when the content browsing mode of the application is a multi-column browsing mode, performing a second touch operation on the original layer according to the unique identifier includes:

[0024] Acquire the region position information corresponding to the target effective region image based on the unique identifier, wherein the region position information is the position information of the screen content corresponding to the target effective region image in the original layer when the image interception operation is performed on the original layer;

[0025] A second touch operation is performed on the original layer based on the region position information.

[0026] In an optional implementation, when the content browsing mode of the application is a single-column browsing mode, performing a second touch operation on the original layer according to the unique identifier includes:

[0027] Acquire an update order corresponding to the target effective area image based on the unique identifier, wherein the update order is an operation execution order when performing a first touch operation on the original layer to obtain the target effective area image;

[0028] Perform a second touch operation on the original layer according to the update order.

[0029] In an optional implementation, the associated content corresponding to the target effective area image is video content, and during the process of performing the second touch operation on the original layer according to the update order, the information aggregation display method further includes:

[0030] The playback status of all video contents triggered during the execution of the second touch operation is set to a mute state.

[0031] In a second aspect of the present application, an information aggregation display device is provided, the device comprising:

[0032] An acquisition module is used to acquire M target images to be displayed, wherein each target image is an image captured when the original layer updates the screen content once, each target image includes multiple content display areas, the original layer is the layer corresponding to the display interface of the application, and M is a positive integer;

[0033] An image extraction module, used to extract a valid area image from each target image, to obtain a total of N valid area images, wherein one valid area image corresponds to a complete content display area, and N is an integer greater than or equal to M;

[0034] An image aggregation module, used for aggregating N valid area images to obtain an aggregated image;

[0035] An image updating module, used for updating information in a target layer based on the aggregated image, wherein the target layer is a layer created in advance and set on the original layer;

[0036] The display module is used to display the updated information in the target layer in the display interface.

[0037] In a third aspect of the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the information aggregation display method introduced in the first aspect is implemented.

[0038] In a fourth aspect of the present application, a processor is provided for running a computer program, and when the computer program is running, the information aggregation display method introduced in the first aspect is executed.

[0039] Compared with the prior art, this application has the following beneficial effects:

[0040] In the technical solution of the present application, M target images captured when the original layer of the display interface of the application updates the screen content are obtained, and the effective area image corresponding to the complete content display area in each target image is extracted to obtain a total of N effective area images; then the N effective area images are aggregated to obtain an aggregated image; then the information in the target layer is updated based on the aggregated image, and the updated information in the target layer is displayed in the display interface. The technical solution of the present application realizes the aggregation and display of the screen content in the display interface of the application directly through the effective area image in the target image captured when the screen content is updated by the acquired original layer, and the target layer pre-created and set on the original layer. In other words, in this solution, the screen content in the display interface of the application can be aggregated and displayed without relying on the SDK provided by the manufacturer. Thereby improving the versatility of the information aggregation display technology, and then solving the technical problem that the prior art relies on the SDK provided by the manufacturer to realize information aggregation display, and has poor versatility. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0042] Figure 1 A flowchart of an information aggregation display method provided in an embodiment of the present application;

[0043] Figure 2A A schematic diagram of an information aggregation display effect provided in an embodiment of the present application;

[0044] Figure 2B A schematic diagram of another information aggregation display effect provided in an embodiment of the present application;

[0045] Figure 2CA schematic diagram of another information aggregation display effect provided in an embodiment of the present application;

[0046] Figure 3 A flowchart of a process of obtaining a target image to be displayed provided in an embodiment of the present application;

[0047] Figure 4 A flowchart of a process of responding to a content update instruction provided in an embodiment of the present application;

[0048] Figure 5 A flowchart of a process for extracting an effective area image provided by an embodiment of the present application;

[0049] Figure 6 A flowchart of an image preprocessing process provided in an embodiment of the present application;

[0050] Figure 7 A schematic diagram of the structure of an original layer and a system layer provided in an embodiment of the present application;

[0051] Figure 8 A schematic diagram of a display interface provided in an embodiment of the present application showing updated information in a target layer;

[0052] Fig. 9 A flowchart of a process for displaying image-related content provided in an embodiment of the present application;

[0053] Fig.10 A schematic diagram of a short video content aggregation display provided in an embodiment of the present application;

[0054] Fig.11 A schematic diagram of a quick browsing carousel effect after short video content is aggregated and displayed provided in an embodiment of the present application;

[0055] Fig.12 A schematic diagram of the structure of an information aggregation display device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] As described above, the current information aggregation display method usually integrates the acquired third-party software development kit into the application to realize the aggregate display of the screen content in the display interface of the application. However, the above-mentioned information aggregation display method relies on the third-party SDK, and it is necessary to obtain the third-party SDK from the third-party manufacturer before the information aggregation display can be realized. If the third-party manufacturer refuses to provide the relevant SDK or problems arise during the use of the SDK, it is difficult to achieve the purpose of information aggregation display. It can be seen that the current information aggregation display technology has the problem of poor versatility because it relies on the SDK implementation provided by third-party manufacturers.

[0057] After research, the inventor proposed a method of intercepting the target image when updating the screen content in the original layer of the display interface of the application, and extracting the effective area image corresponding to the complete content display area in the target image; then aggregating N effective area images to obtain an aggregated image, and based on the aggregated image, updating the information in the target layer that is pre-created and set above the original layer, so as to display the updated information in the target layer in the display interface. In this solution, the screen content in the display interface of the application can be aggregated and displayed without relying on the SDK provided by the manufacturer. This improves the versatility of information aggregation display, thereby solving the technical problem that the prior art relies on the SDK provided by the manufacturer to realize information aggregation display and has poor versatility.

[0058] It should be noted that the information aggregation display method provided in the embodiment of the present application can be directly applied to the operating system of the terminal device, and the information aggregation display can be realized without the manufacturer providing SDK. Among them, the terminal device includes but is not limited to smart phones, tablet computers, laptops, desktops, embedded devices, car computers and other devices. Different terminal devices can choose different types of operating systems according to their uses and performance requirements, so they are not specifically limited here.

[0059] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0060] Method Embodiment

[0061] The present application provides an embodiment of an information aggregation display method. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions. In addition, although the logical order is shown in the flowchart, it is only an example order provided. In some cases, the steps shown or described can be executed in a different order than here.

[0062] Figure 1 A flowchart of an information aggregation display method provided in an embodiment of the present application, such as Figure 1 As shown, the method comprises the following steps:

[0063] Step S101, obtaining M target images to be displayed.

[0064] In an optional embodiment, an information aggregation display system can be used as the execution subject of the information aggregation display method of the embodiment of the present application. For the convenience of description, the information aggregation display system is referred to as the system below.

[0065] In step S101, the original layer is the layer corresponding to the display interface of the application, each target image includes multiple content display areas, M is a positive integer; wherein the content browsing mode of the application includes a single-column browsing mode and a multi-column browsing mode, and the single-column browsing mode can be as follows: Figure 2A As shown, the application's display interface only displays one column of content display area; the multi-column browsing mode can be Figure 2B , Figure 2C As shown, the display interface of the application can display two columns of content display areas, three columns of content display areas, etc.

[0066] In step S101, each target image is an image captured when the original layer updates the screen content once, for example, Figure 2B The display interface shown includes areas (i.e., content display areas) 1 to 6, whose screen contents are the screen contents of the original layer of the display interface updated for the first time, and areas 6 to 11, whose screen contents are the screen contents of the original layer of the display interface updated for the second time. The system can take screenshots of the screen contents after the first and second updates, and obtain the target images corresponding to the screen contents updated for the first time and the target images corresponding to the screen contents updated for the second time, respectively.

[0067] In order to avoid the problem of difficulty in achieving the purpose of information aggregation display due to the inability to obtain relevant SDK or problems with relevant data during the use of SDK, the system of the present application can directly obtain the image data required for information aggregation display (i.e., M target images) without relying on relevant data in a third-party SDK. Figure 3 A flowchart of a process of obtaining a target image to be displayed provided in an embodiment of the present application is as follows: Figure 3 As shown, the process includes the following steps:

[0068] Step S1011, in response to the content update instruction, performing a first touch operation on the original layer, wherein the first touch operation is used to simulate a sliding operation of the user in the original layer to update the screen content in the original layer.

[0069] In step S1011, the content update instruction is an instruction generated when the user clicks the content update button in the target layer. The first touch operation refers to an operation simulating the user sliding up the page in the original layer.

[0070] In this embodiment, if Figure 4As shown, a "change batch" content update button is set in the system layer (i.e., the target layer mentioned above). When the user clicks the content update button, the system responds to the content update instruction and simulates the user's operation of sliding the page up in the original layer to update the screen content in the original layer. The updated original layer includes Figure 4 In other words, when the user clicks the content update button in the target layer, the system can intercept the user's click event (i.e., the operation of clicking the content update button in the target layer) through the system layer, so that the click event will not be passed to the application layer, avoiding the problem of interaction confusion caused by the application layer mistaking the click event for its own operation; then the system can input the sliding event (i.e., simulating the sliding operation of the user in the original layer) to the original layer to update the screen content in the original layer.

[0071] Step S1012, performing an image capture operation to capture the updated screen content in the original layer to obtain a target image.

[0072] For example, the system simulates the user's operation of sliding the page upward in the original layer, and updates the screen content in the original layer to include Figure 4 When the screen content of content display area 1 to content display area 9 shown in the figure is displayed, the system can take a screenshot of the screen content to obtain a target image corresponding to the screen content.

[0073] It should be noted that when the content in the original layer is video content, when the system performs the first touch operation on the original layer, the system can simulate the user's sliding operation in the original layer to click on the video content, so that the video content is paused, so that the system can capture the key pictures required for aggregate display when performing the image capture operation.

[0074] Step S1013, performing the first touch operation and the image capture operation multiple times until M target images are obtained.

[0075] In order to obtain all target images required for aggregate display and achieve the purpose of information aggregate display, in this embodiment, the system may perform the first touch operation and image capture operation on the original layer multiple times until M target images are obtained. Figure 2AAs shown, after the system performs the first touch operation and the image capture operation on the original layer, the system can continue to perform the second first touch operation on the original layer to update the original layer from the screen content corresponding to the content display area 1 to the screen content corresponding to the content display area 2; then the system can perform the second image capture operation on the updated original layer to capture the screen content corresponding to the content display area 2 in the original layer to obtain the image 2 corresponding to the content display area 2; and so on until the image 2 corresponding to the content display area 2 is obtained. Figure 2A Content display areas 3 to 9 shown in FIG. 1 correspond to images 3 to 9 , respectively.

[0076] Step S102 , extracting the effective area image in each target image to obtain a total of N effective area images.

[0077] In step S102, one valid area image corresponds to one complete content display area, and N is an integer greater than or equal to M. For example, Figure 2C In the display interface shown in , area 2 (i.e., content display area) and areas 4-6 in the original layer are complete content display areas, and their corresponding area images are valid area images; area 1, area 3, and areas 7-8 in the original layer are incomplete content display areas. Figure 2C In the display interface shown in , the system simulates the user's operation of sliding the page upward in the original layer to update the screen content in the original layer. Areas 7 to 10 in the updated original layer are complete content display areas, and their corresponding area images are valid area images; areas 12 to 13 in the updated original layer are incomplete content display areas.

[0078] When displaying screen content on the display interface of an application, the prior art usually displays incomplete information in the display interface, taking advantage of the user's desire to explore information and guiding the user to continuously refresh and view new information, thereby increasing the user's stay time and click rate. However, when the user uses an electronic device to continuously refresh and view new information, not only does the user need to frequently scroll up and down or refresh the information, but also stares at the electronic device for a long time, which causes the user to suffer from eye health problems such as visual fatigue and myopia, thereby reducing the user's experience.

[0079] In order to solve the above problems, the system of the present application can be Figure 5 The process shown extracts the effective area image corresponding to the complete content display area in the target image, so as to display the complete information in the display interface of the application. Figure 5 A flowchart of a process of extracting an effective area image provided by an embodiment of the present application is shown in FIG. Figure 5 As shown, the process includes the following steps:

[0080] Step S1021 : identifying boundary coordinate information of multiple content display areas in each target image, and obtaining multiple boundary coordinate information corresponding to each target image.

[0081] In this embodiment, if Figure 6 As shown, the system can firstly perform grayscale processing on the original image (i.e. each target image) and convert the color target image into a grayscale image, wherein the three channels of RGB (full name: Red, Green, Blue) are changed into a single grayscale channel, and the grayscale value is set to 0-255; secondly, in order to improve the image quality, the system can perform noise reduction processing on the grayscale image to remove the "burrs" and "noises" in the image to obtain a grayscale image after noise reduction; then the system can perform binarization processing on the grayscale image after noise reduction, and perform a black-or-white operation on the image composed of grayscale pixels 0-255 according to the set threshold range to obtain a processed image; finally, the system can identify the black-and-white pixel change area in the processed image as a boundary (i.e. Figure 6 The boundary coordinate information of multiple content display areas in each target image is obtained based on the boundaries in the processed image.

[0082] It should be noted that by finding the edges of multiple content display areas in each target image to obtain boundary coordinate information of multiple content display areas in each target image, accurate segmentation data can be provided for subsequent image segmentation, thereby improving the accuracy of effective area image extraction.

[0083] Step S1022 , performing image segmentation on each target image based on a plurality of boundary coordinate information corresponding to the target image, to obtain a total of L regional images.

[0084] In step S1022, L is a positive integer, and L is greater than N.

[0085] For example, taking the above Figure 2B Taking the display interface shown in as an example, when the system performs image segmentation on the target image corresponding to the screen content of the original layer of the display interface updated for the first time, the system can obtain regional images corresponding to areas (i.e., content display areas) 1 to 6 respectively; when the system performs image segmentation on the target image corresponding to the screen content of the original layer of the display interface updated for the second time, the system can obtain regional images corresponding to areas 6 to 11 respectively.

[0086] Step S1023, screening the integrity of the content display area of ​​the L area images to obtain N valid area images.

[0087] In step S1023, the system may screen the integrity of the content display area by the area and / or shape of each of the L regional images to obtain N valid regional images. The system may calculate the area of ​​each of the L regional images; screen out regional images with an area equal to a preset area from the L regional images, and / or screen out regional images with a shape identical to a preset shape from the L regional images to obtain N valid regional images. Among them, the preset area is the same as the area of ​​the regional image corresponding to the complete content display area (i.e., the valid regional image), and the preset shape is the same as the shape of the regional image corresponding to the complete content display area. Based on the above Figure 2B Taking the display interface shown in as an example, the system can calculate the areas of the regional images corresponding to areas 1 to 6 respectively, and by comparing the area of ​​each regional image with the preset area, it is found that the areas of the regional images corresponding to areas 2 to 5 are the same as the preset area, then the regional images corresponding to areas 2 to 5 can be determined as valid regional images.

[0088] Alternatively, pre-set the preset shape to Figure 2B The system can compare the shapes of the regional images corresponding to regions 2 to 5 in the image data with the preset shapes, and if the shapes of the regional images corresponding to regions 2 to 5 are the same as the preset shapes, the regional images corresponding to regions 2 to 5 can be determined as valid regional images. In addition, in order to improve the accuracy of valid regional image recognition, the factors of area and shape can also be considered, and the regional images whose area and shape meet the preset area and preset shape requirements can be regarded as valid regional images.

[0089] Optionally, in order to further avoid long-term gaze on electronic devices, which may lead to visual fatigue and myopia in users, and improve the user experience, this application sets a content display area for browsing time and browsing percentage in the target layer, and displays the length of time users browse content in the application through the above content display area, so as to remind users to stop browsing content in the application appropriately. Figure 2A The content display areas of "Cumulative time", "Continuous time" and "Today's proportion" set in can be used to display the time information of users browsing content in the application, among which the cumulative time is the total cumulative time that users browse content in the application on that day, the continuous time is the total time that users browse content in the application continuously on that day, and the today's proportion is the percentage of the time that users browse content in the application on the terminal device on that day in the total time of using the terminal device.

[0090] Step S103, aggregate the N valid area images to obtain an aggregated image.

[0091] In this embodiment, the system aggregates N effective area images by image stitching and combination to obtain an aggregated image. It should be noted that the system can set the number of effective area images that can be displayed in the system layer according to the needs of the user and the display screen size of the terminal device, wherein the terminal device can be any device with a display screen, such as a smart phone, a tablet computer, a car computer, etc. For example, when the number of effective area images that can be displayed in the system layer is set to 9, the arrangement of the aggregated images can be as follows: Figure 2A The arrangement of images 1 to 9 in the system layer on the right (i.e., effective area images) is shown as follows; when the number of effective area images that can be displayed in the system layer is set to 8, the arrangement of the aggregated images can be as follows Figure 2B The arrangement of images (effective area images) 2 to 9 in the system layer on the right is shown.

[0092] Step S104, updating the information in the target layer based on the aggregated image.

[0093] In step S104, the target layer is a layer that is pre-created and set above the original layer. Figure 7 As shown, the original layer can be located below the system layer (ie, the target layer), the system layer is located above the original layer, and the system layer can cover the content screen in the original layer. For example, the aggregated image includes Figure 7 When the system updates the information in the target layer based on the aggregated image, the updated target layer can be shown as follows: Figure 7 As shown in the system layer in . For another example, the aggregated image includes Figure 2C As shown, when the target image 1 includes regions 1 to 8 in the original layer (i.e., content display regions), and the target image 2 includes regions 7 to 13 in the original layer, the system can convert the effective area images (i.e., Figure 2C The image 2, image 4 to image 10 in the image are aggregated to obtain an aggregated image, and then the information in the target layer can be updated based on the aggregated image. The updated target layer can be Figure 2C The aggregation display effect is shown in .

[0094] That is, the system in the present application can aggregate and display the effective area images in one target image, and can also aggregate and display the effective area images in multiple target images.

[0095] Step S105, displaying the updated information in the target layer in the display interface.

[0096] In this embodiment, the updated information in the target layer is displayed in the display interface as follows: Figure 8 As shown, the system can display the effective area image on the display.

[0097] Based on the scheme introduced in the above steps S101 to S105, it can be known that in the present application, by acquiring M target images captured when the original layer of the display interface of the application updates the screen content, and extracting the effective area image corresponding to the complete content display area in each target image, a total of N effective area images are obtained; then the N effective area images are aggregated to obtain the aggregated image; then the information in the target layer is updated based on the aggregated image, and the updated information in the target layer is displayed in the display interface; the effective area image in the target image captured when the screen content is updated directly through the acquired original layer, and the target layer pre-created and set on the original layer, the screen content in the display interface of the application is aggregated and displayed, and the screen content in the display interface of the application can be aggregated and displayed without relying on the SDK provided by the manufacturer. Thereby, the versatility of information aggregation display is improved, and the technical problem that the prior art relies on the SDK provided by the manufacturer to realize information aggregation display and has poor versatility is solved.

[0098] In order to further improve the versatility of information aggregation display and thus improve the user experience, the system in this application can display the updated information in the target layer in the display interface. Fig. 9 In the process of displaying the associated content shown in the figure, the associated content corresponding to the target effective area image clicked by the user is displayed in the display interface, wherein the associated content corresponding to the target effective area image can be any one of the graphic descriptions, short videos and other contents related to the target effective area image. Fig. 9 As shown, the process includes the following steps:

[0099] Step S901, in response to the information interaction instruction, obtaining a unique identifier corresponding to the target effective area image.

[0100] In step S901, the target effective area image is the effective area image updated in the target layer clicked by the user. The unique identifier corresponding to the target effective area image is a specific code or string that uniquely identifies the image, for example, the name of the target effective area image.

[0101] In this embodiment, the information interaction instruction is an instruction generated when the user clicks on the target effective area image in the system layer. For example, when the user clicks Figure 4When a valid area image 1 is displayed in the system layer, the system can generate an instruction for displaying the associated content corresponding to the valid area image 1 in the display interface, and obtain the name of the valid area image 1 in response to the instruction.

[0102] Step S902: Perform a second touch operation on the original layer according to the unique identifier, wherein the second touch operation is used to simulate the user's touch operation in the original layer, and display the associated content corresponding to the target effective area image in the display interface.

[0103] Optionally, when the content browsing mode of the application is a multi-column browsing mode, the system performs a second touch operation on the original layer based on the unique identifier, including: the system can obtain the area position information corresponding to the target effective area image based on the unique identifier, wherein the area position information is the position information of the screen content corresponding to the target effective area image in the original layer when the image capture operation is performed on the original layer; then the system performs the second touch operation on the original layer based on the area position information.

[0104] Click with the above user Figure 4 Take the valid area image 1 in the system layer as an example, in the content browsing mode of the application Figure 4 In the multi-column browsing mode shown in , the system can obtain the regional position information corresponding to the effective area image 1 according to the name of the above-mentioned effective area image 1. The regional position information is the spatial position information of the screen content corresponding to the effective area image 1 in the original layer when the system simulates the sliding operation of the user in the original layer to update the screen content in the original layer, and performs the image capture operation to obtain the target image where the effective area image 1 is located. After obtaining the regional position information corresponding to the effective area image 1, the system can, based on the regional position information, slide the content of the original layer to the position of the screen content corresponding to the effective area image 1 in the original layer by simulating the touch operation of the user in the original layer, and click the content display area corresponding to the effective area image 1 in the original layer, and display the graphic description or short video and other related content related to the effective area image 1 in the display interface, so as to realize the playback, jump, and detailed browsing of the content.

[0105] Optionally, when the content browsing mode of the application is a single-column browsing mode, a second touch operation is performed on the original layer based on the unique identifier, including: obtaining an update order corresponding to the target effective area image based on the unique identifier, wherein the update order is the order in which operations are executed when the first touch operation is performed on the original layer to obtain the target effective area image; performing the second touch operation on the original layer based on the update order.

[0106] by Fig.10 Taking the short video content aggregation display shown in as an example, the content browsing mode of the application is Fig.10 In the single-column browsing mode shown in , when the user clicks on image 2 (i.e., the target effective area image) in the system layer corresponding to the short video interface 2 on the left, the system can obtain the update order corresponding to image 2 based on the name of image 2, wherein, if image 2 is the image captured when the system simulates the second time that the user slides down in the original layer to update the interface to obtain the short video interface 2, then the update order corresponding to image 2 is the second time; then the system can simulate the user's operation of sliding the interface in the original layer, and slide the content in the original layer to the position of the short video interface 2; finally, the system can simulate the user's click operation to click and play the short video content corresponding to the short video interface 2.

[0107] It should be noted that, by simulating the user's sliding operation in the original layer, the user only needs to click on the corresponding image in the system layer according to his or her preference to browse the related content corresponding to the image. This avoids the problem of the user accidentally touching the refresh button when sliding to browse the content in the original layer, resulting in the loss of the content the user is browsing, thereby improving the user experience.

[0108] Optionally, when the associated content corresponding to the target effective area image is video content, in order to avoid the problem that when the system simulates the user's operation of sliding the interface in the original layer and slides the content in the original layer to the short video interface corresponding to the target effective area image, the short video interface automatically plays the audio of the video content triggered by the system, resulting in a poor user experience, the system in the present application can set the playback status of all video content triggered during the execution of the second touch operation to a mute state during the execution of the second touch operation on the original layer according to the update order.

[0109] In an optional embodiment, if the associated content corresponding to the target effective area image is video content, the short video displayed in a single-column sliding manner in the original layer is displayed in the form of a video. Fig.11 After the multi-column display status is aggregated and displayed in the application display interface, the system can Fig.11 The quick browsing carousel effect shown in the figure realizes the carousel function of the video content in the display interface, and plays the short video content corresponding to images 1 to 8 in sequence. For example, when playing the short video corresponding to image 2, the display effect in the system layer can be as follows: Fig.11 As shown in , the content display area corresponding to image 2 is enlarged to display the short video content to the user. Among them, the user can independently choose to stop or play the video content according to the needs.

[0110] It should be noted that by aggregating the short videos displayed in a single-column sliding manner in the original layer and displaying them in a multi-channel state in the display interface of the application, users can choose the content they want to watch according to their preferences without having to slide the content in the original layer one by one, thereby improving the user experience.

[0111] In an optional embodiment, the information aggregation display method provided in the embodiment of the present application can also aggregate and display the contents in the original layers of the display interfaces of different applications. For example, the system aggregates the effective area images corresponding to the multiple target images in the intercepted application A and the effective area images corresponding to the multiple target images in the intercepted application B to obtain the aggregated image, and updates the system layer based on the aggregated image, and displays it in the display interface of application B or application A, or displays it in the display interface of other applications except application B and application A, thereby further improving the versatility of the information aggregation display.

[0112] Through the information aggregation display method provided by the embodiment of the present application, the effective area image in the target image captured when the screen content is updated directly through the acquired original layer, and the target layer pre-created and set on the original layer, the screen content in the display interface of the application is aggregated and displayed, and the screen content in the display interface of the application can be aggregated and displayed without relying on the SDK provided by the manufacturer. Thereby improving the versatility of information aggregation display, and then solving the technical problem that the prior art relies on the SDK provided by the manufacturer to realize information aggregation display, and has poor versatility. Through the information aggregation display method provided by the embodiment of the present application, it is realized to display the complete content display area in the display interface, without the user frequently scrolling up and down or refreshing the information, avoiding long-term staring at the electronic device, causing the user to have visual fatigue, myopia and eye health problems, thereby improving the user experience. Through the information aggregation display method provided by the embodiment of the present application, it is realized that the system simulates the user's sliding operation in the original layer, avoiding the problem of refreshing the content in the original layer due to misoperation, resulting in the loss of the content that the user is browsing, thereby improving the user experience.

[0113] Device Embodiment

[0114] The embodiment of the present application provides an information aggregation display device, wherein: Fig.12 A schematic diagram of the structure of an information aggregation display device provided in an embodiment of the present application, such as Fig.12 As shown, the device includes: an acquisition module 11, an image extraction module 12, an image aggregation module 13, an image update module 14 and a display module 15. Fig.12 You can see the connection relationship between several modules.

[0115] The acquisition module 11 is used to acquire M target images to be displayed, wherein each target image is an image captured when the original layer updates the screen content once, each target image includes multiple content display areas, the original layer is the layer corresponding to the display interface of the application, and M is a positive integer;

[0116] An image extraction module 12 is used to extract a valid area image from each target image to obtain a total of N valid area images, wherein one valid area image corresponds to a complete content display area, and N is an integer greater than or equal to M;

[0117] An image aggregation module 13, used for aggregating N effective area images to obtain an aggregated image;

[0118] An image updating module 14, configured to update information in a target layer based on the aggregated image, wherein the target layer is a layer pre-created and set above the original layer;

[0119] The display module 15 is used to display the updated information in the target layer in the display interface.

[0120] Optionally, the image extraction module includes: a recognition unit, an image segmentation unit and an image screening unit.

[0121] The identification unit is used to identify the boundary coordinate information of multiple content display areas in each target image, and obtain multiple boundary coordinate information corresponding to each target image;

[0122] An image segmentation unit, configured to segment the target image based on a plurality of boundary coordinate information corresponding to each target image, to obtain a total of L regional images, wherein L is a positive integer and L is greater than N;

[0123] The image screening unit is used to screen the L area images for integrity of the content display area to obtain N valid area images.

[0124] Optionally, the acquisition module includes: a first execution unit, a second execution unit and a third execution unit.

[0125] The first execution unit is used to execute a first touch operation on the original layer in response to the content update instruction, wherein the content update instruction is an instruction generated when the user clicks the content update button in the target layer, and the first touch operation is used to simulate the sliding operation of the user in the original layer to update the screen content in the original layer;

[0126] The second execution unit is used to execute an image capture operation to capture the updated picture content in the original layer to obtain a target image;

[0127] The third execution unit is used to execute the first touch operation and the image capture operation multiple times until M target images are obtained.

[0128] Optionally, the information aggregation display device further includes: an identification acquisition module and a first execution module.

[0129] The first identification acquisition module is used to obtain a unique identifier corresponding to a target effective area image in response to an information interaction instruction after displaying the updated information in the target layer in the display interface, wherein the target effective area image is an updated effective area image in the target layer clicked by the user;

[0130] The first execution module is used to perform a second touch operation on the original layer according to the unique identifier, wherein the second touch operation is used to simulate the user's touch operation in the original layer, and display the associated content corresponding to the target effective area image in the display interface.

[0131] Optionally, the first execution module includes: a position acquisition unit and a fourth execution unit.

[0132] The position acquisition unit is used to acquire the region position information corresponding to the target effective region image based on the unique identifier when the content browsing mode of the application is a multi-column browsing mode, wherein the region position information is the position information of the screen content corresponding to the target effective region image in the original layer when the image capture operation is performed on the original layer;

[0133] The fourth execution unit is used to perform a second touch operation on the original layer based on the area position information.

[0134] Optionally, the first execution module further includes: an order acquisition unit and a fifth execution unit.

[0135] The order acquisition unit is used to acquire the update order corresponding to the target effective area image based on the unique identifier when the content browsing mode of the application is a single-column browsing mode, wherein the update order is the operation execution order when the first touch operation is performed on the original layer to obtain the target effective area image;

[0136] The fifth execution unit is used for executing the second touch operation on the original layer according to the update order.

[0137] Optionally, the information aggregation display device also includes: a setting module, which is used to set the playback status of all video contents triggered during the execution of the second touch operation to a mute state when the associated content corresponding to the target effective area image is video content and when the second touch operation is performed on the original layer according to the update order.

[0138] Storage Medium Embodiments

[0139] The embodiment of the present application provides a computer-readable storage medium, on which a program is stored, wherein when the program is executed by a processor, some or all of the steps in the information aggregation display method described in the aforementioned method embodiment of the present application are implemented. The storage medium can be a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0140] Processor Embodiments

[0141] An embodiment of the present application provides a processor for running a program, wherein part or all of the steps in the information aggregation display method introduced in the aforementioned method embodiment are executed when the program is running.

[0142] It should be noted that each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described above is merely schematic, in which the unit described as a separate component may or may not be physically separated, and the component prompted as a unit may or may not be a physical unit, that is, it may be located in one place, or it may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative work.

[0143] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An information aggregation display method, characterized in that: include: Obtain M target images to be displayed, wherein each of the target images is an image captured when the original layer updates the screen content once, each of the target images includes multiple content display areas, the original layer is the layer corresponding to the display interface of the application, and M is a positive integer; Extracting a valid area image from each of the target images to obtain a total of N valid area images, wherein one valid area image corresponds to a complete content display area, and N is an integer greater than or equal to M; Aggregating the N valid area images to obtain an aggregated image; Update information in a target layer based on the aggregated image, wherein the target layer is a layer created in advance and set above the original layer; The updated information in the target layer is displayed in the display interface.

2. The method according to claim 1, characterized in that: The extracting the effective area image in each of the target images to obtain a total of N effective area images includes: Identify the boundary coordinate information of the plurality of content display areas in each of the target images, and obtain the plurality of boundary coordinate information corresponding to each of the target images; Performing image segmentation on the target image based on the plurality of boundary coordinate information corresponding to each target image to obtain a total of L regional images, where L is a positive integer and L is greater than N; The L area images are screened for integrity of content display areas to obtain N valid area images.

3. The method according to claim 1, characterized in that The obtaining of M target images to be displayed comprises: In response to a content update instruction, performing a first touch operation on the original layer, wherein the content update instruction is an instruction generated when a user clicks a content update button in the target layer, and the first touch operation is used to simulate a sliding operation of the user in the original layer to update the screen content in the original layer; Performing an image capture operation to capture the updated screen content in the original layer to obtain the target image; The first touch operation and the image capture operation are performed multiple times until M target images are obtained.

4. The method according to claim 3, characterized in that: After displaying the updated information in the target layer in the display interface, the method includes: In response to the information interaction instruction, a unique identifier corresponding to a target effective area image is obtained, wherein the target effective area image is an updated effective area image in the target layer clicked by the user; A second touch operation is performed on the original layer according to the unique identifier, wherein the second touch operation is used to simulate the touch operation of the user in the original layer, and display the associated content corresponding to the target effective area image in the display interface.

5. The method according to claim 4, characterized in that When the content browsing mode of the application is a multi-column browsing mode, performing a second touch operation on the original layer according to the unique identifier includes: Acquire the area position information corresponding to the target effective area image based on the unique identifier, wherein the area position information is the position information of the screen content corresponding to the target effective area image in the original layer when the image interception operation is performed on the original layer; The second touch operation is performed on the original layer based on the area position information.

6. The method according to claim 4, characterized in that When the content browsing mode of the application is a single-column browsing mode, performing a second touch operation on the original layer according to the unique identifier includes: Acquire an update order corresponding to the target effective area image based on the unique identifier, wherein the update order is an operation execution order when the first touch operation is performed on the original layer to obtain the target effective area image; The second touch operation is performed on the original layer according to the update order.

7. The method according to claim 6, characterized in that The associated content corresponding to the target effective area image is video content, and in the process of performing the second touch operation on the original layer according to the update order, the method further includes: The playback status of all video contents triggered during the execution of the second touch operation is set to a mute state.

8. An information aggregation display device, characterized in that: include: An acquisition module, used to acquire M target images to be displayed, wherein each of the target images is an image captured when the original layer updates the screen content once, each of the target images includes multiple content display areas, the original layer is the layer corresponding to the display interface of the application, and M is a positive integer; An image extraction module, used to extract a valid area image from each of the target images to obtain a total of N valid area images, wherein one valid area image corresponds to a complete content display area, and N is an integer greater than or equal to M; An image aggregation module, used for aggregating the N valid area images to obtain an aggregated image; An image updating module, used for updating information in a target layer based on the aggregated image, wherein the target layer is a layer created in advance and set on the original layer; The display module is used to display the updated information in the target layer in the display interface.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the information aggregation display method according to any one of claims 1 to 7 is implemented.

10. A processor, characterized in that: Used to run a computer program, which, when running, executes the information aggregation display method according to any one of claims 1 to 7.

Images