Picture detection processing method, device and equipment

Abstract

The embodiment of the invention discloses a picture detection processing method, device and equipment, which are higher in reliability for a scene in which a picture is loaded and displayed by an application and favorable for improving user experience. According to the scheme, the method comprises the steps of determining a picture display size corresponding to a picture component in a target page to be displayed on target terminal equipment; determining an original size of a to-be-downloaded picture corresponding to the picture component; comparing whether the picture display size is inconsistent with the original size; and if yes, recording a corresponding relationship among the picture component, the to-be-downloaded picture and the target terminal equipment, so as to optimize a picture display effect of the picture component.

Description

Technical Field

[0001] The present specification relates to the field of front-end technology, and more particularly to a method, device and equipment for image detection and processing. Background Art

[0002] With the development of Internet technology and the popularization of smart phones, more and more businesses are conducted through various applications. In order to provide better visualization services, applications often display a large number of pictures on the page.

[0003] Users often have this experience in daily life: images on some application pages load very slowly, and may even fail to load and cannot be displayed. Or, even after loading successfully, the display effect is rather strange, which reduces the usability of the entire page and affects the user experience.

[0004] Based on this, for scenarios where applications load and display images, a more reliable solution that helps improve user experience is needed. Summary of the invention

[0005] One or more embodiments of the present specification provide a method, apparatus, device and storage medium for image detection and processing to solve the following technical problem: For scenarios where applications load and display images, a more reliable solution that helps improve user experience is needed.

[0006] To solve the above technical problems, one or more embodiments of this specification are implemented as follows:

[0007] One or more embodiments of this specification provide a method for image detection and processing, including:

[0008] Determine the image display size corresponding to the image component in the target page to be displayed on the target terminal device;

[0009] Determine the original size of the image to be downloaded corresponding to the image component;

[0010] Compare the image display size with the original size to see if they are inconsistent;

[0011] If so, the correspondence between the image component, the image to be downloaded and the target terminal device is recorded to optimize the image display effect of the image component.

[0012] One or more embodiments of this specification provide an image detection processing device, including:

[0013] An image display size determination module determines an image display size corresponding to an image component in a target page to be displayed on a target terminal device;

[0014] An image original size determination module is used to determine the original size of the image to be downloaded corresponding to the image component;

[0015] An image size consistency comparison module compares whether the image display size is inconsistent with the original size;

[0016] The corresponding relationship recording optimization module records the corresponding relationship between the image component, the image to be downloaded and the target terminal device, so as to optimize the image display effect of the image component.

[0017] One or more embodiments of this specification provide an image detection and processing device, including:

[0018] at least one processor; and,

[0019] a memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

[0021] Determine the image display size corresponding to the image component in the target page to be displayed on the target terminal device;

[0022] Determine the original size of the image to be downloaded corresponding to the image component;

[0023] Compare the image display size with the original size to see if they are inconsistent;

[0024] If so, the correspondence between the image component, the image to be downloaded and the target terminal device is recorded to optimize the image display effect of the image component.

[0025] One or more embodiments of this specification provide a non-volatile computer storage medium storing computer executable instructions, wherein the computer executable instructions are configured to:

[0026] Determine the image display size corresponding to the image component in the target page to be displayed on the target terminal device;

[0027] Determine the original size of the image to be downloaded corresponding to the image component;

[0028] Compare the image display size with the original size to see if they are inconsistent;

[0029] If so, the correspondence between the image component, the image to be downloaded and the target terminal device is recorded to optimize the image display effect of the image component.

[0030] At least one of the above-mentioned technical solutions adopted in one or more embodiments of the present specification can achieve the following beneficial effects: by actively detecting whether the image display size corresponding to the image component in the target terminal device is consistent with the original size of the image to be downloaded, and then recording the inconsistency, so as to improve the display effect of the image component on the target terminal devices of the same type or similar situations through means such as controlling the actual downloaded size of the image to be downloaded to force size alignment, forcibly reducing the image display size and the original size of the image to be downloaded, etc., and it also helps to reduce the amount of downloaded data, improve the reliability of loading and display, and thus improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0032] Figure 1 A schematic diagram of a flow chart of an image detection and processing method provided in one or more embodiments of this specification;

[0033] Figure 2 A flowchart of a solution for detecting the image display size corresponding to an image component in a target page to be displayed on a target terminal device provided by one or more embodiments of this specification;

[0034] Figure 3 In an application scenario provided by one or more embodiments of this specification, Figure 1 A schematic diagram of an implementation scheme of the method;

[0035] Figure 4 A schematic diagram of a flow chart of an image loading and display optimization solution based on page splitting provided in one or more embodiments of this specification;

[0036] Figure 5 A schematic diagram of a process flow of a page splitting solution provided for one or more embodiments of this specification;

[0037] Figure 6 A schematic diagram of the structure of an image detection and processing device provided in one or more embodiments of this specification;

[0038] Figure 7 A schematic diagram of the structure of an image detection and processing device provided in one or more embodiments of this specification. DETAILED DESCRIPTION

[0039] The embodiments of this specification provide a method, apparatus, device and storage medium for image detection and processing.

[0040] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0041] For the situation in the background technology, the loading and display problem is often caused by the fact that the size of the image downloaded and further used for loading and display is too large, and the image display size of the corresponding image component used to display the image on the terminal device is small. On the one hand, it leads to a large amount of data to be downloaded, resulting in slow downloading. On the other hand, even if it is downloaded to the terminal device, it will be squeezed and scaled or incompletely displayed when displayed in the image component. In the case of a large number of images to be displayed, it will bring a bad experience to the user, and may even make the user give up waiting and exit the current page. For such a problem, the applicant has tried to request the server to download the image with a fixed size. This method can reduce the size of the downloaded image to a certain extent, but it is still not accurate enough. Because the resolutions of different terminal devices may be different, and the sizes of different resolutions are different, if the size is set small, there will be a stretching effect when displaying. If the size is set large, the download traffic will be wasted. Not only that, the terminal device itself may also take the initiative to scale or magnify the image on its own initiative, making the situation more complicated.

[0042] In order to solve the above problems, the present application actively detects the consistency between the image display size and the size of the image to be downloaded for the current target terminal device, records the inconsistencies, and optimizes the image display effect for the target terminal device or other similar devices accordingly; in particular, it further proposes an optimization solution for size alignment and an optimization solution based on adaptive page splitting and display.

[0043] Based on the above general idea, the present solution will be described in detail below.

[0044] Figure 1 A flowchart of a method for image detection and processing provided for one or more embodiments of this specification. The execution subject of the process can be a terminal device and / or a server device. If executed on a terminal device, it can especially optimize the local image display effect in real time. If executed on a server device, it helps to reuse the detection results and optimize the image display effect for a wider range of terminal devices.

[0045] Figure 1 The process in includes the following steps:

[0046] S102: Determine the image display size corresponding to the image component in the target page to be displayed on the target terminal device.

[0047] In one or more embodiments of the present specification, if the image display size corresponding to the image component can be indicated in the HTML code of the target page, the image display size can be directly used. However, in actual applications, in many cases, the image display size cannot be directly obtained from the HTML code. In such cases, the image component on the target page can be located by using the short time gap when the target page is displayed but the image has not yet been loaded, so as to determine the image display size.

[0048] In addition, the reference image can be used to test the display of the image component on the target terminal device, and the image display size can be indirectly determined based on the actual display effect. In this case, the reference image can be the image to be downloaded itself or other images. For other images, if necessary, the code can be temporarily modified in the HTML code of the target page to reference the local image as the reference image, or the image download link can be modified to download other remote images as reference images.

[0049] S104: Determine the original size of the image to be downloaded corresponding to the image component.

[0050] The image to be downloaded is initially on the server side. When the target page is displayed on the target terminal device, the image to be downloaded needs to be downloaded from the server side, and then loaded and displayed through the image component in the target page.

[0051] The image to be downloaded usually has only one size on the server, namely the original size. Assuming that the original size is exactly the same as the above-mentioned image display size, then directly using the image to be downloaded to display on the above-mentioned image component will most likely achieve a better display effect, which is considered to be a more ideal situation by this application. This application focuses on the case of inconsistent sizes, especially the case where the original size is larger than or even significantly larger than the image display size.

[0052] In one or more embodiments of the present specification, the image display size and the original size may both refer to pixel size, so that the detection result can be better applicable to terminal devices with different resolutions.

[0053] S106: Compare the image display size with the original size to see if they are inconsistent.

[0054] If the image display size is exactly the same as the original size, the two are considered to be consistent. In practical applications, the standard can be appropriately relaxed. For example, if the difference between the image display size and the original size is small and within the tolerance range of this application, the two can also be considered to be consistent.

[0055] In the case of inconsistency, if the original size is larger than the image display size, it may cause the problems mentioned above, including time-consuming download and high traffic, unreliability, poor display effect, etc. If the original size is smaller than the image display size, it may also cause poor display effect problems such as stretching and deformation.

[0056] S108: If yes, then record the correspondence between the image component, the image to be downloaded and the target terminal device to optimize the image display effect of the image component.

[0057] In one or more embodiments of the present specification, efforts are made to discover the above-mentioned inconsistencies as quickly as possible, and then provide corresponding optimization solutions to improve the picture download experience and loading display effect. In order to optimize in a targeted manner, the above-mentioned correspondence is recorded. It should be noted in particular that in the correspondence, an association is made with the target terminal device. The target terminal device here can refer to a specific device (for example, the current device), or it can refer to some devices with commonalities, such as a class of mobile terminal devices of the same model, a class of mobile terminal devices of the same resolution, a class of mobile terminal devices of the same screen size, and so on. Similarly, the above process can be performed for different target terminal devices to more accurately find out which target terminal devices have corresponding problems for optimization.

[0058] In the corresponding relationship, the corresponding image link or other identification information can be used to represent the image to be downloaded, the corresponding model or screen resolution and other identification information can be used to represent the target terminal device, and the corresponding component name, image display size, or position in the page and other identification information can be used to represent the image component. More content can also be associated with the corresponding relationship according to actual needs to better guide optimization, such as loading time, actual display effect, etc.

[0059] In one or more embodiments of the present specification, in the event of inconsistency, if the original size is larger than the image display size, the correspondence between the image component, the image to be downloaded and the target terminal device is recorded to reduce the data volume of the image to be downloaded (for example, scaling and cropping, converting the image format to an advanced format, etc.), converting it into a size consistent with the image display size, and then providing it to the target terminal device for download so that it can be displayed through the image component.

[0060] For example, based on the recorded correspondence, when you want to use the current or the same type of target terminal device indicated by the correspondence in the future, you can obtain the image link address corresponding to the image component, and carry additional parameters for the image link address to request the server to download the image. The additional parameters indicate the local image display size. The receiving server returns the image to be downloaded that is consistent with the image display size and displays it through the image component, thereby obtaining a better display effect and improving the download experience. Among them, the image to be downloaded that is consistent with the image display size is generated by the server according to the image display size, by pre-processing or real-time reprocessing of the larger original size of the image to be downloaded.

[0061] pass Figure 1 The method actively detects whether the image display size corresponding to the image component of the target terminal device is consistent with the original size of the image to be downloaded, and then records the inconsistency, so as to improve the display effect of the image component on the target terminal devices of the same type or similar situations by means such as controlling the actual downloaded size of the image to be downloaded to force the size to be aligned, forcibly reducing the image display size and the original size of the image to be downloaded, etc., and also helps to reduce the amount of downloaded data, improve the reliability of loading and display, and thus improve the user experience.

[0062] based on Figure 1 This specification also provides some specific implementation plans and extension plans of the method, which will be described below.

[0063] In the case that the target page does not directly provide the image display size parameters corresponding to the image component, the present application can proactively determine the image display size by analyzing the HTML content and screenshots of the page. Based on this, one or more embodiments of the present specification provide a flow chart of a detection scheme for the image display size corresponding to the image component in the target page to be displayed on the target terminal device, see Figure 2 .

[0064] Figure 2 The process in includes the following steps:

[0065] S202: Taking a screenshot of the target terminal device displaying the target page to obtain a screenshot image.

[0066] The screenshots taken here will be affected by the resolution of the target terminal device itself and the zoom display of the target page. Therefore, from the perspective of pixel size, it can help to more accurately locate the position of the image component during actual operation. For the image component, assuming that there is a reference image for display (not necessarily the image to be downloaded itself), the screenshot can be taken after the reference image is loaded and displayed through the image component; if there is no reference image, the screenshot can be taken when the target page is initially loaded but the image has not yet been loaded.

[0067] Screen capture can be achieved through the API provided by the operating system, command line tools, or programming libraries.

[0068] Furthermore, detection can be performed in the area corresponding to the rough position in the screenshot image to determine the precise position of the image component. The subsequent steps exemplarily provide a detection method with a reference image.

[0069] The screenshots obtained by the screenshot can be preprocessed to improve the accuracy of the control positioning in the subsequent steps. The preprocessing may include one or more of the following: converting to a grayscale image to reduce the complexity of subsequent calculations; applying an image filter (such as Gaussian blur) to reduce noise; performing edge detection to highlight the boundaries of image components.

[0070] S204: Obtain HTML content of the target page, parse the HTML content according to the size of the screenshot, and determine a rough position of the image component in the target page according to the parsing result.

[0071] In one or more embodiments of the present specification, the HTML content is parsed according to the size of the screenshot to obtain a parsed target page (equivalent to being temporarily stored in memory after rendering for further processing), and then the rough position of the image component is determined in the parsed target page. It should be noted that the parsed and rendered image component may be covered by other layers due to its large size, so the position determined here may not be accurate enough, and it is called a rough position, which needs to be further combined with the screenshot image for WYSIWYG analysis.

[0072] In the process of parsing the HTML content of the target page, combined with the reference to the size of the screenshot, the image tag can be searched therein to locate the rough position of the image component. At the same time, the relative position of the image component can also be determined based on prominent paragraphs, text and other tags.

[0073] S206: Acquire the original image corresponding to the reference image, wherein when taking the screenshot, the reference image is displayed in the target page through the image component.

[0074] S208: Perform template matching or feature matching according to the area of ​​the original image corresponding to the rough position in the screenshot image.

[0075] Template matching is used to find the location of a small image in a large image. Specifically, a sliding window is used to find the area in the large image that best matches the template image, thereby determining the location of the small image.

[0076] Feature matching is based on detecting and comparing key points in images. For example, feature points are first detected in two images separately, and then matching pairs of feature points are found to determine the position.

[0077] In image processing libraries such as OpenCV, template matching (such as cv2.matchTemplate, etc.) or feature matching (such as SIFT, SURF, ORB and other algorithms) can be implemented by calling corresponding functions, thereby improving matching efficiency and reducing processing costs.

[0078] S210: Draw boundary marks around the successfully matched area or feature point set to obtain the precise position of the image component.

[0079] After finding the best matching area or feature point set, the more specific location of the image component in the screenshot can be determined. After positioning, a bounding box or other marker can be drawn at the location to visually represent the precise location of the image component.

[0080] S212: Determine the image display size corresponding to the image component according to the precise position.

[0081] Based on the marked bounding box or other marks, the image display size can be accurately calculated.

[0082] Furthermore, verification and adjustment can be performed. The positioning results are verified and the algorithm parameters are adjusted as needed to improve the accuracy and robustness of positioning. This can include adjusting edge detection parameters, improving template image quality, optimizing feature matching algorithms, etc.

[0083] According to the above description, more intuitively, one or more embodiments of this specification also provide an application scenario: Figure 1 A schematic diagram of an embodiment of the method of Figure 3 , the figure shows the brief steps of the solution. In this application scenario, the above detection logic is implemented as a corresponding SDK, called the detection SDK. The detection SDK can be deployed on the client and / or server of the application. For ease of understanding, it is regarded as a separate entity in the figure. In this process, the focus is mainly on detecting and recording abnormal situations. Therefore, it can be separated from the actual business online operation environment and executed in a pre-prepared test scenario.

[0084] Figure 3 The process in includes the following steps:

[0085] The client obtains the page data of the target page (such as a web page file, etc.) from the server so as to display the target page on the client.

[0086] The detection SDK obtains the screen resolution of the target terminal device where the client is currently located from the client, and obtains the image component elements according to the page data under the screen resolution condition, and then analyzes and determines the corresponding image display size.

[0087] The client's server requests to download the image to be loaded and displayed through the image component.

[0088] The detection SDK obtains the link address of the image from the client for later association, and also obtains the original size of the image. Compare the original size with the image display size to see if they are consistent. If the original size is larger than the image display size, it is considered that there may be a display effect problem. Therefore, the corresponding relationship between the "target page + image component + image link + device model" involved can be recorded accordingly, and reported to the server accordingly. By analogy, each target page is detected and processed, and the server can generate a list of image components corresponding to the problem page, and then optimize the processing for the corresponding terminal device.

[0089] In addition to the optimization scheme of scaling and cropping the to-be-downloaded images that are too large in original size to correspond to the image display size of the image component on the terminal device, one or more embodiments of this specification also provide an optimization scheme for image loading and display based on page splitting. Figure 4 The figure is a flow chart of another optimization scheme.

[0090] Figure 4 The solution in includes the following steps:

[0091] S402: After comparing whether the image display size is inconsistent with the original size, if so, obtaining and parsing the HTML content of the target page, and determining whether there are multiple concentrated image components in the target page according to the parsing result.

[0092] In one or more embodiments of the present specification, considering that in actual applications, there are often multiple images that need to be loaded and displayed in a page, and these multiple images may have similar problems of poor loading and display experience due to size misalignment, it is considered to actively separate these contents from the page dimension, so that current resources can be concentrated on loading and displaying some of the images, while the other images can be more naturally delayed in loading and displaying.

[0093] Based on this idea, by judging whether there are multiple concentrated image components in the target page, the current loading and display pressure can be inferred. Furthermore, it can be judged whether there are multiple concentrated image components in the target page that have the above-mentioned inconsistency problem (mainly referring to the situation where the original size is larger than the image display size). If so, content separation can be considered. Moreover, the conclusion of content separation can be directly reused in similar situations in the future, thereby improving efficiency. For example, according to the performance of the current terminal device and the current network speed, if the situation is good, content separation can be omitted and the image can be loaded and displayed normally. Otherwise, content separation can be performed and the image can be loaded and displayed in parts.

[0094] This application provides an exemplary solution for separating the content of a split web page. In addition, in the same page, a part of the content can be embedded in the control instead of being displayed directly. The former solution is more natural for users and it is not easy to miss the content, so the following steps are mainly explained by taking the former solution as an example.

[0095] S404: If so, determine at least one image component whose image display size is smaller than the original size as the target image component.

[0096] The target image component is a component that may have an adverse impact on the user experience, and can be used as a basis for how to split the page.

[0097] S406: splitting the HTML content according to the position of the image component in the HTML content, and generating a first page and a second page according to the two parts of HTML sub-content obtained by the splitting, so that the image component is located in the second page.

[0098] The target image component can be used as the dividing point of the page split. For example, assuming that there are enough image components before the target image component, the target image component can be considered to be divided into the latter part of the page split (corresponding to the second page). On the contrary, since the system may have enough resources to process the target image component, the target image component can be divided into the former part of the page split (corresponding to the first page). In step S406, the former method is exemplarily adopted to delay the processing of the target image component.

[0099] The split HTML sub-content can be adaptively added with some auxiliary content (for example, element tags missing due to splitting, transitional content added to modify page boundaries, newly applied page styles, etc.) to form a new page.

[0100] S408: Display the first page first.

[0101] When the first page is displayed, the second page will not be displayed temporarily. They are different pages and need to be switched. This application considers using a method based on user-initiated triggering to switch to the second page. In this way, the user can focus on the first page, and the second page can be preloaded without the user noticing, which helps the subsequent experience to better display the second page.

[0102] S410: If an operation performed by the user according to a set method on the content in the first page is received, the second page is displayed in response to the operation.

[0103] It should be noted that, based on the page splitting, the target image component can also be processed by forced size alignment, which can more reliably improve the user experience.

[0104] In one or more embodiments of the present specification, after all the image components in the first page have displayed images, if a user clicks on the displayed image, the page jumps to the second page, or the displayed image is enlarged and displayed first, and then the page jumps to the second page. Of course, these two methods are exemplary, and there are more optional methods that can be used as needed. For example, a tab can be generated for the second page, and a more conspicuous prompt can be given for the tab. If the user triggers the tab, the page jumps from the first page to the second page.

[0105] Furthermore, when displaying the second page, since the images in the first page have been downloaded and loaded before, the corresponding image components and image content in the first page can be automatically injected into the second page, thereby displaying more images in the original page in the second page.

[0106] In order to achieve page splitting more accurately and reliably, one or more embodiments of this specification also provide a flow chart of a page splitting solution, see Figure 5 .

[0107] Figure 5 The process in includes the following steps:

[0108] S502: Determine, among the plurality of concentratedly distributed image components, each image component whose image display size is smaller than the original size of the image component.

[0109] S504: Estimate the image download and loading time of each image component.

[0110] Here, you can estimate the image download loading time based on the current network speed or average network speed, the data volume of the image to be downloaded corresponding to the image component, and the processing performance of the current terminal device. The results can also be reused during the test process or the actual online process.

[0111] S506: Determine a picture component at a page splitting point as a target picture component according to the picture downloading and loading time and the order of the picture components in the target page.

[0112] In one or more embodiments of the present specification, for example, the download and loading times of each image are connected to form a complete and sequential time range, and then the time range is divided into two parts according to a set time division ratio, wherein the time length of the first part is significantly shorter than the time length of the second part, and then the dividing point moment of the two parts of time is found, and the image component closest to the dividing point moment can be used as the target image component.

[0113] S508: Use the target image component as the first image component included in the second page, split the target page to obtain the second page, and generate the first page according to the remaining content, so that the image loading and display time required for the first page is significantly less than the image loading and display time required for the second page.

[0114] pass Figure 5 The solution can adaptively split the first page with better loading and display experience, and can also reasonably control the time division ratio so that the second page can also complete the pre-download of pictures as much as possible while the user is watching the first page, thereby helping to obtain a better overall experience effect for the loading and display of the target page.

[0115] Based on the same idea, one or more embodiments of this specification also provide devices and apparatuses corresponding to the above methods, such as Figure 6 , Figure 7 The apparatus and device can accordingly execute the above method and related optional solutions.

[0116] Figure 6 A schematic diagram of a structure of an image detection and processing device provided in one or more embodiments of this specification, the device comprising:

[0117] The picture display size determination module 602 determines the picture display size corresponding to the picture component in the target page to be displayed on the target terminal device;

[0118] The picture original size determination module 604 determines the original size of the picture to be downloaded corresponding to the picture component;

[0119] The picture size consistency comparison module 606 compares whether the picture display size is inconsistent with the original size;

[0120] The corresponding relationship recording and optimizing module 608 records the corresponding relationship between the image component, the image to be downloaded and the target terminal device, so as to optimize the image display effect of the image component.

[0121] Optionally, the correspondence record optimization module 608 records the correspondence between the image component, the image to be downloaded and the target terminal device if the original size is larger than the image display size, so as to reduce the data volume of the image to be downloaded, convert it into a size consistent with the image display size, and then provide it to the target terminal device for download so as to be displayed through the image component.

[0122] Optionally, the image display size determination module 602 takes a screenshot of the target terminal device displaying the target page to obtain a screenshot image;

[0123] Obtaining HTML content of the target page, parsing the HTML content according to the size of the screenshot, and determining a rough position of the image component in the target page according to the parsing result;

[0124] A detection is performed in the area corresponding to the rough position in the screenshot image to determine the precise position of the image component, and a picture display size corresponding to the image component is determined according to the precise position.

[0125] Optionally, when taking the screenshot, a reference picture is displayed in the target page through the picture component;

[0126] The picture display size determination module 602 obtains the original picture corresponding to the reference picture;

[0127] Performing template matching or feature matching according to the area of ​​the original image corresponding to the rough position in the screenshot image;

[0128] When a boundary marker is drawn around a successfully matched area or feature point set, the precise position of the image component is obtained.

[0129] Optionally, the correspondence record optimization module 608 obtains the image link address corresponding to the image component after determining the image display size corresponding to the image component in the target page to be displayed on the target terminal device;

[0130] Requesting the server to download the image by carrying additional parameters for the image link address, wherein the additional parameters indicate the display size of the image;

[0131] Receive the image to be downloaded that is consistent with the image display size returned by the server, and display it through the image component, wherein the image to be downloaded that is consistent with the image display size is generated by the server reprocessing the image to be downloaded of the original size according to the image display size.

[0132] Optionally, the corresponding relationship record optimization module 608, after comparing whether the image display size is inconsistent with the original size, if yes, splits the target page into a first page and a second page according to the position of the image component;

[0133] First display the first page;

[0134] If an operation performed by the user on the content of the first page in a set manner is received, the second page is displayed in response to the operation.

[0135] Optionally, the correspondence record optimization module 608 obtains and parses the HTML content of the target page, and determines whether there are multiple centrally distributed image components in the target page according to the parsing result;

[0136] If yes, determining at least one image component whose image display size is smaller than the original size as the target image component;

[0137] According to the position of the image component in the HTML content, the HTML content is split, and according to the two parts of HTML sub-content obtained by the split, a first page and a second page are generated accordingly, so that the image component is located in the second page.

[0138] Optionally, the correspondence record optimization module 608 determines, among the plurality of concentratedly distributed picture components, each picture component whose picture display size is smaller than the picture component of the original size;

[0139] Estimate the image download and loading time of each image component;

[0140] According to the image download loading time and the order of the image components in the target page, a picture component at a page splitting point is determined as the target picture component.

[0141] Optionally, after all image components in the first page have displayed images, the corresponding relationship record optimization module 608 jumps to the second page if a click operation from the user on the displayed image is received, or first enlarges and displays the displayed image and then jumps to the second page.

[0142] Optionally, the target terminal device refers to a type of mobile terminal devices of the same model.

[0143] Figure 7 A schematic diagram of the structure of an image detection and processing device provided for one or more embodiments of this specification, the device comprising:

[0144] at least one processor; and,

[0145] a memory communicatively connected to the at least one processor; wherein,

[0146] The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

[0147] Determine the image display size corresponding to the image component in the target page to be displayed on the target terminal device;

[0148] Determine the original size of the image to be downloaded corresponding to the image component;

[0149] Compare the image display size with the original size to see if they are inconsistent;

[0150] If so, the correspondence between the image component, the image to be downloaded and the target terminal device is recorded to optimize the image display effect of the image component.

[0151] Based on the same idea, one or more embodiments of this specification further provide a non-volatile computer storage medium storing computer executable instructions, wherein the computer executable instructions are configured as follows:

[0152] Determine the image display size corresponding to the image component in the target page to be displayed on the target terminal device;

[0153] Determine the original size of the image to be downloaded corresponding to the image component;

[0154] Compare the image display size with the original size to see if they are inconsistent;

[0155] If so, the correspondence between the image component, the image to be downloaded and the target terminal device is recorded to optimize the image display effect of the image component.

[0156] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0157] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0158] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0159] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0160] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification may be in the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the embodiments of this specification may be in the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0161] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0162] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0163] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0164] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0165] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0166] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0167] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0168] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0169] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment, and non-volatile computer storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0170] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0171] The above description is only one or more embodiments of this specification and is not intended to limit this specification. For those skilled in the art, one or more embodiments of this specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of this specification shall be included in the scope of the claims of this specification.

Claims

1. A method for image detection and processing, comprising: Determine the image display size corresponding to the image component in the target page to be displayed on the target terminal device; Determine the original size of the image to be downloaded corresponding to the image component; Compare the image display size with the original size to see if they are inconsistent; If so, the correspondence between the image component, the image to be downloaded and the target terminal device is recorded to optimize the image display effect of the image component.

2. The method according to claim 1, wherein if yes, recording the correspondence between the image component, the image to be downloaded and the target terminal device to optimize the image display effect of the image component, specifically comprising: If the original size is larger than the image display size, the correspondence between the image component, the image to be downloaded and the target terminal device is recorded to reduce the data volume of the image to be downloaded, convert it into a size consistent with the image display size, and then provide it to the target terminal device for download so that it can be displayed through the image component.

3. The method according to claim 1, wherein determining the image display size corresponding to the image component in the target page to be displayed on the target terminal device specifically comprises: Taking a screenshot of the target terminal device displaying the target page to obtain a screenshot image; Obtaining HTML content of the target page, parsing the HTML content according to the size of the screenshot, and determining a rough position of the image component in the target page according to the parsing result; A detection is performed in the area corresponding to the rough position in the screenshot image to determine the precise position of the image component, and a picture display size corresponding to the image component is determined according to the precise position.

4. The method according to claim 3, wherein when taking the screenshot, the reference picture is displayed in the target page through the picture component; The detecting in the area corresponding to the rough position in the screenshot image to determine the precise position of the image component specifically includes: Obtaining an original picture corresponding to the reference picture; Performing template matching or feature matching according to the area of ​​the original image corresponding to the rough position in the screenshot image; When a boundary marker is drawn around a successfully matched area or feature point set, the precise position of the image component is obtained.

5. The method according to claim 1, after determining the image display size corresponding to the image component in the target page to be displayed on the target terminal device, the method further comprises: Get the image link address corresponding to the image component; Requesting the server to download the image by carrying additional parameters for the image link address, wherein the additional parameters indicate the display size of the image; Receive the image to be downloaded that is consistent with the image display size returned by the server, and display it through the image component, wherein the image to be downloaded that is consistent with the image display size is generated by the server reprocessing the image to be downloaded of the original size according to the image display size.

6. The method according to claim 1, after comparing whether the image display size is inconsistent with the original size, the method further comprises: If yes, splitting the target page into a first page and a second page according to the position of the image component; First display the first page; If an operation performed by the user on the content of the first page in a set manner is received, the second page is displayed in response to the operation.

7. The method according to claim 6, wherein the step of splitting the target page into a first page and a second page according to the position of the image component comprises: Obtaining and parsing the HTML content of the target page, and determining whether there are multiple centrally distributed image components in the target page according to the parsing result; If yes, determining at least one image component whose image display size is smaller than the original size as the target image component; According to the position of the image component in the HTML content, the HTML content is split, and according to the two parts of HTML sub-content obtained by the split, a first page and a second page are generated accordingly, so that the image component is located in the second page.

8. The method according to claim 7, wherein determining at least one image component whose image display size is smaller than the original size as the target image component specifically comprises: Determine, among the plurality of concentratedly distributed image components, each image component whose image display size is smaller than the image component of the original size; Estimate the image download and loading time of each image component; According to the image download loading time and the order of the image components in the target page, a picture component at a page splitting point is determined as the target picture component.

9. The method according to claim 6, wherein if an operation performed by a user according to a set method on the content of the first page is received, then in response to the operation, the second page is displayed, specifically comprising: After all the image components in the first page have displayed images, if a click operation from the user on the displayed image is received, the page is jumped to the second page, or the displayed image is first enlarged and displayed, and then the page is jumped to the second page.

10. The method according to any one of claims 1 to 9, wherein the target terminal device refers to a type of mobile terminal devices of the same model.

11. A picture detection and processing device, comprising: An image display size determination module determines an image display size corresponding to an image component in a target page to be displayed on a target terminal device; An image original size determination module is used to determine the original size of the image to be downloaded corresponding to the image component; An image size consistency comparison module compares whether the image display size is inconsistent with the original size; The corresponding relationship recording optimization module records the corresponding relationship between the image component, the image to be downloaded and the target terminal device, so as to optimize the image display effect of the image component.

12. The device as claimed in claim 11, the corresponding relationship recording optimization module, if the original size is larger than the image display size, records the corresponding relationship between the image component, the image to be downloaded and the target terminal device, and is used to reduce the data volume of the image to be downloaded, convert it into a size consistent with the image display size, and then provide it to the target terminal device for download so as to be displayed through the image component.

13. The device according to claim 11, wherein the image display size determination module takes a screenshot of the target terminal device displaying the target page to obtain a screenshot image; Obtaining HTML content of the target page, parsing the HTML content according to the size of the screenshot, and determining a rough position of the image component in the target page according to the parsing result; A detection is performed in the area corresponding to the rough position in the screenshot image to determine the precise position of the image component, and a picture display size corresponding to the image component is determined according to the precise position.

14. The device according to claim 13, wherein when taking the screenshot, the target page displays a reference image through the image component; The picture display size determination module obtains the original picture corresponding to the reference picture; Performing template matching or feature matching according to the area of ​​the original image corresponding to the rough position in the screenshot image; When a boundary marker is drawn around a successfully matched area or feature point set, the precise position of the image component is obtained.

15. The apparatus according to claim 11, wherein the corresponding relationship record optimization module, after determining the image display size corresponding to the image component in the target page to be displayed on the target terminal device, obtains the image link address corresponding to the image component; Requesting the server to download the image by carrying additional parameters for the image link address, wherein the additional parameters indicate the display size of the image; Receive the image to be downloaded that is consistent with the image display size returned by the server, and display it through the image component, wherein the image to be downloaded that is consistent with the image display size is generated by the server reprocessing the image to be downloaded of the original size according to the image display size.

16. The device of claim 11, wherein the corresponding relationship record optimization module, after comparing whether the image display size is inconsistent with the original size, if yes, splits the target page into a first page and a second page according to the position of the image component; First display the first page; If an operation performed by the user on the content of the first page in a set manner is received, the second page is displayed in response to the operation.

17. The device according to claim 16, wherein the correspondence record optimization module obtains and parses the HTML content of the target page, and determines whether there are multiple centrally distributed image components in the target page according to the parsing result; If yes, determining at least one image component whose image display size is smaller than the original size as the target image component; According to the position of the image component in the HTML content, the HTML content is split, and according to the two parts of HTML sub-content obtained by the split, a first page and a second page are generated accordingly, so that the image component is located in the second page.

18. The apparatus according to claim 17, wherein the corresponding relationship record optimization module determines, among the plurality of concentratedly distributed image components, each image component whose image display size is smaller than the original size; Estimate the image download and loading time of each image component; According to the image download loading time and the order of the image components in the target page, a picture component at a page splitting point is determined as the target picture component.

19. The device as described in claim 16, wherein the correspondence record optimization module, after all image components in the first page have displayed images, jumps to the second page if a click operation from the user on the displayed image is received, or first enlarges and displays the displayed image and then jumps to the second page.

20. The apparatus according to any one of claims 11 to 19, wherein the target terminal device refers to a type of mobile terminal devices of the same model.

21. A picture detection and processing device, comprising: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to execute: Determine the image display size corresponding to the image component in the target page to be displayed on the target terminal device; Determine the original size of the image to be downloaded corresponding to the image component; Compare the image display size with the original size to see if they are inconsistent; If so, the correspondence between the image component, the image to be downloaded and the target terminal device is recorded to optimize the image display effect of the image component.

Images