Picture positioning method and device, electronic equipment and computer readable storage medium

Abstract

The application discloses a picture positioning method and device, electronic equipment and computer readable storage medium, and belongs to the technical field of picture processing. The method comprises the following steps: acquiring a target page; extracting feature information of pictures in the target page, clustering based on the feature information, and obtaining a plurality of clustered picture groups according to a clustering result; the picture size of any clustered picture group in the plurality of clustered picture groups is greater than a first target value, and / or the picture capacity is greater than a second target value; generating a screenshot of the target page, detecting a picture control in the screenshot, and obtaining a detection result; matching the pictures of the plurality of clustered picture groups with the picture control in the detection result, and acquiring the position of the picture control that succeeds in the matching in the target page. The scheme realizes automatic positioning of pictures with super size or super capacity, and improves the positioning efficiency of the pictures.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of picture processing, and in particular to a picture positioning method and device, electronic equipment and a computer readable storage medium. BACKGROUND

[0002] In the process of network information acquisition, the display of network pictures is an important factor affecting user experience. The display of network pictures is mainly affected by network picture capacity and network picture size. Among them, the continuous improvement of network picture capacity increases the traffic consumed by displaying network pictures while providing users with a better browsing experience. In addition, larger capacity increases the time for downloading and decoding network pictures, increasing the user's waiting time cost. In addition, when the size of the network picture is larger than the size of the terminal browsing window, the user can only browse part of the content of the network picture, and the user's browsing experience will also be reduced. Therefore, it is necessary to position the picture, and then provide technical support for picture troubleshooting.

[0003] In related technologies, picture positioning is performed manually. However, manual positioning of pictures has high labor costs, is not sustainable, and has low positioning efficiency. SUMMARY

[0004] The embodiments of the present application provide a picture positioning method, device, electronic equipment and computer readable storage medium, which can be used to solve the problems in the related art. The technical solution is as follows:

[0005] In one aspect, the embodiments of the present application provide a picture positioning method, which comprises:

[0006] Obtaining a target page;

[0007] Extracting feature information of pictures in the target page, clustering based on the feature information, and obtaining a plurality of clustered picture groups according to the clustering result. The picture size of any clustered picture group in the plurality of clustered picture groups is greater than a first target value, and / or the picture capacity is greater than a second target value;

[0008] Generating a screenshot of the target page, detecting picture controls in the screenshot, and obtaining a detection result;

[0009] Matching the pictures of the plurality of clustered picture groups with the picture controls in the detection result, and obtaining the position of the picture controls in the target page that match successfully.

[0010] In one possible implementation, matching the pictures of the plurality of clustered picture groups with the picture controls in the detection result comprises:

[0011] determining a difference between the size of the picture control in the detection result and a picture size of a picture of any of the plurality of clustered picture groups;

[0012] in response to the difference between the size of any picture control in the detection result and a picture size of any picture of any clustered picture group being not greater than a first value, determining a difference between a capacity of the any picture control and a picture capacity of the any picture;

[0013] in response to the difference between the capacity of the any picture control and the picture capacity of the any picture being not greater than a second value, regarding the any picture control as a successfully matched picture control.

[0014] In a possible implementation, matching the pictures of the plurality of clustered picture groups with the picture control in the detection result comprises:

[0015] obtaining a plurality of target pictures, any target picture in the plurality of target pictures being used to indicate one clustered picture group in the plurality of clustered picture groups;

[0016] in response to the difference between the size of any picture control in the detection result and a picture size of any target picture in the plurality of target pictures being not greater than a third value, and the difference between the capacity of the any picture control and a picture capacity of the any target picture being not greater than a fourth value, matching the any picture control with the pictures of the clustered picture group indicated by the any target picture;

[0017] in response to the difference between the size of the any picture control and a picture size of any picture in the clustered picture group indicated by the any target picture being not greater than the first value, and the difference between the capacity of the any picture control and the picture capacity of the any picture being not greater than the second value, regarding the any picture control as a successfully matched picture control.

[0018] In a possible implementation, obtaining the plurality of target pictures comprises:

[0019] for any clustered picture group in the plurality of clustered picture groups, obtaining feature information of a plurality of pictures of the any clustered picture group, the feature information of the plurality of pictures comprising feature information of each picture, and the feature information of each picture comprising a plurality of categories and feature values corresponding to the plurality of categories;

[0020] for any category in the plurality of categories, obtaining a target feature value of the any category according to the feature values of the plurality of pictures belonging to the any category;

[0021] obtaining a target picture corresponding to the any clustered picture group according to the target feature values of the plurality of categories.

[0022] In a possible implementation, after obtaining the position of the successfully matched picture control in the target page, the method further comprises:

[0023] based on the acquired position of the matched picture control in the target page, adjusting the picture corresponding to the picture control.

[0024] In another aspect, an apparatus for picture positioning is provided, the apparatus comprising:

[0025] an acquisition module configured to acquire a target page and extract feature information of pictures in the target page;

[0026] a clustering module configured to cluster based on the feature information, and obtain a plurality of clustered picture groups according to a clustering result, wherein a picture size of a picture in any of the plurality of clustered picture groups is greater than a first target value, and / or a picture capacity is greater than a second target value;

[0027] a detection module configured to generate a screenshot of the target page, detect a picture control in the screenshot, and obtain a detection result;

[0028] a matching module configured to match the pictures in the plurality of clustered picture groups in the clustering result with the picture controls in the detection result;

[0029] The acquisition module is further configured to acquire a position of the matched picture control in the target page.

[0030] In a possible implementation, the matching module is configured to determine a difference between a size of the picture control in the detection result and a size of the picture in any of the plurality of clustered picture groups; and in response to the difference between the size of any picture control in the detection result and the picture size of any picture in any of the plurality of clustered picture groups being not greater than a first value, determine a difference between a capacity of the any picture control and a picture capacity of the any picture.

[0031] In a possible implementation, the apparatus further comprises an adjustment module configured to, based on the acquired position of the matched picture control in the target page, adjust the picture corresponding to the picture control.

[0032] In another aspect, an electronic device is provided, the electronic device comprising a processor and a memory, the memory having stored therein at least one program code or instruction, the at least one program code or instruction being loaded and executed by the processor to cause the electronic device to implement any of the above-described methods for picture positioning.

[0033] In another aspect, a computer-readable storage medium is also provided, the computer-readable storage medium having stored therein at least one program code or instruction, the at least one program code or instruction being loaded and executed by a processor to cause a computer to implement any of the above-described methods for picture positioning.

[0034] In another aspect, a computer program or computer program product is provided, in which at least one computer instruction is stored, and the at least one computer instruction is loaded and executed by a processor to enable the computer program or computer program product to implement any of the above-described picture positioning methods.

[0035] The technical solutions provided by the embodiments of the present application bring at least the following beneficial effects:

[0036] By clustering the pictures on the target page according to the feature information of the pictures, the pictures with a size greater than a first target value and / or with a capacity greater than a second target value are clustered into a cluster picture group, thereby reducing the search range of the pictures with an oversized size or an oversized capacity. Then, the picture controls in the screenshot of the target page are detected. Since the positions of the picture controls on the target page can be determined, the pictures in the cluster picture groups are matched with the picture controls in the detection result, and the positions of the picture controls in the page that are matched successfully are obtained, thereby realizing automatic positioning of the pictures with an oversized size or an oversized capacity on the target page and improving the positioning efficiency of the pictures. BRIEF DESCRIPTION OF DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort based on these drawings.

[0038] Figure 1 is a schematic diagram of an implementation environment of a picture positioning provided by an embodiment of the present application;

[0039] Figure 2 is a flowchart of a picture positioning method provided by an embodiment of the present application;

[0040] Figure 3 is a schematic diagram of a screenshot of a target page provided by an embodiment of the present application;

[0041] Figure 4 is a schematic diagram of a picture positioning device provided by an embodiment of the present application;

[0042] Figure 5 is a structural schematic diagram of an electronic device for picture positioning provided by an embodiment of the present application. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solutions and advantages of the present application more clear, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0044] Figure 1This is a schematic diagram of an implementation environment for image positioning provided in an embodiment of this application, such as... Figure 1 As shown, the implementation environment includes: server 101 and terminal 102.

[0045] Server 101 is used to send pages to terminal 102. Terminal 102 is used to automatically locate images in the pages sent by server 101, and also to adjust the located images. In one possible implementation, terminal 102 is also used to store the position of the images on the page.

[0046] Server 101 and terminal 102 establish a communication connection via a wired or wireless network. Server 101 can be a single server, a server cluster consisting of multiple servers, or a cloud computing service center. Terminal 102 can be an electronic device such as a mobile phone, tablet computer, or personal computer. Terminal 102 is used to execute the image positioning method provided in the embodiments of this application.

[0047] Terminal 102 can refer to one of multiple terminals, and this embodiment only uses terminal 102 as an example. Those skilled in the art will know that the number of terminals 102 can be more or less. For example, there may be only one terminal 102, or there may be dozens or hundreds of terminals 102, or even more. This application embodiment does not limit the number or type of terminals 102.

[0048] Based on the above implementation environment, this application provides a method for image positioning, in order to... Figure 2 The flowchart shown in this embodiment of the present application illustrates a method for image localization. This method can be implemented by... Figure 1 Executed in terminal 102. For example... Figure 2 As shown, the method includes the following steps 201 to 204.

[0049] Step 201: Obtain the target page.

[0050] In one possible implementation, the target page is an activity containing an image or a scheme that can launch an activity containing an image.

[0051] In one possible implementation, the target page is obtained by the terminal sending a request to the server to obtain the target page, the server responding to the received request to obtain the target page sending the target page to the terminal, and the terminal receiving the target page sent by the server.

[0052] In a possible implementation, the manner of triggering the terminal to send a request for obtaining the target page to the server includes: in response to a user clicking a client icon on the terminal, the terminal sends a request for obtaining a client page to the server, or in response to a user clicking a link in an interface of the client on the terminal, the terminal sends a request for obtaining a page corresponding to the link to the server.

[0053] In step 202, feature information of the pictures in the target page is extracted, clustering is performed based on the feature information, and a plurality of clustered picture groups is obtained according to a clustering result. The picture size of the pictures in any one of the plurality of clustered picture groups is greater than a first target value, and / or the picture capacity is greater than a second target value.

[0054] In this step, the feature information of the pictures in the target page is extracted, and the feature information of the pictures includes but is not limited to an ID (identity document), a picture size, a picture capacity, and a URL (uniform resource locator).

[0055] In a possible implementation, when the obtained target page is a scheme capable of starting an activity containing pictures, the feature information of the pictures in the target page includes steps 2021 and 2022.

[0056] In step 2021, based on the obtained scheme, the activity containing pictures is started.

[0057] In this step, the manner of starting the activity containing pictures is not limited in the embodiments of the application. Alternatively, the activity containing pictures is started by using a computer function, for example, the activity containing pictures is started by using a startActivity() function in the scheme.

[0058] In step 2022, the feature information of the pictures in the activity is extracted.

[0059] In this step, the feature information of the pictures in the activity is extracted by using a computer function. For example, the ID of the picture is obtained by using a getID() function, the width of the picture is extracted by using a getWidth() function, the height of the picture is extracted by using a getHeight() function, the picture capacity of the picture is extracted by using a getByteCount() function, and the URL of the picture is extracted by using a getResource.getDrawable() function. The feature information of all the pictures in the activity is obtained by traversing the pictures in the activity.

[0060] In a possible implementation, when the obtained target page is an activity, the process of extracting the feature information of the pictures in the target page is the same as the implementation process of step 2022, and details are not described herein again.

[0061] In a possible implementation, the clustering algorithm used in the clustering can be any one of a K-Means clustering, a mean shift clustering, a DBSCAN (density-based spatial clustering of applications with noise), a maximum expectation clustering with a Gaussian mixture model, and a condensed hierarchical clustering.

[0062] In a possible implementation, the K-Means clustering algorithm is used for clustering, and the clustering process includes steps 1 to 5.

[0063] Step 1, set a target number of clusters k, k is an integer greater than or equal to 1.

[0064] Step 2, randomly select k pictures from the pictures of the activity as initial cluster centers.

[0065] Step 3, calculate the distance of any picture of the activity to each initial cluster center, and assign the any picture to the cluster picture group in which the initial cluster center with the shortest distance is located according to the calculated distance, to obtain k picture classes.

[0066] In this step, the distance of any picture of the activity to each initial cluster center is calculated according to the feature information of the picture. Alternatively, the picture size and the picture capacity of the picture are used for calculation. The method of calculating the distance includes, but is not limited to, any one of Euclidean distance-straight line distance calculation, Manhattan distance calculation, and cosine distance calculation.

[0067] Step 4, update the cluster center of each picture class.

[0068] In this step, for the obtained k picture classes, the method of updating the cluster center of the picture class can be to calculate the average value of the feature information of the pictures in the picture class, and obtain the new cluster center of the picture class according to the obtained average value.

[0069] Step 5, recalculate the distance of any picture of the activity to the updated cluster center of each picture class, assign the any picture to the cluster center with the shortest distance, and update the cluster center of each picture class; repeat steps 3 and 4 until the updated cluster center is unchanged, and obtain the clustering result, that is, k picture classes.

[0070] Exemplarily, the target page includes five pictures, i.e., picture 1, picture 2, picture 3, picture 4 and picture 5, and the feature information of each picture extracted is shown in Table 1.

[0071] Table 1

[0072] Picture Width Height Picture capacity 1 5 3 2MB 2 14 4 2MB 3 14 4 2MB 4 5 3 4MB 5 14 4 4MB

[0073] In Table 1, the width and height of each picture are the feature information of the picture size type; the width of picture 1 is 5, the height is 3, and the picture capacity is 2MB, and the feature information of other pictures is shown in Table 1, which is not described herein.

[0074] According to the feature information of the pictures extracted, clustering is performed, for example, the target number of clusters is 4. The clustering results include but are not limited to: picture 1 is clustered into a first picture group, picture 2 and picture 3 are clustered into a second picture group, picture 4 is clustered into a third picture group, and picture 5 is clustered into a fourth picture group.

[0075] According to the clustering results, a plurality of clustered picture groups are obtained, wherein the picture size of any picture in the plurality of clustered picture groups is greater than a first target value, and / or the picture capacity is greater than a second target value. The first target value and the second target value can be set according to experience, and optionally, the first target value is the size value of the terminal window, and the second target value is 3MB.

[0076] Taking the clustering results of the target page including five pictures as an example, the width of the terminal window is 12, the height of the terminal window is 18, the first target value is 12*18, and the second target value is 3MB. In the above clustering results, the picture size of the picture in the first picture group is less than the first target value, and the picture capacity is less than the second target value; the picture size of the picture in the second picture group is greater than the first target value, and the picture capacity is less than the second target value; the picture size of the picture in the third picture group is less than the first target value, and the picture capacity is greater than the second target value; and the picture size of the picture in the fourth picture group is greater than the first target value, and the picture capacity is greater than the second target value. Therefore, the second picture group, the third picture group and the fourth picture group are the plurality of clustered picture groups obtained.

[0077] In step 203, a screenshot of the target page is generated, a picture control in the screenshot is detected, and a detection result is obtained.

[0078] The embodiments of the present application do not limit the way of generating the screenshot of the target page. In a possible implementation, when the obtained target page is an activity containing pictures, a computer function is used to generate the screenshot of the activity, for example, the getDrawingCache() function is used to generate the screenshot of the activity; when the obtained target page is a scheme capable of starting an activity containing pictures, a first computer function is used to start the activity containing pictures, and a second computer function is used to generate the screenshot of the activity, for example, the first computer function is the startActivity() function, and the second computer function is the getDrawingCache() function.

[0079] In a possible implementation, the picture control in the screenshot of the target page is detected by a target detection method, and the target detection method includes any one of an R-CNN (region-based convolutional neural networks) method, a Fast R-CNN (fast region-based convolutional networks) method, a Faster R-CNN (faster region-based convolutional networks) method, a YOLO (you only look once) method, and an SSD (single shot multibox detector) method.

[0080] In a possible implementation, the R-CNN method is used for target detection, and the process of target detection includes steps 1 to 4.

[0081] Step 1, generating a candidate region of the target quantity in the screenshot, the candidate region including a candidate box, the candidate box being used to determine the boundary of the candidate region.

[0082] In a possible implementation, the selective search algorithm is used to generate the candidate region. In addition, the present application does not limit the target quantity, for example, the target quantity ranges from 1000 to 2000.

[0083] Step 2, extracting the feature of each candidate region by a CNN (convolutional neural networks).

[0084] Step 3, based on the features of each candidate region extracted, detecting the candidate regions belonging to the picture controls by a support vector machine (SVM) classifier, and obtaining a plurality of candidate regions of picture controls according to the detected candidate regions belonging to the picture controls.

[0085] In a possible implementation, among the detected candidate regions belonging to the picture controls, adjacent candidate regions are taken as candidate regions belonging to the same picture control.

[0086] Step 4, finely correcting the candidate boxes of the plurality of candidate regions of picture controls by using a regressor, and obtaining corrected candidate regions of the plurality of picture controls; and obtaining a detection result including a plurality of picture controls according to the corrected candidate regions of the plurality of picture controls, wherein the picture controls include sizes of the picture controls and capacities of the picture controls.

[0087] In a possible implementation, the regressor is a linear ridge regressor.

[0088] Step 204, matching the pictures of the plurality of clustered picture groups with the picture controls in the detection result, and obtaining the positions of the picture controls matched successfully in the target page.

[0089] In a possible implementation, matching the pictures of the plurality of clustered picture groups with the picture controls in the detection result includes, but is not limited to, the following two manners.

[0090] Manner one: determining the difference between the size of any picture control in the detection result and the picture size of any picture of any clustered picture group in the plurality of clustered picture groups; in response to the difference between the size of the any picture control and the picture size of any picture of the any clustered picture group being not greater than a first value, determining the difference between the capacity of the any picture control and the picture capacity of the any picture; and in response to the difference between the capacity of the any picture control and the picture capacity of the any picture being not greater than a second value, taking the any picture control as a picture control matched successfully.

[0091] The first value and the second value can be set according to experience, and embodiments of the present application are not limited. Exemplarily, the first value is 1% of the size of the detected picture control, and the second value is 1% of the capacity of the detected picture control.

[0092] Exemplarily, the plurality of clustered picture groups obtained in step 202 and the detection result of the screenshot of the target page are taken as examples for description. Figure 3 The screenshot of the target page is taken as an example for description, and the detection result of the screenshot of the target page is shown in Table 2.

[0093] Table 2

[0094] Picture control Width Height Capacity 1 5 3 2MB 2 14 4 2MB 3 14 4 2MB 4 5 3 4MB 5 14 4 4MB

[0095] The width and height of the picture control are dimensions of the picture control. The picture control 1 and the picture control 2 are taken as examples for illustration.

[0096] The difference between the picture size of the picture control 1 and the picture size of the picture in the second picture group is determined. In response to the difference between the picture size of the picture control 1 and the picture size of the picture in the second picture group being greater than the first value, the difference between the picture size of the picture control 1 and the picture size of the picture in the third picture group is determined. In response to the difference between the picture size of the picture control 1 and the picture size of the picture in the third picture group not being greater than the first value, the difference between the picture capacity of the picture control 1 and the picture capacity of the picture in the third picture group is determined. In response to the difference between the picture capacity of the picture control 1 and the picture capacity of the picture in the third picture group being greater than the second value, the difference between the picture size of the picture control 1 and the picture size of the picture in the fourth picture group is determined. In response to the difference between the picture size of the picture control 1 and the picture size of the picture in the fourth picture group being greater than the first value, the picture control 1 is determined as a picture control of which matching fails.

[0097] The difference between the picture size of the picture control 2 and the picture size of the picture in the second picture group is determined. In response to the difference between the picture size of the picture control 2 and the picture size of the picture in the second picture group not being greater than the first value, the difference between the capacity of the picture control 2 and the picture capacity of the picture is determined. In response to the difference between the capacity of the picture control 2 and the picture capacity of the picture not being greater than the second value, the picture control 2 is determined as a picture control of which matching succeeds.

[0098] The matching process of other picture controls in the detection result is the same as the above process principle, and will not be described here.

[0099] The second mode is to obtain a plurality of target pictures, wherein any target picture in the plurality of target pictures is used to indicate a cluster picture group in the plurality of cluster picture groups; the difference between the size of the picture control in the detection result and the picture size of any target picture in the plurality of target pictures is determined; in response to the difference between the size of any picture control in the detection result and the picture size of any target picture in the plurality of target pictures not being greater than a third value, the difference between the capacity of the picture control and the picture capacity of the target picture is determined; in response to the difference between the capacity of the picture control and the picture capacity of the target picture not being greater than a fourth value, the picture control and the picture in the cluster picture group indicated by the target picture are matched;

[0100] determining a difference between the size of the any picture control and a picture size of a picture in the cluster picture group indicated by the any target picture; in response to the difference between the size of the any picture control and the picture size of the picture in the cluster picture group indicated by the any target picture being not greater than a first value, determining a difference between the capacity of the any picture control and a picture capacity of the any picture; in response to the difference between the capacity of the any picture control and the picture capacity of the any picture being not greater than a second value, determining the any picture control as a matching successful picture control.

[0101] The third value and the fourth value can be set according to experience, and embodiments of the present application are not limited. Illustratively, the third value is 1% of the size of the detected picture control, and the fourth value is 1% of the capacity of the detected picture control.

[0102] In a possible implementation, the manner of obtaining the plurality of target pictures includes steps 1 to 3.

[0103] Step 1, for any cluster picture group in the plurality of cluster picture groups, obtaining feature information of a plurality of pictures of the any cluster picture group, the feature information of the plurality of pictures including feature information of each picture, and the feature information of each picture including a plurality of categories and feature values corresponding to the plurality of categories.

[0104] Step 2, for any category in the plurality of categories, obtaining a target feature value of the any category according to the feature values of the plurality of pictures belonging to the any category.

[0105] In a possible implementation, the target feature value is a weighted average of the feature values of the plurality of pictures belonging to the any category.

[0106] Step 3, obtaining a target picture corresponding to the any cluster picture group according to the target feature values of the plurality of categories.

[0107] In a possible implementation, for any cluster picture group in the plurality of cluster picture groups, obtaining feature information of a plurality of pictures of the any cluster picture group, the feature information of each picture in the plurality of pictures including a feature value of a picture size and a feature value of a picture capacity;

[0108] obtaining a target feature value of the picture size category according to the feature values of the plurality of pictures belonging to the picture size category, and obtaining a target feature value of the picture capacity category according to the feature values of the plurality of pictures belonging to the picture capacity category;

[0109] obtaining a target picture corresponding to the any cluster picture group according to the obtained target feature value of the picture size category and the target feature value of the picture capacity category.

[0110] Illustratively, a second picture group in the plurality of cluster picture groups obtained in step 202 is taken as an example for description.

[0111] The second picture group includes picture 2 and picture 3, the feature value of the picture size category of picture 2 is 14*4, the feature value of the picture capacity category is 2MB, the feature value of the picture size category of picture 3 is 14*4, and the feature value of the picture capacity category is 2MB.

[0112] According to the feature values of picture 2 and picture 3 belonging to the picture size category, the target feature value of the picture size category is obtained, and according to the feature values of picture 2 and picture 3 belonging to the picture capacity category, the target feature value of the picture capacity category is obtained. For example, a weighted average value of the feature values of picture 2 and picture 3 of the picture size category is calculated, and the weighted average value is taken as the target feature value of the picture size category, for example, the target feature value of the picture size category is 14*4; a weighted average value of the feature values of picture 2 and picture 3 of the picture capacity category is calculated, and the weighted average value is taken as the target feature value of the picture capacity category, for example, the target feature value of the picture capacity category is 2MB.

[0113] According to the target feature value of the picture size category and the target feature value of the picture category, a target picture is obtained, and the target picture is a target picture corresponding to the clustering picture group.

[0114] In a possible implementation, the position of the picture control in the target page is a coordinate of the picture control relative to at least one corner of the target page, for example, the position of the picture control in the target page is a coordinate of the picture control relative to the top left corner of the target page.

[0115] The above method obtains a plurality of target pictures, matches any picture control in the detection result with any target picture in the plurality of target pictures, obtains a clustering picture group corresponding to a target picture matched successfully, narrows down the search range of pictures of super size or super capacity, and then matches the picture control matched successfully with any picture in the clustering picture group, thereby improving the matching efficiency of the picture control in the detection result and the pictures of the plurality of clustering picture groups.

[0116] In a possible implementation, after step 204, the picture positioning method further includes:

[0117] Step 205, adjusting the picture corresponding to the picture control based on the obtained position of the picture control matched successfully in the target page.

[0118] In a possible implementation, when the picture size is greater than a first target value, the picture size is adjusted to be not more than the first target value, and when the picture capacity is greater than a second target value, the picture capacity is adjusted to be not more than the second target value. Illustratively, the first target value is the size of the terminal window, and the second target value is 3MB.

[0119] The method clusters pictures on a target page according to feature information of the pictures, and clusters pictures with a picture size greater than a first target value and / or a picture capacity greater than a second target value into a cluster picture group, thereby reducing the search range of pictures with an oversized size or capacity. Then, a picture control in a screenshot of the target page is detected. Since the position of the picture control on the target page can be determined, pictures in multiple cluster picture groups are matched with the picture control in the detection result, and the position of a picture control that passes the matching is obtained, thereby realizing automatic positioning of pictures with an oversized size or capacity on the target page and improving the positioning efficiency of the pictures. In addition, adjusting the pictures reduces memory occupation and waiting time, and improves user experience.

[0120] With reference to Figure 4 The embodiment of the present application provides a picture positioning device, which comprises:

[0121] The acquisition module 401 is configured to acquire a target page, and extract feature information of pictures in the target page.

[0122] The clustering module 402 is configured to cluster based on the feature information, and obtain multiple cluster picture groups according to a clustering result. Pictures in any cluster picture group of the multiple cluster picture groups have a picture size greater than a first target value and / or a picture capacity greater than a second target value.

[0123] The detection module 403 is configured to generate a screenshot of the target page, detect picture controls in the screenshot, and obtain a detection result.

[0124] The matching module 404 is configured to match pictures in multiple cluster picture groups with the picture controls in the detection result.

[0125] The acquisition module 401 is further configured to obtain the position of the picture control that passes the matching in the target page.

[0126] In a possible implementation, the acquisition module 401 is further configured to acquire feature information of multiple pictures in any cluster picture group of the multiple cluster picture groups. The feature information of the multiple pictures includes feature information of each picture, and the feature information of each picture includes multiple categories and feature values corresponding to the multiple categories. A target feature value of any category of the multiple categories is obtained according to feature values of multiple pictures belonging to the any category. A target picture corresponding to the any cluster picture group is obtained according to the target feature values of the multiple categories.

[0127] In a possible implementation, the matching module 404 is configured to determine a difference between the size of the picture control in the detection result and the picture size of any picture in any of the plurality of clustered picture groups; and in response to the difference between the size of any picture control in the detection result and the picture size of any picture in the any clustered picture group being not greater than a first value, determine a difference between the capacity of the any picture control and the picture capacity of the any picture.

[0128] In a possible implementation, the matching module 404 is configured to determine a difference between the size of the picture control in the detection result and the picture size of any target picture in the plurality of target pictures; in response to the difference between the size of any picture control in the detection result and the picture size of any target picture being not greater than a third value, determine a difference between the capacity of the any picture control and the picture capacity of the any target picture; and in response to the difference between the capacity of the any picture control and the picture capacity of the any target picture being not greater than a fourth value, match the any picture control with the pictures of the clustered picture group indicated by the any target picture.

[0129] determine a difference between the size of the any picture control and the picture size of any picture in the clustered picture group indicated by the any target picture; and in response to the difference between the size of the any picture control and the picture size of the any picture being not greater than a first value, determine a difference between the capacity of the any picture control and the picture capacity of the any picture.

[0130] In a possible implementation, the picture positioning apparatus further includes:

[0131] The adjusting module is configured to adjust the picture corresponding to the picture control based on the position of the picture control in the target page that is obtained.

[0132] The apparatus described above clusters pictures on a target page based on the feature information of the pictures, clusters pictures with a picture size greater than a first target value and / or a picture capacity greater than a second target value into a clustered picture group, and narrows down the search range of pictures with an oversized size or capacity. Then, the apparatus detects a picture control in a screenshot of the target page. Since the position of the picture control in the target page can be determined, the pictures of the plurality of clustered picture groups are matched with the picture control in the detection result, and the position of the picture control in the page that is matched successfully is obtained, thereby realizing automatic positioning of pictures with an oversized size or capacity in the target page and improving the positioning efficiency of the pictures. In addition, by adjusting the pictures, the memory occupation and the waiting time are reduced, and the user experience is improved.

[0133] It should be noted that the apparatus for image positioning provided in the above embodiments is only used as an example to illustrate the division of the above functional modules when performing image positioning. In actual applications, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the apparatus for image positioning is divided into different functional modules to complete all or part of the above described functions. In addition, the apparatus for image positioning provided in the above embodiments and the method for image positioning provided in the method embodiments belong to the same concept, and the specific implementation process is described in the method embodiments, which will not be described here.

[0134] Figure 5 A structural block diagram of an electronic device 500 provided by an example embodiment of the present application is shown. The electronic device 500 can be a portable mobile terminal, such as a smartphone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV), a notebook computer, or a desktop computer. The electronic device 500 can also be referred to as a user device, a portable terminal, a laptop terminal, a desktop terminal, or other names.

[0135] Generally, the electronic device 500 includes a processor 501 and a memory 502.

[0136] The processor 501 can include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 501 can be implemented in at least one of a hardware form of a DSP (Digital Signal Processing), an FPGA (Field Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 501 can also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a CPU (Central Processing Unit). The coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 501 can be integrated with a GPU (Graphics Processing Unit) for rendering and drawing content required to be displayed on a display screen. In some embodiments, the processor 501 can also include an AI (Artificial Intelligence) processor for processing machine learning related computing operations.

[0137] The memory 502 can include one or more computer-readable storage media. The computer-readable storage media can be non-transitory. The memory 502 can also include high-speed random access memory and can include nonvolatile memory, such as one or more magnetic disk storage devices, optical storage devices, flash memory devices, or other nonvolatile solid-state storage devices. In some embodiments, the non-transitory computer-readable storage medium of the memory 502 is used to store at least one instruction for execution by the processor 501 to enable the electronic device 500 to implement the method of picture positioning provided by the method embodiments of the present application.

[0138] In some embodiments, the electronic device 500 can also optionally include a peripheral device interface 503 and at least one peripheral device. The processor 501, the memory 502, and the peripheral device interface 503 can be connected through a bus or a signal line. Each peripheral device can be connected to the peripheral device interface 503 through a bus, a signal line, or a circuit board. Specifically, the peripheral device includes at least one of a radio frequency circuit 504, a display screen 505, a camera assembly 506, an audio circuit 507, a positioning assembly 508, and a power supply 509.

[0139] The peripheral device interface 503 can be used to connect at least one peripheral device related to input / output (I / O) to the processor 501 and the memory 502. In some embodiments, the processor 501, the memory 502, and the peripheral device interface 503 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 501, the memory 502, and the peripheral device interface 503 can be implemented on a separate chip or circuit board, and the present embodiment does not limit this.

[0140] The radio frequency circuit 504 is configured to receive and send RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 504 communicates with communication networks and other communication devices through electromagnetic signals. The radio frequency circuit 504 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the radio frequency circuit 504 includes an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a subscriber identity module card, and the like. The radio frequency circuit 504 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes but is not limited to the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network, and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 504 can also include NFC (Near Field Communication) related circuitry, which is not limited in the present application.

[0141] The display screen 505 is configured to display a UI (User Interface). The UI can include graphics, text, icons, video, and any combination thereof. When the display screen 505 is a touch display screen, the display screen 505 also has the ability to collect touch signals on or above the surface of the display screen 505. The touch signals can be input as control signals to the processor 501 for processing. At this time, the display screen 505 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 505 can be one, arranged on the front panel of the electronic device 500; in other embodiments, the display screen 505 can be at least two, arranged on different surfaces of the electronic device 500 or in a folding design; in other embodiments, the display screen 505 can be a flexible display screen, arranged on a curved surface or a folding surface of the electronic device 500. Even, the display screen 505 can also be arranged in an irregular shape, that is, a special-shaped screen. The display screen 505 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0142] The camera component 506 is configured to capture images or videos. Optionally, the camera component 506 includes a front camera and a rear camera. Generally, the front camera is disposed on the front panel of the terminal, and the rear camera is disposed on the back of the terminal. In some embodiments, the rear camera is at least two, which are any one of a main camera, a depth-of-field camera, a wide-angle camera, and a long-focus camera, to realize the background blur function of the main camera and the depth-of-field camera, the panorama shooting and VR (Virtual Reality) shooting function of the main camera and the wide-angle camera, or other fusion shooting functions. In some embodiments, the camera component 506 can further include a flash. The flash can be a single-color-temperature flash or a dual-color-temperature flash. The dual-color-temperature flash refers to the combination of a warm light flash and a cold light flash, which can be used for light compensation under different color temperatures.

[0143] The audio circuit 507 can include a microphone and a speaker. The microphone is configured to capture sound waves of a user and an environment, and convert the sound waves into an electrical signal input to the processor 501 for processing or input to the radio frequency circuit 504 to realize voice communication. For the purpose of stereo sound collection or noise reduction, the microphone can be multiple, which are respectively disposed at different parts of the electronic device 500. The microphone can also be an array microphone or an omnidirectional collection microphone. The speaker is configured to convert an electrical signal from the processor 501 or the radio frequency circuit 504 into sound waves. The speaker can be a traditional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert an electrical signal into a sound wave audible to humans, but also convert an electrical signal into an inaudible sound wave to humans for ranging purposes. In some embodiments, the audio circuit 507 can further include a headphone jack.

[0144] The positioning component 508 is configured to locate the current geographical position of the electronic device 500 to realize navigation or LBS (Location Based Service). The positioning component 508 can be a positioning component based on the GPS (Global Positioning System) of the United States, the Beidou system of China, or the Galileo system of Russia.

[0145] The power supply 509 is configured to supply power to each component in the electronic device 500. The power supply 509 can be alternating current, direct current, a disposable battery, or a rechargeable battery. When the power supply 509 includes a rechargeable battery, the rechargeable battery can be a wired charging battery or a wireless charging battery. The wired charging battery is a battery charged through a wired line, and the wireless charging battery is a battery charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0146] In some embodiments, the electronic device 500 further comprises one or more sensors 510. The one or more sensors 510 include, but are not limited to, an acceleration sensor 511, a gyroscope sensor 512, a pressure sensor 513, a fingerprint sensor 514, an optical sensor 515, and a proximity sensor 516.

[0147] The acceleration sensor 511 can detect the acceleration magnitude in three coordinate axes of a coordinate system established by the electronic device 500. For example, the acceleration sensor 511 can be used to detect the components of the gravitational acceleration in three coordinate axes. The processor 501 can control the display screen 505 to display the user interface in a landscape view or a portrait view according to the gravitational acceleration signals collected by the acceleration sensor 511. The acceleration sensor 511 can also be used for game or user motion data collection.

[0148] The gyroscope sensor 512 can detect the body orientation and rotation angle of the electronic device 500. The gyroscope sensor 512 can work with the acceleration sensor 511 to collect the 3D motion of the user to the electronic device 500. The processor 501 can implement the following functions according to the data collected by the gyroscope sensor 512: motion sensing (e.g., changing the UI according to the user's tilt operation), image stabilization when shooting, game control, and inertial navigation.

[0149] The pressure sensor 513 can be disposed on the side frame of the electronic device 500 and / or the lower layer of the display screen 505. When the pressure sensor 513 is disposed on the side frame of the electronic device 500, the user's holding signal to the electronic device 500 can be detected, and the left-hand or right-hand recognition or shortcut operation can be performed by the processor 501 according to the holding signal collected by the pressure sensor 513. When the pressure sensor 513 is disposed on the lower layer of the display screen 505, the controllable control on the UI interface can be controlled by the processor 501 according to the pressure operation of the user to the display screen 505. The controllable control includes at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0150] The fingerprint sensor 514 is used to collect the fingerprint of the user. The identity of the user can be recognized by the processor 501 according to the fingerprint collected by the fingerprint sensor 514, or by the fingerprint sensor 514 according to the collected fingerprint. When the identity of the user is recognized as a trusted identity, the user is authorized by the processor 501 to perform related sensitive operations, including unlocking the screen, viewing encrypted information, downloading software, payment, and changing settings. The fingerprint sensor 514 can be disposed on the front, back, or side of the electronic device 500. When a physical button or a manufacturer's logo is disposed on the electronic device 500, the fingerprint sensor 514 can be integrated with the physical button or the manufacturer's logo.

[0151] An optical sensor 515 is used to collect ambient light intensity. In one embodiment, the processor 501 can control the display brightness of the display screen 505 based on the ambient light intensity collected by the optical sensor 515. Specifically, when the ambient light intensity is high, the display brightness of the display screen 505 is increased; when the ambient light intensity is low, the display brightness of the display screen 505 is decreased. In another embodiment, the processor 501 can also dynamically adjust the shooting parameters of the camera assembly 506 based on the ambient light intensity collected by the optical sensor 515.

[0152] A proximity sensor 516, also known as a distance sensor, is typically mounted on the front panel of an electronic device 500. The proximity sensor 516 is used to detect the distance between the user and the front of the electronic device 500. In one embodiment, when the proximity sensor 516 detects that the distance between the user and the front of the electronic device 500 is gradually decreasing, the processor 501 controls the display screen 505 to switch from a screen-on state to a screen-off state; when the proximity sensor 516 detects that the distance between the user and the front of the electronic device 500 is gradually increasing, the processor 501 controls the display screen 505 to switch from a screen-off state to a screen-on state.

[0153] Those skilled in the art will understand that Figure 5 The structure shown does not constitute a limitation on the electronic device 500, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0154] In an exemplary embodiment, a computer-readable storage medium is also provided, which stores at least one piece of program code that is loaded and executed by a processor to enable a computer to implement any of the above-described image positioning methods.

[0155] Optionally, the aforementioned computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0156] In an exemplary embodiment, a computer program or computer program product is also provided, which stores at least one computer instruction, which is loaded and executed by a processor to enable the computer program or computer program product to implement any of the above-described image positioning methods.

[0157] It should be understood that the "multiple" mentioned herein refers to two or more than two. The "and / or" describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the three cases of A existing alone, A and B existing together, and B existing alone. The character " / " generally represents that the front and rear associated objects are in an "or" relationship.

[0158] The above-mentioned sequence numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0159] The above-mentioned only for the exemplary embodiments of the present application, and does not limit the present application, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A method for image localization, characterized in that, The method includes: acquiring a target page; extracting feature information of images within the target page; performing clustering based on the feature information; obtaining multiple clustered image groups based on the clustering results; wherein the image size of any image in any of the multiple clustered image groups is greater than a first target value, and / or the image size is greater than a second target value; generating a screenshot of the target page; detecting image controls in the screenshot; obtaining detection results; matching the images of the multiple clustered image groups with the image controls in the detection results; and obtaining the position of the successfully matched image controls on the target page. The step of matching the images of the plurality of clustered image groups with the image controls in the detection results includes: acquiring a plurality of target images, wherein any one of the plurality of target images is used to indicate one of the clustered image groups; in response to the fact that the difference between the size of any image control in the detection results and the image size of any target image in the plurality of target images is not greater than a third value, and the difference between the capacity of any image control and the image capacity of any target image is not greater than a fourth value, matching the any image control with the image of the clustered image group indicated by the any target image; in response to the fact that the difference between the size of any image control and the image size of any image in the clustered image group indicated by the any target image is not greater than a first value, and the difference between the capacity of any image control and the image capacity of any image is not greater than a second value, then the any image control is considered a successfully matched image control; The step of acquiring multiple target images includes: for any clustered image group among the multiple clustered image groups, acquiring feature information of multiple images in the any clustered image group, wherein the feature information of each image in the multiple images includes multiple categories and feature values ​​corresponding to the multiple categories; for any category among the multiple categories, obtaining the target feature value of the any category based on the feature values ​​of the multiple images belonging to the any category; and acquiring the target image corresponding to the any clustered image group based on the target feature values ​​of the multiple categories.

2. The method according to claim 1, characterized in that, The step of matching the images of the plurality of clustered image groups with the image controls in the detection results includes: determining the difference between the size of the image control in the detection results and the image size of any image in any of the plurality of clustered image groups; in response to the difference between the size of any image control in the detection results and the image size of any image in any of any clustered image groups not being greater than a first value, determining the difference between the capacity of any image control and the capacity of any image; in response to the difference between the capacity of any image control and the capacity of any image not being greater than a second value, then identifying the image control as a successfully matched image control.

3. The method according to any one of claims 1-2, characterized in that, After obtaining the position of the successfully matched image control on the target page, the method further includes: adjusting the image corresponding to the image control based on the obtained position of the successfully matched image control on the target page.

4. A device for image positioning, characterized in that, The device includes: an acquisition module for acquiring a target page and extracting feature information of images within the target page; a clustering module for clustering based on the feature information, obtaining multiple clustered image groups according to the clustering results, wherein the image size of any image in any of the multiple clustered image groups is greater than a first target value, and / or the image size is greater than a second target value; a detection module for generating a screenshot of the target page, detecting image controls in the screenshot, and obtaining a detection result; and a matching module for matching the images of the multiple clustered image groups with the image controls in the detection result, including: acquiring multiple target images, wherein any one of the multiple target images is used to indicate the multiple clustered image groups. A cluster of images in a group of images; in response to the difference between the size of any image control in the detection result and the image size of any target image in the plurality of target images not being greater than a third value, and the difference between the capacity of any image control and the image capacity of any target image not being greater than a fourth value, the image control is matched with the image in the cluster of images indicated by the target image; in response to the difference between the size of any image control and the image size of any image in the cluster of images indicated by the target image not being greater than a first value, and the difference between the capacity of any image control and the image capacity of any image not being greater than a second value, the image control is considered a successfully matched image control; The step of acquiring multiple target images includes: for any clustered image group among the multiple clustered image groups, acquiring feature information of multiple images in the any clustered image group, wherein the feature information of each image in the multiple images includes multiple categories and feature values ​​corresponding to the multiple categories; for any category among the multiple categories, obtaining the target feature value of the any category based on the feature values ​​of the multiple images belonging to the any category; acquiring the target image corresponding to the any clustered image group based on the target feature values ​​of the multiple categories; the acquisition module is also used to acquire the position of the successfully matched image control on the target page.

5. The apparatus according to claim 4, characterized in that, The device further includes an adjustment module, which is used to adjust the image corresponding to the image control based on the position of the successfully matched image control on the target page.

6. An electronic device, characterized in that, The electronic device includes a processor and a memory, wherein the memory stores at least one program code or instruction, which is loaded and executed by the processor to enable the electronic device to implement the image positioning method as described in any one of claims 1 to 3.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one piece of program code or instruction, which is loaded and executed by a processor to enable the computer to implement the image positioning method as described in any one of claims 1 to 3.

Images