Method and system for obtaining similar urls

Abstract

The application is suitable for the technical field of similarity calculation, and particularly relates to a similar URL obtaining method and system and a similar user obtaining method and system. The similar URL obtaining method comprises the following steps: obtaining a first URL; obtaining first click times of the first URL under a plurality of search words respectively; obtaining a second URL clicked under the plurality of search words; obtaining second click times of the second URL under the plurality of search words respectively; calculating behavior similarity of the first URL and the second URL according to the first click times and the second click times; and judging whether the second URL is a similar URL of the first URL according to the behavior similarity. Therefore, the application can effectively expand similarity, and expand and popularize advertisement information.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of similarity calculation, in particular to a method and system for obtaining similar URLs. BACKGROUND

[0002] In the prior art, targeted user expansion is an important part of displaying advertisements, which determines which users will be shown the advertisements. Traditionally, an advertiser provides a batch of URLs based on their understanding of the advertisement, so as to show the advertisement to users who have clicked on these URLs.

[0003] Such a practice has certain drawbacks:

[0004] 1. The advertiser's thinking is limited, and the provided URLs are not comprehensive enough, which affects the exposure of the advertisement;

[0005] 2. The advertiser's positioning of users is not accurate enough, and the provided URLs may not be consistent with the positioning of the advertisement.

[0006] In summary, the prior art has obvious disadvantages and defects in actual use, and therefore needs to be improved. SUMMARY

[0007] To overcome the above-mentioned defects, the present application provides a method and system for obtaining similar URLs and a method and system for obtaining similar users, which can effectively expand the similarity and promote the expansion of advertisements and other information.

[0008] To achieve the above-mentioned purpose, the present application provides a method for obtaining similar URLs, comprising:

[0009] obtaining a first URL;

[0010] obtaining the first click count of the first URL under a plurality of search terms;

[0011] obtaining a second URL that has been clicked under the plurality of search terms;

[0012] obtaining the second click count of the second URL under the plurality of search terms;

[0013] calculating the behavior similarity between the first URL and the second URL according to the first click count and the second click count;

[0014] determining whether the second URL is a similar URL of the first URL according to the behavior similarity.

[0015] According to the method for obtaining similar URLs, the step of obtaining a second URL that has been clicked under the plurality of search terms further comprises:

[0016] counting a co-occurrence number of the first url and the second url;

[0017] judging whether the co-occurrence number is less than or equal to a first threshold value, if yes, the second url is not a similar url of the first url.

[0018] According to the method for obtaining the similar url, the calculation formula of the behavior similarity is:

[0019]

[0020] Wherein, sim(url1, url2) represents the behavior similarity; the several search words are query1, query2, query3, ……queryn respectively. n pv 11 represents the first click number of the first url when searching query1, pv 21 represents the first click number of the first url when searching query2, and so on; pv 12 represents the second click number of the second url when searching query1, pv 22 represents the second click number of the second url when searching query2, and so on; n is the total number of the several search words.

[0021] According to the method for obtaining the similar url, the step of judging whether the second url is the similar url of the first url according to the behavior similarity includes:

[0022] judging whether the behavior similarity is greater than a second threshold value, if yes, the second url is the similar url of the first url.

[0023] According to the method for obtaining the similar url, the step of judging whether the second url is the similar url of the first url according to the behavior similarity includes:

[0024] obtaining a first title of the website of the first url, and obtaining a second title of the website of the second url;

[0025] calculating the semantic similarity of the first title and the second title;

[0026] fusing the semantic similarity and the behavior similarity to obtain a comprehensive similarity;

[0027] judging whether the comprehensive similarity is greater than a third threshold value, if yes, the second url is the similar url of the first url.

[0028] To achieve another inventive purpose of the present application, there is also provided a similar url obtaining system, comprising:

[0029] A first url obtaining module is configured to obtain a first url;

[0030] A first click number obtaining module is configured to obtain a first click number of the first url under a plurality of search terms;

[0031] A second url obtaining module is configured to obtain a second url clicked under the plurality of search terms;

[0032] A second click number obtaining module is configured to obtain a second click number of the second url under the plurality of search terms;

[0033] A behavior similarity calculating module is configured to calculate a behavior similarity between the first url and the second url according to the first click number and the second click number;

[0034] A website title obtaining module is configured to obtain a first title of a website of the first url and a second title of a website of the second url;

[0035] A semantic similarity calculating module is configured to calculate a semantic similarity between the first title and the second title;

[0036] A comprehensive similarity calculating module is configured to fuse the semantic similarity with the behavior similarity to obtain a comprehensive similarity;

[0037] A similar url judging module is configured to judge whether the comprehensive similarity is greater than a third threshold value, and if yes, the second url is a similar url of the first url.

[0038] To achieve another inventive purpose of the present application, there is also provided a similar user obtaining method, comprising:

[0039] Obtaining a first user;

[0040] Obtaining a first click number of the first user clicking a plurality of websites;

[0041] Obtaining all users clicking the plurality of websites;

[0042] Obtaining a second click number of the all users clicking the plurality of websites;

[0043] Calculating a similarity according to the first click number and the second click number, and judging a second user with a high similarity to the first user from the all users.

[0044] According to the similar user acquisition method, further comprising:

[0045] Acquiring first portrait data and first behavior data of the first user;

[0046] Acquiring second portrait data and second behavior data of the second user;

[0047] Representing the first portrait data and the first behavior data as a first vector;

[0048] Representing the second portrait data and the second behavior data as a second vector;

[0049] Calculating a similarity score of the first user and the second user according to the first vector and the second vector;

[0050] Determining whether the similarity score is greater than a first threshold value, and if yes, the second user is a similar user of the first user.

[0051] In order to achieve another object of the present application, a similar user acquisition system is further provided, comprising:

[0052] A first user acquisition module for acquiring a first user;

[0053] A first click number acquisition module for acquiring first click numbers of the first user on a plurality of websites;

[0054] A user acquisition module for acquiring all users who click on the plurality of websites;

[0055] A second click number acquisition module for acquiring second click numbers of the all users on the plurality of websites;

[0056] A second user acquisition module for calculating similarity according to the first click numbers and the second click numbers, and determining a second user with high similarity to the first user from the all users.

[0057] According to the similar user acquisition system, further comprising:

[0058] A first data acquisition module for acquiring first portrait data and first behavior data of the first user;

[0059] A second data acquisition module for acquiring second portrait data and second behavior data of the second user;

[0060] A first vector representation module for representing the first portrait data and the first behavior data as a first vector;

[0061] a second vector representation module configured to represent the second image data and the second behavior data as a second vector;

[0062] a similar user obtaining module configured to calculate a similarity score of the first user and the second user according to the first vector and the second vector, and determine whether the similarity score is greater than a first threshold value, and if yes, the second user is a similar user of the first user.

[0063] The application obtains a first url through a similar url obtaining method; the first url can be a url provided by an advertiser and needing to recommend an advertisement to a user; first click times of the first url under a plurality of search words are obtained; a second url clicked under the plurality of search words is obtained; second click times of the second url under the plurality of search words are obtained; a behavior similarity of the first url and the second url is calculated according to the first click times and the second click times; whether the second url is a similar url of the first url is determined according to the behavior similarity; the similar url can be taken as an expansion object, and information published on the first url can also be published on the similar url. Therefore, the application can effectively expand the similarity, and expand and popularize the information such as an advertisement. BRIEF DESCRIPTION OF DRAWINGS

[0064] Figure 1 is a structure diagram of a similar url obtaining system provided by a preferred embodiment of the application;

[0065] Figure 2 is one of flowcharts of a similar url obtaining method provided by a preferred embodiment of the application;

[0066] Figure 3 is another of flowcharts of a similar url obtaining method provided by a preferred embodiment of the application;

[0067] Figure 4 is one of structure diagrams of a similar user obtaining system provided by a preferred embodiment of the application;

[0068] Figure 5 is another of structure diagrams of a similar user obtaining system provided by a preferred embodiment of the application;

[0069] Figure 6 is a flowchart of a similar user obtaining method provided by a preferred embodiment of the application;

[0070] Figure 7 is a structure diagram of a Dssm model provided by a preferred embodiment of the application. DETAILED DESCRIPTION

[0071] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application.

[0072] It should be noted that the use of "one embodiment", "an embodiment", "example embodiment", etc. in the specification indicates that the described embodiment can include a particular feature, structure, or characteristic, but every embodiment can not necessarily include the particular feature, structure, or characteristic. In addition, such expressions are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to incorporate such feature, structure, or characteristic into other embodiments whether or not they are explicitly described or not.

[0073] In addition, some terms are used in the specification and subsequent claims to refer to certain components or parts, and those skilled in the art should understand that manufacturers can use different names or terms to refer to the same component or part. The specification and subsequent claims do not distinguish components or parts by name, but by functional differences. Throughout the specification and subsequent claims, "including" and "containing" are open terms, which should be interpreted as "including but not limited to". In addition, the term "connected" includes any direct and indirect electrical connection means. Indirect electrical connection means includes connection through other devices.

[0074] Figure 1 is a structural schematic diagram of the similar url acquisition system 100 provided by the present embodiment, and in the first embodiment of the present application, a similar url acquisition system 100 is provided, which comprises:

[0075] The first url acquisition module 10 is configured to acquire a first url.

[0076] The first click number acquisition module 20 is configured to acquire a first click number of the first url under a plurality of search terms.

[0077] The second url acquisition module 30 is configured to acquire a second url clicked under the plurality of search terms.

[0078] The second click number acquisition module 40 is configured to acquire a second click number of the second url under the plurality of search terms.

[0079] The behavior similarity calculation module 50 is configured to calculate a behavior similarity between the first url and the second url according to the first click number and the second click number.

[0080] a website title obtaining module 60, configured to obtain a first title of a website of the first URL and a second title of a website of the second URL;

[0081] a semantic similarity calculating module 70, configured to calculate a semantic similarity between the first title and the second title;

[0082] a comprehensive similarity calculating module 80, configured to fuse the semantic similarity and the behavior similarity to obtain a comprehensive similarity;

[0083] a similar URL judging module 90, configured to judge whether the comprehensive similarity is greater than a third threshold value, and if yes, the second URL is a similar URL of the first URL.

[0084] In this embodiment, the similar url obtaining system 100 can expand the url for advertisement recommendation, where the url refers to the website address related to the advertisement of the advertiser. For example, the advertiser needs to launch the game type advertisement, and the advertiser will provide a batch of competitor website addresses. As long as the user clicks on these website addresses, the advertiser's advertisement will be displayed. First, the advertiser provides a batch of seed urls (all first urls), and as long as the user clicks on these first urls, the user will be displayed with the advertisement of the advertiser. By expanding the similar url, the advertiser has a batch of seed urls (i.e. first urls) in addition to the eurl (i.e. similar url) expanded by the seed url. As long as the user clicks on the seed url or the similar url expanded by the seed url, the user can be displayed with the advertisement of the advertiser. The problem of incomplete url provided by the advertiser is solved. Specifically, the first url is obtained by the first url obtaining module 10, and the first url can be provided by the advertiser. A plurality of co-click data can be obtained through the browser log, each co-click data including a search term, a url clicked under the search term, and a number of clicks on the url under the search term, thereby obtaining a plurality of search terms related to the first url, and obtaining a first number of clicks on the first url by the user when searching through the search term in the browser; the second number of clicks on the second url by the user when searching through the search term in the browser can be obtained through the browser log; the first url and the second url are collectively referred to as url-pair, and the behavior similarity (i.e. url-pair similarity) between the first url and the second url is calculated according to the first number of clicks and the second number of clicks; the behavior similarity can be used to determine whether the second url is a similar url, but in order to more accurately determine the similarity, the semantic similarity is further used to assist the behavior similarity in determining the similarity. The semantic similarity and the behavior similarity are fused for comprehensive consideration. The semantic similarity refers to the semantic similarity between the first title of the website of the first url and the second title of the website of the second url, and the first title and the second title are combined into a title-pair. People intuitively determine whether two urls are similar or not based on the semantics between the website contents, so the similarity between two websites can be determined by the titles of the websites. When the comprehensive similarity determined by the similar url determining module 90 is greater than a third threshold value, the second url is a similar url of the first url. The third threshold value is preferably 0.5. Thus, the similar url obtaining system 100 obtains the similar url of the seed url provided by the advertiser, and more comprehensive information such as advertisements is displayed to the user.

[0085] Figure 2is a flow chart of the similar url obtaining method of the present application, which can be implemented by the similar url obtaining system 100 as shown in the figure, the similar url obtaining method comprising: Figure 1

[0086] Step S201, obtaining a first url; implemented by the first url obtaining module 10;

[0087] Step S202, obtaining first click times of the first url under several search terms respectively; implemented by the first click times obtaining module 20;

[0088] Step S203, obtaining second urls clicked under the several search terms; implemented by the second url obtaining module 30;

[0089] Step S204, obtaining second click times of the second urls under the several search terms respectively; implemented by the second click times obtaining module 40;

[0090] Step S205, calculating behavior similarity of the first url and the second url according to the first click times and the second click times; implemented by the behavior similarity calculating module 50;

[0091] Step S206, judging whether the second url is a similar url of the first url according to the behavior similarity.

[0092] In this embodiment, the similar url obtaining method can be extended to recommend urls of advertisements, where url refers to the website related to the advertisements of the advertisers. The first url can be obtained by the advertisers; several co-click data can be obtained from the logs of the browser, each co-click data including a search term, a url clicked under the search term, and the number of clicks of the url under the search term, thereby obtaining several search terms related to the first url, and obtaining the first click times of the first url clicked by the user when searching by the search term in the browser; the second click times of the second url clicked by the user when searching by the search term in the browser can be obtained from the logs of the browser; the first url and the second url are collectively referred to as url-pair, the behavior similarity of the first url and the second url (i.e. the similarity of the url-pair) is calculated according to the first click times and the second click times, the second url can be judged to be a similar url according to the behavior similarity, if the second url is a similar url, the advertisement of the advertiser can be displayed in the second url, thereby realizing more comprehensive display of the advertisements and other information to the user, and increasing the exposure of the advertisements.

[0093] ​In an embodiment of the present application, the step S203 further comprises:

[0094] calculating the co-occurrence times of the first url and the second url;

[0095] judging whether the co-occurrence times are less than or equal to a first threshold value, if yes, the second url is not the similar url of the first url.

[0096] In this embodiment, in order to prevent the noise data caused by the user's mis-click, the mis-clicked second url is removed by calculating the co-occurrence times; if the co-occurrence times of the first url and the second url are above the average level, it is considered that the above two urls are the data generated by the normal click; the co-occurrence times of the first url and the second url refer to the times of being clicked together under how many search words. For example, the first url and the second url are clicked when searching query1, and the co-occurrence times of the first url and the second url are 1 time; if the first url and the second url are clicked when searching query1,…,queryn, the co-occurrence times of the first url and the second url are n times. The first threshold value is preferably 3, because the number of co-clicks of the user's search query and url is very large, and it is relatively costly to process, and most of the data are long-tailed, and when the data level is controlled to the order of ten million, the first threshold value of the co-occurrence times is 3.

[0097] Specifically, the calculation formula of the behavior similarity is:

[0098]

[0099] wherein, sim(url1, url2) represents the behavior similarity; the search words are query1, query2, query3,…,queryn respectively. n pv 11 represents the first click times of the first url when searching query1, pv 21 represents the first click times of the first url when searching query2, and the like; pv 12 represents the second click times of the second url when searching query1, pv 22 represents the second click times of the second url when searching query2, and the like; n is the total number of the search words.

[0100] In an embodiment of the present application, the step S206 comprises:

[0101] determining whether the behavior similarity is greater than a second threshold value, and if yes, the second URL is a similar URL of the first URL.

[0102] In an embodiment of the present application, the step S206 comprises:

[0103] obtaining a first title of a website of the first URL and a second title of a website of the second URL;

[0104] calculating a semantic similarity of the first title and the second title;

[0105] fusing the semantic similarity with the behavior similarity to obtain a comprehensive similarity;

[0106] determining whether the comprehensive similarity is greater than a third threshold value, and if yes, the second URL is a similar URL of the first URL.

[0107] In this embodiment, in order to more accurately determine the similarity of the first URL and the second URL, the semantic similarity is further used to assist the behavior similarity to determine. The semantic similarity and the behavior similarity are fused to comprehensively consider. The semantic similarity refers to the semantic similarity of the first title of the website of the first URL and the second title of the website of the second URL. The first title and the second title are integrated into a title-pair. People intuitively determine whether two URLs are similar or not based on the semantics between the contents of the websites. Therefore, the similarity between two websites can be determined by using the titles of the websites. When the similar URL determining module 90 determines that the comprehensive similarity is greater than the third threshold value, the second URL is a similar URL of the first URL. Since there is a mapping table of a URL and a title, the first title corresponding to the first URL is obtained from the mapping table, and the second title corresponding to the second URL is obtained from the mapping table, so as to obtain the title-pair. The third threshold value is preferably 0.5.

[0108] In an embodiment of the present application, the step of calculating the semantic similarity of the first title and the second title comprises:

[0109] extracting a first semantic vector of the first title and extracting a second semantic vector of the second title;

[0110] calculating the semantic similarity according to the first semantic vector and the second semantic vector by using cosine similarity.

[0111] In this embodiment, the first semantic vector and the second semantic vector can be obtained by a similarity algorithm such as Fasttext or bert; Fasttext and bert are two natural language processing algorithms, and are open source codes that can extract the vector of a sentence.

[0112] Specifically, the first semantic vector is represented as (x1, …, xn);

[0113] The second semantic vector is represented as (y1, …, yn);

[0114] The calculation formula of the semantic similarity is:

[0115]

[0116] Wherein, cos(θ) represents the semantic similarity.

[0117] Specifically, the calculation formula of the comprehensive similarity obtained by fusing the semantic similarity and the behavior similarity is:

[0118] y=αsimilarAction+βsimilarSentiment

[0119] Wherein, similarAction represents the behavior similarity, similarSentiment represents the semantic similarity, and α and β are two hyperparameters used to control the proportion of the behavior similarity and the semantic similarity in the similarity of the first url and the second url.

[0120] In this embodiment, the values of α and β are preferably set to 0.5. The values of α and β represent the importance of the behavior similarity and the semantic similarity, and depend on the focus point in the task. If more attention is paid to the semantics, the value of β can be adjusted to be larger, and the value of α can be adjusted to be smaller;

[0121] In one embodiment of the application, the step of judging whether the comprehensive similarity is greater than a third threshold value, if yes, the second url is a similar url of the first url, comprises:

[0122] Obtaining the first pv amount of the website of the first url, and obtaining the second pv amount of the website of the second url; calculating the multiple of the difference between the first pv amount and the second pv amount;

[0123] If it is judged that the comprehensive similarity is greater than the third threshold value and the multiple of the difference between the first pv amount and the second pv amount is less than a fourth threshold value, the second url is a similar url of the first url.

[0124] In this embodiment, Pv represents the daily access volume of the website, which reflects the magnitude of the website to some extent. Avoiding the advertiser to provide a niche website, such as "home of cars", the similar website developed is "sohu car", if the advertisement is displayed to all users who will point to "sohu car", the exposure is too large, which will cause the advertiser's fund to be spent too fast, and bring risks. Therefore, the second url which is too different from the magnitude of the seed url is also filtered out. The specific implementation process is: obtaining the user browsing record from September 17, 2019 to October 17, 2019, and counting the daily access volume of each website, if the first pv quantity and the second pv quantity differ by 100 times, the second url will be filtered.

[0125] Figure 3 The flow chart of the acquisition method of the similar url of the preferred embodiment is as follows, specifically:

[0126] Firstly, a plurality of co-click data are acquired, which can be acquired through browser logs, each co-click data includes a search word, a url clicked under the search word, and a number of clicks of the url clicked under the search word; therefore, the above co-click data includes a first url (the first url is a seed url, which can be provided by an advertiser), a plurality of search words related to the first url, a first number of clicks, a second url, and a second number of clicks. Since the search word related to the first url is searched, not only the first url but also the second url is clicked, therefore, the first url and the second url can have similarity, the first url and the second url are collectively referred to as url-pair. The behavior similarity of the url-pair is calculated.

[0127] Then, in order to prevent noise data caused by user's mis-click, it is judged whether the co-occurrence times of the url-pair is greater than 3, if yes, the subsequent steps are continued to be executed;

[0128] Then, in order to more accurately judge the similarity of the first url and the second url, the first title and the second title are acquired, the first title and the second title are combined into a title-pair, and the semantic similarity of the title-pair is calculated through Fasttext and bert.

[0129] Then, the behavior similarity of the url-pair and the semantic similarity of the title-pair are comprehensively calculated to obtain a comprehensive similarity.

[0130] Finally, it is judged whether the comprehensive similarity is greater than 0.5 and the pv magnitude is less than 100 times, if yes, the first url and the second url are similar urls.

[0131] Figure 4is a structural schematic diagram of the similar user obtaining system 200 provided by the embodiment, and the similar user obtaining system 200 provided by an embodiment of the application comprises:

[0132] The first user obtaining module 210 is used for obtaining a first user.

[0133] The first click number obtaining module 220 is used for obtaining first click numbers of the first user respectively clicking a plurality of websites.

[0134] The user obtaining module 230 is used for obtaining all users clicking the plurality of websites.

[0135] The second click number obtaining module 240 is used for obtaining second click numbers of the all users respectively clicking the plurality of websites.

[0136] The second user obtaining module 250 is used for performing similarity calculation according to the first click numbers and the second click numbers, and judging second users with high similarity to the first user from the all users.

[0137] In this embodiment, the similar user acquisition system 200 can be used to expand users for advertising recommendation, not only relevant advertising recommendation for seed users, but also relevant advertising recommendation for similar users expanded from seed users, so as to realize appropriate and reasonable increase of the exposure of the advertisement of the advertiser. First, the seed user (i.e. the first user) is acquired, the user who clicks the advertisement of the advertiser can be acquired according to the browsing record of the user, as long as the user clicks the advertisement of the advertiser, it indicates that the user is interested in the advertisement of the advertiser, which is the target user of the advertiser, so the user is taken as the seed user; the similar user with high similarity to the seed user is expanded, and the advertisement can be recommended to the similar user to increase the exposure of the advertisement. Specifically, the first user clicks a plurality of first click times of the website respectively, the more times a user clicks the same website, the more the user pays attention to or likes the website, the number of clicks of a user on a website can represent the interest degree of the user on the website, the more the number of clicks, the greater the interest degree, the less the number of clicks, the smaller the interest degree. The similarity of the interest of two users on some websites can be used to determine whether the two users are similar. Thus, when the seed user clicks a plurality of websites, all users who click a plurality of websites can be acquired, and the second click times of the all users on a plurality of websites are acquired, the similarity is calculated according to the first click times and the second click times, the second user with high similarity to the first user is determined from the all users, the second user can be taken as the similar user, and the advertisement of the advertiser is also recommended to the similar user, so as to solve the inaccurate problem of the user positioning of the advertiser and realize the increase of the advertising exposure.

[0138] Figure 5 is a structural schematic diagram of the similar user acquisition system 200 provided by the embodiment, in an embodiment of the application, further comprising:

[0139] The first data acquisition module 260 is used to acquire the first portrait data and the first behavior data of the first user.

[0140] The second data acquisition module 270 is used to acquire the second portrait data and the second behavior data of the second user.

[0141] The first vector representation module 280 is used to represent the first portrait data and the first behavior data into a first vector.

[0142] The second vector representation module 290 is used to represent the second portrait data and the second behavior data into a second vector.

[0143] The similar user obtaining module 300 is configured to calculate a similarity score of the first user and the second user according to the first vector and the second vector; and determine whether the similarity score is greater than a first threshold value, and if yes, the second user is a similar user of the first user.

[0144] In this embodiment, in order to more accurately identify similar users and improve the cost performance of resource use, further similarity determination can be performed. The similarity of two users can be determined by user portrait data and behavior data of the two users. The user portrait data refers to user characteristics such as age, gender, city, and the like. The user portrait data is currently maintained internally. The user behavior data refers to data such as recent search records and recent click advertisement records. The recent data can be data in the last day or the last seven days, and the like, which can be determined according to a specific application scenario. A dssm model can be used to calculate a similarity score (see Figure 7 ). The full name of the Dssm model is Deep Structured Semantic Models. The use of the dssm is only the use of the idea of the two-tower model therein, that is, the seed user walks a network, and the similar user expanded from the seed user walks another network, and then the outputs of the two networks are used to calculate the similarity. The first data obtaining module 260 and the second data obtaining module 270 are two input layers in the Dssm model. The first data obtaining module 260 obtains first portrait data and first behavior data of the first user, and the second data obtaining module 270 obtains second portrait data and second behavior data of the second user. The first vector representation module 280 and the second vector representation module 290 are two representation layers in the Dssm model, which are used to represent the user data input by the first data obtaining module 260 and the second data obtaining module 270 into vectors, respectively. Because the basic attribute characteristics and behavior characteristics of the user are high-dimensional discrete characteristics, in order to facilitate the calculation of the model, it is necessary to map them to a dense vector space. The similar user obtaining module 300 is a matching layer in the Dssm model, which is used to calculate a similarity score of the first user and the second user. The similarity score is a value between 0 and 1. The closer to 1, the more similar the two users are, and the closer to 0, the less similar the two users are.

[0145] Specifically, the first portrait data includes gender, age, and location of the first user.

[0146] The first behavior data includes browsing information records, search information records, and click information records of the first user.

[0147] The second portrait data includes gender, age, and location of the second user.

[0148] The second behavior data includes browsing information records, search information records, and click information records of the second user.

[0149] Specifically, the first vector representation module 280 represents the first vector as (x1, …, xn);

[0150] The second vector representation module 290 represents the second vector as (y1, …, yn);

[0151] The similar user obtaining module 300 includes a similarity score calculation module 301, which calculates the similarity score according to the following formula:

[0152]

[0153] Wherein, cos(θ) represents the similarity score.

[0154] Specifically, the second user obtaining module 250 includes a similarity calculation module 251, which calculates the similarity according to the following formula:

[0155]

[0156] Wherein, sim(user1, user2) represents the similarity, and the several websites are url1, url2, url3, …, urln. n , pv 11 represents the first click times of the first user clicking url1, pv 21 represents the first click times of the first user clicking url2, and so on; pv 12 represents the second click times of the second user clicking url1, pv 22 represents the second click times of the second user clicking url2, and so on; n is the total number of the several websites.

[0157] Figure 6 The flowchart of the similar user obtaining method of the present application can be realized by the similar user obtaining system 200 as shown in Figures 3-4 The similar user obtaining method includes:

[0158] Step S601, obtaining a first user; realized by the first user obtaining module 210;

[0159] Step S602, obtaining first click times of the first user clicking several websites; realized by the first click times obtaining module 220;

[0160] Step S603, all users clicking on several websites are obtained; the user acquisition module 230 is used to achieve the step;

[0161] Step S604, the second click times of all users clicking on several websites are obtained; the second click times acquisition module 240 is used to achieve the step;

[0162] Step S605, similarity calculation is performed according to the first click times and the second click times, and second users with high similarity to the first user are determined from all users; the second user acquisition module 250 is used to achieve the step.

[0163] In an embodiment of the present application, the present application further comprises:

[0164] First portrait data and first behavior data of the first user are obtained; the first data acquisition module 260 is used to achieve the step;

[0165] Second portrait data and second behavior data of the second user are obtained; the second data acquisition module 270 is used to achieve the step;

[0166] The first portrait data and the first behavior data are represented as a first vector; the first vector representation module 280 is used to achieve the step;

[0167] The second portrait data and the second behavior data are represented as a second vector; the second vector representation module 290 is used to achieve the step;

[0168] The similarity score of the first user and the second user is calculated according to the first vector and the second vector;

[0169] It is judged whether the similarity score is greater than a first threshold value, if yes, the second user is a similar user of the first user; the similar user acquisition module 300 is used to achieve the step.

[0170] Specifically, the first portrait data includes gender, age, and location of the first user;

[0171] The first behavior data includes browsing information records, search information records, and click information records of the first user;

[0172] The second portrait data includes gender, age, and location of the second user;

[0173] The second behavior data includes browsing information records, search information records, and click information records of the second user.

[0174] Specifically, the first vector is represented as (x1, …, xn);

[0175] The second vector is represented as (y1, …, yn);

[0176] The calculation formula of the similarity score is:

[0177]

[0178] Wherein, cos (θ) represents the similarity score.

[0179] Specifically, the calculation formula for similarity calculation according to the first click number and the second click number is:

[0180]

[0181] Wherein, sim (user1, user2) represents similarity, and the several websites are url1, url2, url3, ……urln. n , pv 11 represents the first click number of the first user clicking url1, pv 21 represents the first click number of the first user clicking url2, and so on; pv 12 represents the second click number of the second user clicking url1, pv 22 represents the second click number of the second user clicking url2, and so on; and n is the total number of the several websites.

[0182] In summary, the first url is obtained by the method for obtaining similar urls; the first url can be a url provided by an advertiser and needing to recommend an advertisement to a user; the first click number of the first url under several search words is obtained; the second url clicked under the several search words is obtained; the second click number of the second url under the several search words is obtained; the behavior similarity of the first url and the second url is calculated according to the first click number and the second click number; and whether the second url is a similar url of the first url is judged according to the behavior similarity. The similar url can be taken as an expansion object, and the information published on the first url is also published on the similar url. Therefore, the present application can effectively expand the similarity, and expand and popularize the information such as advertisements.

[0183] Of course, the present application can have other various embodiments, and those skilled in the art can make various corresponding changes and modifications according to the present application without departing from the spirit and essence of the present application. However, these corresponding changes and modifications should all belong to the protection scope of the claims attached to the present application.

[0184] The present application also discloses a method for obtaining similar urls, comprising:

[0185] Get the first URL;

[0186] Get the first click count of the first URL under several search terms;

[0187] Obtain the second URL that was clicked under the aforementioned search terms;

[0188] Obtain the second number of clicks for each of the several search terms for the second URL;

[0189] Calculate the behavioral similarity between the first URL and the second URL based on the first number of clicks and the second number of clicks;

[0190] Based on the behavioral similarity, it is determined whether the second URL is a similar URL to the first URL.

[0191] A2. According to the method for obtaining similar URLs described in A1, the step of obtaining the second URL clicked under the plurality of search terms further includes:

[0192] Calculate the number of times the first URL and the second URL co-occur.

[0193] Determine whether the number of co-occurrences is less than or equal to a first threshold. If so, then the second URL is not a similar URL to the first URL.

[0194] A3. Based on the method for obtaining similar URLs described in A1, the formula for calculating the behavioral similarity is:

[0195]

[0196] Where sim(url1,url2) represents the behavior similarity; the search terms are query1, query2, query3, ..., query. n ,pv 11 This represents the number of times the first URL was clicked when searching query1, PV. 21 This indicates the number of times the first URL was clicked when searching query2, and so on; PV 12 This indicates the number of clicks on the second URL when searching query1, PV. 22 This indicates the number of times the second URL is clicked when searching query2, and so on; n is the total number of the search terms.

[0197] A4. According to the method for obtaining similar URLs described in A1, the step of determining whether the second URL is a similar URL to the first URL based on the behavioral similarity includes:

[0198] determining whether the behavior similarity is greater than a second threshold value, and if so, the second URL is a similar URL of the first URL.

[0199] A5, the method according to A1, the step of determining whether the second URL is a similar URL of the first URL according to the behavior similarity comprises:

[0200] obtaining a first title of a website of the first URL, and obtaining a second title of a website of the second URL;

[0201] calculating a semantic similarity of the first title and the second title;

[0202] fusing the semantic similarity and the behavior similarity to obtain a comprehensive similarity;

[0203] determining whether the comprehensive similarity is greater than a third threshold value, and if so, the second URL is a similar URL of the first URL.

[0204] A6, the method according to A5, the step of calculating the semantic similarity of the first title and the second title comprises:

[0205] extracting a first semantic vector of the first title and extracting a second semantic vector of the second title;

[0206] calculating the semantic similarity according to the first semantic vector and the second semantic vector by cosine similarity.

[0207] A7, the method according to A6,

[0208] the first semantic vector is represented as (x1, …, xn);

[0209] the second semantic vector is represented as (y1, …, yn);

[0210] the calculation formula of the semantic similarity is:

[0211]

[0212] wherein, cos(θ) represents the semantic similarity.

[0213] A8, the method according to A6, the calculation formula of the comprehensive similarity obtained by fusing the semantic similarity and the behavior similarity is:

[0214] y = α similarAction + β similarSentiment

[0215] wherein the similarAction represents the behavior similarity, the similarSentiment represents the semantic similarity, and a and β are two hyperparameters used to control the proportion of the behavior similarity and the semantic similarity in the similarity of the first url and the second url.

[0216] A9. The method of obtaining a similar url according to A5, wherein the step of determining whether the comprehensive similarity is greater than a third threshold value, if yes, the second url is a similar url of the first url, comprises:

[0217] obtaining a first pv quantity of the website of the first url, and obtaining a second pv quantity of the website of the second url; calculating a multiple of the difference between the first pv quantity and the second pv quantity

[0218] if it is determined that the comprehensive similarity is greater than the third threshold value and the multiple of the difference between the first pv quantity and the second pv quantity is less than a fourth threshold value, the second url is a similar url of the first url.

[0219] B10. A system for obtaining a similar url, comprising:

[0220] a first url obtaining module, configured to obtain a first url;

[0221] a first click quantity obtaining module, configured to obtain a first click quantity of the first url under a plurality of search terms;

[0222] a second url obtaining module, configured to obtain a second url clicked under the plurality of search terms;

[0223] a second click quantity obtaining module, configured to obtain a second click quantity of the second url under the plurality of search terms;

[0224] a behavior similarity calculating module, configured to calculate a behavior similarity of the first url and the second url according to the first click quantity and the second click quantity;

[0225] a website title obtaining module, configured to obtain a first title of the website of the first url, and obtain a second title of the website of the second url;

[0226] a semantic similarity calculating module, configured to calculate a semantic similarity of the first title and the second title;

[0227] a comprehensive similarity calculating module, configured to fuse the semantic similarity and the behavior similarity to obtain a comprehensive similarity;

[0228] The similar URL judging module is configured to judge whether the comprehensive similarity is greater than a third threshold value, and if yes, the second URL is a similar URL of the first URL.

[0229] C11. A method for obtaining similar users, comprising:

[0230] obtaining a first user;

[0231] obtaining first click times of the first user on a plurality of websites;

[0232] obtaining all users who click on the plurality of websites;

[0233] obtaining second click times of the all users on the plurality of websites;

[0234] performing similarity calculation according to the first click times and the second click times, and judging second users with high similarity to the first user from the all users.

[0235] C12. The method of C11, further comprising:

[0236] obtaining first portrait data and first behavior data of the first user;

[0237] obtaining second portrait data and second behavior data of the second user;

[0238] representing the first portrait data and the first behavior data as a first vector;

[0239] representing the second portrait data and the second behavior data as a second vector;

[0240] calculating a similarity score of the first user and the second user according to the first vector and the second vector;

[0241] judging whether the similarity score is greater than a first threshold value, and if yes, the second user is a similar user of the first user.

[0242] C13. The method of C12, characterized in that:

[0243] the first portrait data comprises gender, age, and location of the first user;

[0244] the first behavior data comprises browsing information records, search information records, and click information records of the first user;

[0245] the second portrait data comprises gender, age, and location of the second user;

[0246] The second behavior data includes browsing information records, search information records, and click information records of the second user.

[0247] C14. The method of acquiring similar users according to C12, characterized in that,

[0248] The first vector is represented as (x1, …, xn);

[0249] The second vector is represented as (y1, …, yn);

[0250] The calculation formula of the similarity score is:

[0251]

[0252] Wherein, cos(θ) represents the similarity score.

[0253] C15. The method of acquiring similar users according to C11, characterized in that, the calculation formula of similarity calculation according to the first click times and the second click times is:

[0254]

[0255] Wherein, sim(user1, user2) represents similarity, and the several websites are url1, url2, url3, ……urln. n , pv 11 represents the first click times of the first user clicking url1, pv 21 represents the first click times of the first user clicking url2, and so on; pv 12 represents the second click times of the second user clicking url1, pv 22 represents the second click times of the second user clicking url2, and so on; n is the total number of the several websites.

[0256] D16. A system for acquiring similar users, comprising:

[0257] A first user acquisition module for acquiring a first user;

[0258] A first click times acquisition module for acquiring first click times of the first user clicking several websites;

[0259] A user acquisition module for acquiring all users clicking several websites;

[0260] A second click times acquisition module for acquiring second click times of the all users clicking several websites;

[0261] The second user acquisition module is configured to determine a second user with high similarity to the first user from all users according to the first click number and the second click number.

[0262] D17. The similar user acquisition system of D16, further comprising:

[0263] The first data acquisition module is configured to acquire first portrait data and first behavior data of the first user.

[0264] The second data acquisition module is configured to acquire second portrait data and second behavior data of the second user.

[0265] The first vector representation module is configured to represent the first portrait data and the first behavior data as a first vector.

[0266] The second vector representation module is configured to represent the second portrait data and the second behavior data as a second vector.

[0267] The similar user acquisition module is configured to calculate a similarity score of the first user and the second user according to the first vector and the second vector, and determine whether the similarity score is greater than a first threshold value, and if so, the second user is a similar user of the first user.

[0268] D18. The similar user acquisition system of D17, wherein the first portrait data comprises gender, age, and location of the first user.

[0269] The first behavior data comprises browsing information records, search information records, and click information records of the first user.

[0270] The second portrait data comprises gender, age, and location of the second user.

[0271] The second behavior data comprises browsing information records, search information records, and click information records of the second user.

[0272] D19. The similar user acquisition system of D17,

[0273] The first vector representation module represents the first vector as (x1, …, xn).

[0274] The second vector representation module represents the second vector as (y1, …, yn).

[0275] The similar user acquisition module comprises a similarity score calculation module, and a calculation formula of the similarity score calculated by the similarity score calculation module is:

[0276]

[0277] wherein cos(θ) represents the similarity score.

[0278] D20, the similar user acquisition system according to D16, the second user acquisition module comprises a similarity calculation module, and a calculation formula for similarity calculation of the similarity calculation module is:

[0279]

[0280] wherein sim(user1, user2) represents the similarity, and the plurality of websites are url1, url2, url3, ……urln respectively. n , pv 11 represents the first click times of the first user clicking url1, pv 21 represents the first click times of the first user clicking url2, and so on; pv 12 represents the second click times of the second user clicking url1, pv 22 represents the second click times of the second user clicking url2, and so on; and n is the total number of the plurality of websites.

Claims

1. A method for obtaining similar URLs, characterized in that, include: Get the first URL; Get the first click count of the first URL under several search terms; Obtain the second URL that was clicked under the aforementioned search terms; Obtain the second number of clicks for each of the several search terms for the second URL; Calculate the behavioral similarity between the first URL and the second URL based on the first number of clicks and the second number of clicks; Based on the behavioral similarity, determine whether the second URL is a similar URL to the first URL; The step of determining whether the second URL is a similar URL to the first URL based on the behavioral similarity includes: Obtain the first title of the website with the first URL, and obtain the second title of the website with the second URL; Calculate the semantic similarity between the first title and the second title; The semantic similarity is fused with the behavioral similarity to obtain a comprehensive similarity. Determine whether the overall similarity is greater than a third threshold. If so, the second URL is a similar URL to the first URL.

2. The method for obtaining similar URLs according to claim 1, characterized in that, The step of obtaining the second URL that is clicked under the search terms also includes: Calculate the number of times the first URL and the second URL co-occur. Determine whether the number of co-occurrences is less than or equal to a first threshold. If so, then the second URL is not a similar URL to the first URL.

3. The method for obtaining similar URLs according to claim 1, characterized in that, The formula for calculating the behavioral similarity is: Where sim(url1,url2) represents the behavior similarity; the search terms are query1, query2, query3, ..., query. n ,pv 11 This represents the number of times the first URL was clicked when searching query1, PV. 21 This indicates the number of times the first URL was clicked when searching query2, and so on; PV 12 This indicates the number of clicks on the second URL when searching query1, PV. 22 This indicates the number of times the second URL is clicked when searching query2, and so on; n is the total number of the search terms.

4. The method for obtaining similar URLs according to claim 1, characterized in that, The step of determining whether the second URL is a similar URL to the first URL based on the behavioral similarity includes: Determine whether the behavioral similarity is greater than a second threshold. If so, the second URL is a similar URL to the first URL.

5. The method for obtaining similar URLs according to claim 1, characterized in that, The step of calculating the semantic similarity between the first title and the second title includes: Extract the first semantic vector of the first title and extract the second semantic vector of the second title; The semantic similarity is calculated using cosine similarity based on the first semantic vector and the second semantic vector.

6. The method for obtaining similar URLs according to claim 5, characterized in that, The first semantic vector is represented as (x1, ..., xn); The second semantic vector is represented as (y1, ..., yn); The formula for calculating semantic similarity is: Wherein, cos(θ) represents the semantic similarity.

7. The method for obtaining similar URLs according to claim 5, characterized in that, The formula for calculating the comprehensive similarity by fusing the semantic similarity with the behavioral similarity is as follows: y=αsimilarAction+βsimilarSentiment Wherein, similarAction represents behavioral similarity, similarSentiment represents semantic similarity, and α and β are two hyperparameters used to control the weight of behavioral similarity and semantic similarity in the similarity between the first URL and the second URL.

8. The method for obtaining similar URLs according to claim 1, characterized in that, The step of determining whether the overall similarity is greater than a third threshold, and if so, then the second URL is a similar URL to the first URL, includes: Obtain the first pageview (PV) of the website with the first URL, and obtain the second pageview (PV) of the website with the second URL; calculate the multiple by which the first PV and the second PV differ. If the overall similarity is greater than the third threshold and the difference between the first PV and the second PV is less than the fourth threshold, then the second URL is a similar URL to the first URL.

9. A system for obtaining similar URLs, characterized in that, include: The first URL retrieval module is used to retrieve the first URL. The first click count acquisition module is used to acquire the first click count of the first URL under several search terms; The second URL acquisition module is used to acquire the second URL that is clicked under the aforementioned search terms; The second click count acquisition module is used to acquire the second click count of the second URL under the several search terms; The behavior similarity calculation module is used to calculate the behavior similarity between the first URL and the second URL based on the first number of clicks and the second number of clicks. The website title retrieval module is used to retrieve the first title of the website with the first URL and the second title of the website with the second URL. A semantic similarity calculation module is used to calculate the semantic similarity between the first title and the second title; The comprehensive similarity calculation module is used to fuse the semantic similarity with the behavioral similarity to obtain a comprehensive similarity. The similar URL judgment module is used to determine whether the overall similarity is greater than a third threshold. If so, the second URL is a similar URL to the first URL.

Images