Webpage resource recommendation method and device, electronic equipment, storage medium and program product
By generating and processing text vectors for web page links, combining Word2Vec and TF-IDF algorithms, the problems of high computing complexity and data sparsity in the existing technology are solved, and efficient and accurate web page resource recommendations are achieved.
Patent Information
- Application Number
- CN202411770970.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-04
- Publication Date
- 2025-03-04
AI Technical Summary
Existing web resource recommendation methods have high computational complexity when processing large-scale data sets, and may have data sparsity and cold start problems, making it difficult to effectively recommend new users or unpopular items.
By obtaining the link text content on the target web page, generating text vectors, and weighted calculations are performed based on the Word2Vec model and the TF-IDF algorithm to obtain a quantitative representation of the link. Then, the target keyword is extracted based on the user's web resource request, the correlation between the keyword and the link text vector is calculated, and the link with the highest correlation is recommended.
It improves the accuracy and efficiency of web resource recommendations, can effectively handle large-scale data sets, reduces data sparseness and cold start problems, and improves the recommendation effect of new users and unpopular items.
Smart Images

Figure CN119557515A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of next generation Internet technology, and more specifically, to a webpage resource recommendation method, device, electronic device, storage medium and program product. Background Art
[0002] The adoption of IPv6 technology provides a wider address space for the network transmission of educational resources, solves the problem of insufficient IPv4 address resources, and makes it possible for each device to be uniquely identified, thereby improving the scalability and security of the network. While ensuring the smooth flow of information on the education network and providing students and teachers with a richer and more efficient experience in obtaining educational resources, we will build an education network IPv6 feature resource recommendation website to improve the accessibility of educational resources, promote the sharing and use of educational resources, and promote the process of educational informatization and modernization.
[0003] There are three types of common resource recommendations: First, content-based recommendation is a very direct recommendation method. This algorithm has high requirements for the accuracy and completeness of the description of item features, and the recommendation effect for new users or unpopular items may not be good. Second, collaborative filtering recommendation is to discover the user's interest preferences by analyzing the user's behavior data, and make recommendations based on the preference information of other users. However, for large-scale data sets, the computational complexity of collaborative filtering algorithms is high, and there may be data sparsity and cold start problems. Third, knowledge-based recommendation is a recommendation method based on logical reasoning technology. It provides personalized recommendation services for users by analyzing their needs and preferences. It provides users with recommended content that meets their needs through reasoning and rule matching. However, this algorithm has high requirements for domain knowledge and rules. Summary of the invention
[0004] In view of this, the present disclosure provides a web resource recommendation method, device, electronic device, storage medium and program product.
[0005] One aspect of the present disclosure provides a webpage resource recommendation method, comprising:
[0006] Acquire the text content of at least one link existing on the target webpage, and generate a text vector of the text content of each link, wherein each element value in the text vector represents a keyword included in the text content;
[0007] Performing weighted calculation based on the element value of each keyword in the text vector to obtain a quantitative representation of the link;
[0008] Obtaining a webpage resource request from a user, and extracting a target keyword from the webpage resource request;
[0009] The relevance between the target keyword and the text vector of each link is obtained based on the quantitative representation of the link, and the link with the highest relevance is recommended to the user.
[0010] According to an embodiment of the present disclosure, obtaining the text content of at least one link existing on the target webpage includes:
[0011] Collecting web page data of the target web page based on a preset crawler module;
[0012] The web page data is preprocessed to obtain text content of at least one link on the target web page.
[0013] According to an embodiment of the present disclosure, generating a text vector of the text content of each link includes:
[0014] The text content of each link is converted into a text vector based on a preset Word2Vec model;
[0015] The keywords included in the text content are given weights through the TF-IDF algorithm.
[0016] According to an embodiment of the present disclosure, obtaining the relevance between the target keyword and the text vector of each link based on the quantified representation of the link, and recommending the link with the highest relevance to the user includes:
[0017] Calculate the weight of the target keyword and the vector value corresponding to each of the links based on the TF-IDF algorithm;
[0018] Calculating a representation value of each of the links corresponding to the target keyword based on the weight and the vector value of each of the links corresponding to the target keyword;
[0019] The correlation between the representative value and the quantitative representation of each link is calculated, and the link with the highest correlation is selected for recommendation to the user.
[0020] According to an embodiment of the present disclosure, the method includes:
[0021] New webpage text content is obtained, and incremental training is performed on the Word2Vec model based on the new webpage text content to improve the accuracy of the Word2Vec model.
[0022] According to an embodiment of the present disclosure, the acquiring new webpage text content and performing incremental training on the Word2Vec model based on the new webpage text content includes:
[0023] Acquire new webpage text content, extract words from the webpage text content and add them to the corpus of the Word2Vec model;
[0024] The web page text content is counted as sentences into the Word2Vec model for vector calculation training, wherein the words in the web page text content are used as part of the vector calculation training.
[0025] Another aspect of the present disclosure provides a webpage resource recommendation device, comprising:
[0026] A content acquisition module, used to acquire the text content of at least one link existing on the target web page, and generate a text vector of the text content of each link, wherein each element value in the text vector represents a keyword included in the text content;
[0027] A quantitative calculation module, used for performing weighted calculation based on the element value of each keyword in the text vector to obtain a quantitative representation of the link;
[0028] A keyword extraction module, used to obtain a user's web resource request and extract target keywords from the web resource request;
[0029] The association recommendation module is used to calculate the association degree between the target keyword and the text vector of each link based on the quantitative representation of the link, and recommend the link with the highest association degree to the user.
[0030] Another aspect of the present disclosure provides an electronic device, comprising:
[0031] one or more processors;
[0032] a memory for storing one or more programs,
[0033] When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described above.
[0034] Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the above method when executed.
[0035] Another aspect of the present disclosure provides a computer program product, which includes computer executable instructions, and when the instructions are executed, are used to implement the method as described above. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:
[0037] Figure 1 A schematic diagram of obtaining web page data to which the present disclosure can be applied is schematically shown;
[0038] Figure 2 A flowchart of a webpage resource recommendation method according to an embodiment of the present disclosure is schematically shown;
[0039] Figure 3 A block diagram schematically shows a webpage resource recommendation device according to an embodiment of the present disclosure; and
[0040] Figure 4 A block diagram of an electronic device 400 suitable for implementing a robot according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION
[0041] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.
[0042] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.
[0043] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.
[0044] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).
[0045] In the embodiments of the present disclosure, the collection, updating, analysis, processing, use, transmission, provision, disclosure, storage, etc. of the data involved (for example, including but not limited to user personal information) are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. In particular, necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security, and national security.
[0046] Figure 1 The schematic diagram of obtaining web page data to which the present disclosure can be applied is schematically shown.
[0047] like Figure 1 As shown, in the web content recommendation project, it is necessary to obtain the links and content in the web page and display them to the user. As shown in the figure, when probe 1 crawls the internal links and content from the website, it needs to bypass the website's anti-crawling mechanism. However, in the initial situation, for a new website, the crawler program first tries to use the crawler framework to perform a reverse analysis of the website, and then assigns the crawling task to probe 2. Then, based on the headless browser and packet capture tool, a targeted anti-crawling breakthrough strategy is developed. The interactive crawler technology route is tried, which can simulate link clicks for crawling and successfully crawl the website's internal links. At this time, the dispatch center can receive the website content and complete internal link links of the website. The description of the crawler process here is to facilitate the description of the principle of the invention later.
[0048] Figure 2 The flowchart of the web page resource recommendation method according to the embodiment of the present disclosure is schematically shown.
[0049] like Figure 2 As shown, the method includes operations S210 to S240.
[0050] S210, obtaining text content of at least one link existing on the target web page, generating text vectors of the text content of each link, wherein each element value in the text vector represents a keyword included in the text content.
[0051] S210 includes S211~S212.
[0052] S211, collecting web page data of the target web page based on a preset crawler module.
[0053] S212: pre-process the web page data to obtain text content of at least one link on the target web page.
[0054] Suppose a website on the education network: the homepage address is http: / / www.xxx.edu.cn.
[0055] The corresponding relationship table includes: http: / / one.xxx.edu.cn, http: / / two.xxx.edu.cn, http: / / three.xxx.edu.cn, that is, the website link has three sub-website links in addition to the homepage.
[0056] The purpose of recommending high-quality resources is to search for corpus problems in the education network, find recommended new words in the corpus through the Word2Vec model, and recommend the above links according to the corresponding relationship and relevance, such as: http: / / two.xxx.edu.cn, http: / / one.xxx.edu.cn, http: / / three.xxx.edu.cn.
[0057] S210 also includes S213~S214.
[0058] S213, converting the text content of each link into a text vector based on a preset Word2Vec model.
[0059] S214, assigning weights to the keywords included in the text content through the TF-IDF algorithm.
[0060] In the disclosed embodiment, the relevant web page data collected by the homepage crawler cannot be used as the original corpus for information discovery. It is necessary to effectively clean the data, organize the data format, eliminate data anomalies and data redundancy. And complete the preprocessing task according to the information discovery model, and organize the available corpus data. How to convert text into a computable vector is the key in natural language processing. The Word2Vec process is essentially a neural network including a hidden layer, and what is finally obtained is the weight of a neural network. Each word will be treated as an n-dimensional vector, and each word and the expected adjacent words will be used as the input of the neural network, and the output node represents the vector corresponding to this word. For a word term in the text content, its context is context, and the final goal is to construct a mapping f(term,context), use term and context as model input, and finally obtain the vector corresponding to the word.
[0061] The web page content linked by the website is taken as input, the text content in each linked web page is taken as a document, and the web page is abstracted into a multi-dimensional vector. The vector is calculated by assigning keyword weights by TF-IDF, and each word has a corresponding weight.
[0062] S220, performing weighted calculation based on the element value of each keyword in the text vector to obtain a quantitative representation of the link.
[0063] Different words affect the quantitative representation of the web page, that is, the vector of the web page, according to their weight in the web page. This value can be used as the quantitative representation of the web page corresponding to the keyword, and also represents the correspondence between the corpus and the link address. For example: establish a mapping relationship between the obtained keywords and links: YY-http: / / two.xxx.edu.cn: 90%, YY-http: / / one.xxx.edu.cn: 80%, YY-http: / / three.xxx.edu.cn: 70%; ZZ-http: / / three.xxx.edu.cn: 75%, ZZ-http: / / one.xxx.edu.cn: 65%, ZZ-http: / / two.xxx.edu.cn: 50%.
[0064] In the process of applying the Word2Vec model, the Word2Vec model can also be incrementally trained. For example, new webpage text content is obtained, and the Word2Vec model is incrementally trained based on the new webpage text content to improve the accuracy of the Word2Vec model.
[0065] Incremental training of the Word2Vec model based on new web page text content includes S221~S222.
[0066] S221, obtaining new web page text content, extracting words from the web page text content and adding them to the corpus of the Word2Vec model.
[0067] S222, counting the web page text content in units of sentences into the Word2Vec model for vector calculation training, wherein the words in the web page text content are used as part of the vector calculation training.
[0068] First, the words that appear in the new corpus but not in the old corpus need to be added to the entire corpus; then the new corpus is added to the Word2Vec training model as a training input in units of sentences, so that in the web pages containing new words, the new word information will also be added to the corpus vector value as part of the vector calculation, so that the corpus can contain new word information and improve the accuracy of the model. Fine-tune the model on the trained Word2Vec model. In order to better recommend resources for Chinese, the corpus needs to be expanded for some special characters or languages. Use other methods such as Prompt-Tuning without adjusting parameters, only changing the information of the input data.
[0069] S230, obtaining a webpage resource request from a user, and extracting a target keyword from the webpage resource request.
[0070] S240, obtaining the relevance between the target keyword and the text vector of each link based on the quantitative representation of the link, and recommending the link with the highest relevance to the user.
[0071] S240 includes S241~S243.
[0072] S241, calculating the weight of the target keyword and the vector value of each corresponding link based on the TF-IDF algorithm.
[0073] S242: Calculate the representation value of each link corresponding to the target keyword based on the weight and the vector value of each link corresponding to the target keyword.
[0074] S243, calculating the correlation between the representation value and the quantitative representation of each link, and selecting the link with the highest correlation to recommend to the user.
[0075] For example, the dispatch center receives a user request: "Recommend something about YY". The backend uses the word2vec model to convert the request into a corresponding relationship showing the corpus YY, and sends the relevance to the front-end page in order from high to low: 1) http: / / two.xxx.edu.cn, 2) http: / / one.xxx.edu.cn, 3) http: / / three.xxx.edu.cn. At this point, the server successfully provides high-quality resource recommendation services to the outside world.
[0076] The disclosed embodiment proposes a web resource recommendation method, which uses a large artificial intelligence model to automatically process website data, classify documents, etc., significantly improving work efficiency and productivity; it can provide personalized recommendations and services based on user interests and behavioral habits to enhance user experience. AI is integrated with high-quality resources to provide value delivery services for network access users, increase user stickiness, enhance user dependence and trust in the platform, improve user loyalty and retention, and form a good word-of-mouth effect. In this method, knowledge-based recommendations and collaborative filtering recommendations are combined, which can give full play to the advantages of the two methods and improve the accuracy and diversity of recommendations.
[0077] Figure 3 The block diagram of the webpage resource recommendation device according to an embodiment of the present disclosure is schematically shown.
[0078] like Figure 3 As shown, the webpage resource recommendation device 300 includes a content acquisition module 310 , a quantitative calculation module 320 , a keyword extraction module 330 and an associated recommendation module 340 .
[0079] The content acquisition module 310 is used to acquire the text content of at least one link existing on the target webpage, and generate a text vector of the text content of each link, wherein each element value in the text vector represents a keyword included in the text content.
[0080] The quantitative calculation module 320 is used to perform weighted calculation based on the element value of each keyword in the text vector to obtain a quantitative representation of the link.
[0081] The keyword extraction module 330 is used to obtain the user's web resource request and extract the target keyword from the web resource request.
[0082] The association recommendation module 340 is used to calculate the association between the target keyword and the text vector of each link based on the quantitative representation of the link, and recommend the link with the highest association to the user.
[0083] According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits, or at least part of the functions of any one of them can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be at least partially implemented as hardware circuits, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems on chips, systems on substrates, systems on packages, application specific integrated circuits (ASICs), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging circuits, or can be implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules, submodules, units, and subunits can be at least partially implemented as computer program modules, and when the computer program modules are run, the corresponding functions can be performed.
[0084] For example, any multiple of the content acquisition module 310, the quantitative calculation module 320, the keyword extraction module 330, and the association recommendation module 340 can be combined in one module / unit / sub-unit for implementation, or any one of the modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functions of one or more of these modules / units / sub-units can be combined with at least part of the functions of other modules / units / sub-units and implemented in one module / unit / sub-unit. According to an embodiment of the present disclosure, at least one of the content acquisition module 310, the quantitative calculation module 320, the keyword extraction module 330, and the association recommendation module 340 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, at least one of the content acquisition module 310 , the quantitative calculation module 320 , the keyword extraction module 330 and the associated recommendation module 340 may be at least partially implemented as a computer program module, which may perform a corresponding function when executed.
[0085] It should be noted that the web resource recommendation device part in the embodiment of the present disclosure corresponds to the web resource recommendation method part of the present disclosure. The description of the web resource recommendation device part specifically refers to the web resource recommendation method part, which will not be repeated here.
[0086] Figure 4 A block diagram of an electronic device suitable for implementing the method described above according to an embodiment of the present disclosure is schematically shown. Figure 4 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0087] like Figure 4 As shown, the electronic device 400 according to an embodiment of the present disclosure includes a processor 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage part 408 to a random access memory (RAM) 403. The processor 401 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 401 may also include an onboard memory for caching purposes. The processor 401 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.
[0088] In RAM 403, various programs and data required for the operation of electronic device 400 are stored. Processor 401, ROM 402 and RAM 403 are connected to each other via bus 404. Processor 401 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 402 and / or RAM 403. It should be noted that the program can also be stored in one or more memories other than ROM 402 and RAM 403. Processor 401 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.
[0089] According to an embodiment of the present disclosure, the electronic device 400 may further include an input / output (I / O) interface 405, which is also connected to the bus 404. The electronic device 400 may further include one or more of the following components connected to the input / output (I / O) interface 405: an input portion 406 including a keyboard, a mouse, etc.; an output portion 407 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 408 including a hard disk, etc.; and a communication portion 409 including a network interface card such as a LAN card, a modem, etc. The communication portion 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the input / output (I / O) interface 405 as needed. A removable medium 411, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 410 as needed, so that a computer program read therefrom is installed into the storage portion 408 as needed.
[0090] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 409, and / or installed from the removable medium 411. When the computer program is executed by the processor 401, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.
[0091] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.
[0092] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device.
[0093] For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 402 and / or the RAM 403 described above and / or one or more memories other than the ROM 402 and the RAM 403 .
[0094] The embodiments of the present disclosure also include a computer program product, which includes a computer program, which contains program code for executing the method provided by the embodiments of the present disclosure. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the web resource recommendation method provided by the embodiments of the present disclosure.
[0095] When the computer program is executed by the processor 401, the above functions defined in the system / device of the embodiment of the present disclosure are executed. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.
[0096] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 409, and / or installed from the removable medium 411. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.
[0097] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).
[0098] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments of the present disclosure can be combined and / or combined in a variety of ways, even if such a combination or combination is not explicitly recorded in the present disclosure. In particular, without departing from the spirit and teaching of the present disclosure, the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.
[0099] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.
Claims
1. A webpage resource recommendation method, comprising: Acquire the text content of at least one link existing on the target webpage, and generate a text vector of the text content of each link, wherein each element value in the text vector represents a keyword included in the text content; Performing weighted calculation based on the element value of each keyword in the text vector to obtain a quantitative representation of the link; Obtaining a webpage resource request from a user, and extracting a target keyword from the webpage resource request; The relevance between the target keyword and the text vector of each link is obtained based on the quantitative representation of the link, and the link with the highest relevance is recommended to the user.
2. The method according to claim 1, wherein: The step of obtaining the text content of at least one link on the target webpage includes: Collecting web page data of the target web page based on a preset crawler module; The web page data is preprocessed to obtain text content of at least one link on the target web page.
3. The method according to claim 1, wherein: The generating of the text vector of the text content of each link comprises: The text content of each link is converted into a text vector based on a preset Word2Vec model; The keywords included in the text content are given weights through the TF-IDF algorithm.
4. The method according to claim 1, wherein: The obtaining the relevance between the target keyword and the text vector of each link based on the quantized representation of the link, and recommending the link with the highest relevance to the user comprises: Calculate the weight of the target keyword and the vector value corresponding to each of the links based on the TF-IDF algorithm; Calculating a representation value of each of the links corresponding to the target keyword based on the weight and the vector value of each of the links corresponding to the target keyword; The correlation between the representative value and the quantitative representation of each link is calculated, and the link with the highest correlation is selected for recommendation to the user.
5. The method according to claim 3, wherein: The method comprises: New webpage text content is obtained, and incremental training is performed on the Word2Vec model based on the new webpage text content to improve the accuracy of the Word2Vec model.
6. The method according to claim 3, wherein: The acquiring new webpage text content and performing incremental training on the Word2Vec model based on the new webpage text content comprises: Acquire new webpage text content, extract words from the webpage text content and add them to the corpus of the Word2Vec model; The web page text content is counted as sentences into the Word2Vec model for vector calculation training, wherein the words in the web page text content are used as part of the vector calculation training.
7. A web resource recommendation device, comprising: A content acquisition module, used to acquire the text content of at least one link existing on the target web page, and generate a text vector of the text content of each link, wherein each element value in the text vector represents a keyword included in the text content; A quantitative calculation module, used for performing weighted calculation based on the element value of each keyword in the text vector to obtain a quantitative representation of the link; A keyword extraction module, used to obtain a user's web resource request and extract target keywords from the web resource request; The association recommendation module is used to calculate the association degree between the target keyword and the text vector of each link based on the quantitative representation of the link, and recommend the link with the highest association degree to the user.
8. An electronic device comprising: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors execute the method according to any one of claims 1 to 6.
9. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the method according to any one of claims 1 to 6.
10. A computer program product, comprising computer executable instructions, wherein the instructions are used to implement the method according to any one of claims 1 to 6 when executed.