Multimodal Web Information Retrieval Static Ranking Learning Methods, Systems, Devices, and Media

Abstract

This invention discloses a multimodal Web information retrieval static ranking learning method, system, device, and medium. The method includes acquiring a screenshot and information of a target webpage, and obtaining webpage text information, HTML text, HTML tags, and webpage statistical feature information based on the screenshot and information; mapping and extracting features from the webpage text information, HTML text information, HTML tag information, and webpage statistical feature information respectively, obtaining text feature vectors, HTML tag feature vectors, HTML text feature vectors, and webpage statistical feature vectors; combining the four feature vectors to obtain a combined feature vector; and evaluating and classifying the webpage based on the combined feature vector to generate a predicted evaluation level for the webpage. This invention explores the inherent characteristics of webpage information in depth from multiple dimensions, enabling a more accurate, objective, and reasonable evaluation of the Web content quality of target webpages, improving evaluation accuracy, and making it less susceptible to various cheating methods.

Description

Technical Field

[0001] This invention belongs to the field of web information retrieval technology, specifically relating to a multimodal web information retrieval static ranking learning method, system, device, and medium. Background Technology

[0002] Web information refers to the vast amount of web pages and other related content existing on the Internet, including websites, blogs, social media posts, news articles, and so on. Web information not only contains various types of high-quality information, but also various types of false, deceptive, cheating, and abusive information. Therefore, given the abundance and uneven quality of online content, identifying high-quality online content has become increasingly important and urgent.

[0003] Static ranking learning for Web Information Retrieval, also known as Web Content Quality Evaluation, Query Independent Ranking, or Static Ranking, is used to evaluate the quality of Web content and categorize or rank online content of varying quality. Static ranking learning is a core and fundamental algorithm for search engines, recommendation systems, and intelligent dialogue services. For search engines, static ranking is a core algorithm; its evaluation results serve as a reference for the "importance" of search ranking and can also be used as an important basis for guiding the crawling of massive amounts of web pages.

[0004] Current research on static ranking in web information retrieval has yielded some results and applied various techniques, such as PageRank, TrustRank, Truncated PageRank, and other link analysis algorithms, effectively suppressing link manipulation. However, there is a lack of effective solutions to simultaneously address various forms of cheating, including hidden manipulation, embedding manipulation, reclassification manipulation, and cloaking. Meanwhile, with the increasing diversification of internet content formats and the continuous expansion of data scale, especially with the rampant abuse driven by profit, such as web cheating and false advertising, the technical difficulty of efficiently identifying web content of different qualities is growing. At the same time, the demand for high-quality information from internet users and large models like LLMs is becoming increasingly strong. Therefore, mainstream static ranking methods for web information retrieval are easily affected by abuse, resulting in lower-than-expected ranking accuracy, which presents new challenges to existing static ranking methods. Summary of the Invention

[0005] The main objective of this invention is to overcome the shortcomings of existing technologies, such as low ranking accuracy and susceptibility to various cheating methods, and to propose a multimodal Web information retrieval static ranking learning method, system, device, and medium.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A static ranking learning method for multimodal Web information retrieval includes the following steps:

[0008] Obtain a screenshot of the target webpage, extract the webpage text information based on the screenshot, and map the webpage text information into a first text vector sequence;

[0009] Obtain the HTML text of the target webpage and map the HTML text information into a first HTML text vector sequence;

[0010] Obtain the HTML tags of the target webpage and map the HTML tag information to the first HTML tag vector sequence;

[0011] Based on the web page statistical features of the target web page, the web page statistical feature information is mapped into a first web page statistical feature vector sequence; the web page statistical features include link analysis features, content heuristic features, website affiliation features, and time series features;

[0012] Based on one-hot encoding and word embedding vector pattern, the second text feature vector matrix, the second HTML text feature vector matrix, and the second HTML tag feature vector matrix are obtained according to the first text vector sequence, the first HTML text vector sequence, and the first HTML tag vector sequence. The Transformer encoder model is used to extract features from the second text feature vector matrix, the second HTML text feature vector matrix, and the second HTML tag feature vector matrix to obtain text feature vectors, HTML text feature vectors, and HTML tag feature vectors.

[0013] Based on the feature combination and concatenation model, the statistical feature vector sequence of the first webpage is concatenated to obtain the statistical feature vector matrix of the second webpage; the DNN model is used to extract features from the statistical feature vector matrix of the second webpage to obtain the webpage statistical feature vector.

[0014] The text feature vector, HTML text feature vector, HTML tag feature vector and web page statistical feature vector are combined to obtain the combined feature vector.

[0015] The evaluation and grading are performed based on the combined feature vectors, generating a predicted evaluation grade for the webpage.

[0016] This invention also includes a multimodal Web information retrieval static ranking learning system. The system adopts the multimodal Web information retrieval static ranking learning method provided by this invention. The system includes an information extraction module, a feature extraction module, a feature combination module, and a quality evaluation module.

[0017] The information extraction module is used to obtain screenshots and information of the target webpage and to obtain webpage text information, HTML text, HTML tags, and webpage statistical feature information based on the screenshots and information.

[0018] The feature extraction module is used to map web page text information, HTML text information, HTML tag information, and web page statistical feature information into a first text vector sequence, a first HTML text vector sequence, a first HTML tag vector sequence, and a first web page statistical feature vector sequence, respectively; it is also used to extract features from the first text vector sequence, the first HTML text vector sequence, the first HTML tag vector sequence, and the first web page statistical feature vector sequence, respectively, to obtain text feature vectors, HTML text feature vectors, HTML tag feature vectors, and web page statistical feature vectors, respectively;

[0019] The feature combination module is used to combine text feature vectors, HTML text feature vectors, HTML tag feature vectors, and web page statistical feature vectors to obtain a combined feature vector.

[0020] The quality evaluation module is used to evaluate and classify web pages based on combined feature vectors, and generate a predicted evaluation level for the web page.

[0021] The feature extraction module includes a DNN model and a Transformer encoder model;

[0022] The feature combination module has a fully connected layer. Through the fully connected layer, the text feature vector with text context feature information, the HTML text feature vector with HTML text context feature information, the HTML tag feature vector with HTML tag context feature information, and the web page statistical feature vector with web page statistical feature information are combined to calculate the best fusion method and obtain feature results suitable for classification analysis.

[0023] The quality evaluation module is equipped with a DNN model, which evaluates and classifies the quality of Web content based on combined feature vectors to determine the predicted evaluation level of the current webpage.

[0024] The present invention also includes a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the multimodal Web information retrieval static ranking learning method provided by the present invention.

[0025] The present invention also includes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the multimodal Web information retrieval static ranking learning method provided by the present invention.

[0026] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0027] 1. This invention comprehensively considers multimodal factors. In addition to link analysis features, content heuristic features, webpage attribution features, and temporal features, it further includes two different modal semantic information: text content information from the machine's perspective and HTML tag information, and webpage screenshot OCR text information from the user's perspective. It mines the above-mentioned multi-perspective identification information through a deep learning point-wise architecture, realizing a comprehensive consideration of various cheating factors such as link cheating, hiding cheating, embedding cheating, redirect cheating, and cloaking, while taking into account website content, attribution, and temporal characteristics. Compared with existing technologies, it explores the inherent characteristics of webpage information more deeply from multiple dimensions, and evaluates the Web content quality of target webpages more accurately, objectively, and reasonably, thereby improving the evaluation accuracy. Attached Figure Description

[0028] Figure 1 This is a flowchart of the method of the present invention;

[0029] Figure 2 This is a schematic diagram of the method of the present invention;

[0030] Figure 3 This is a schematic diagram of the text feature vector transformation in the embodiment;

[0031] Figure 4 This is a schematic diagram of the HTML text feature vector transformation in the embodiment;

[0032] Figure 5 This is a schematic diagram of the transformation of HTML tag feature vectors in the embodiment;

[0033] Figure 6 This is a schematic diagram of the transformation of the statistical feature vector of the webpage in the embodiment;

[0034] Figure 7 This is a schematic diagram of the feature extraction process using the Transformer encoder model in the embodiment;

[0035] Figure 8 This is a schematic diagram of the system of the present invention. Detailed Implementation

[0036] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0037] Example

[0038] like Figure 1 and Figure 2 As shown, this invention provides a static ranking learning method for multimodal Web information retrieval, comprising the following steps:

[0039] Obtain a screenshot of the target webpage, extract the webpage text information based on the screenshot, and map the webpage text information into a first text vector sequence, specifically:

[0040] The retrieved webpage text information is defined as input data x. text And treat it as a text sequence, x text It will be divided into a series of discrete text units, namely x text ={t1,t2,t3,…,t L}, where L is the original length of the input data, t i x represents text Each discrete text unit in the text; by setting a fixed length of L c After truncation or padding, a new sequence of character elements is obtained. That is, the first text vector sequence, where express Each discrete text unit in the text.

[0041] Obtain the HTML text of the target webpage, and map the HTML text information into a first HTML text vector sequence, specifically:

[0042] The obtained HTML text information of the webpage is defined as input data x. htmlText And treat it as a sequence of HTML text, x htmlText It is divided into a series of discrete HTML text units, namely x htmlText ={h1,h2,h3,…,h hL}, where hL is the original length of the input data, h i x represents htmlText Each discrete HTML text unit in the document; by setting a fixed length of L h After truncation or padding, a new sequence of HTML text elements is obtained. That is, the first HTML text vector sequence, where express Each discrete HTML text unit in the text.

[0043] Obtain the HTML tags of the target webpage, and map the HTML tag information to a first HTML tag vector sequence, specifically:

[0044] The obtained HTML tag information of the webpage is defined as input data x. htmlTag And treat it as a sequence of HTML tags, x htmlTag It will be divided into a series of discrete HTML tag units, namely x htmlTag ={g1,g2,g3,…,g gL}, where gL is the original length of the input data, g i x represents htmlTag Each discrete HTML tag unit in the text; by setting a fixed length of L g After truncation or padding, a new sequence of HTML tag elements is obtained. That is, the first HTML tag vector sequence, where express Each discrete HTML tag unit in the text.

[0045] In this embodiment, a screenshot of the target webpage is obtained, and the webpage text information is obtained based on the screenshot, including:

[0046] Webpage screenshots are collected based on link addresses (URLs); text box detection is performed on the webpage screenshots to obtain all text box areas in the webpage screenshots; image recognition algorithms are used to perform text recognition on the webpage screenshots based on the text box areas to obtain webpage text information with sequence.

[0047] In this embodiment, by making full use of the visual information in the webpage and from the perspective of the webpage user, technologies such as OCR can be used to bypass various cheating methods to a certain extent and directly extract real and visible webpage text information and webpage image information.

[0048] In some embodiments, a screenshot of the target webpage is obtained through strategies such as delaying and waiting, and this screenshot is the image information required later.

[0049] In some embodiments, for a webpage screenshot, a webpage text detection algorithm is used to detect all regions containing text in the webpage screenshot, and these regions are represented in the form of rectangular coordinates [x1, y1, x2, y2, x3, y3, x4, y4]. The x and y coordinates of the four points at the bottom left, top left, bottom right, and top right of the text box are used to form a matrix called the text box.

[0050] In some embodiments, an OCR algorithm is used as an image recognition algorithm.

[0051] In some embodiments, image processing functions from the OpenCV library are used as image recognition algorithms.

[0052] In some embodiments, to ensure the accuracy of text recognition, text boxes with excessively small areas are filtered out. Then, relevant image processing functions from the OpenCV library are used to sequentially extract text box regions from the webpage screenshot based on their coordinates and store them in memory. If a text box region has a tilt angle, it is horizontalized using affine transformation to improve recognition accuracy. After obtaining all text box regions from the webpage screenshot, multiple text box regions are extracted from memory in parallel using multi-threading. These regions are then input into an algorithm model (such as YOLOv4), and the corresponding text is recognized based on text features. After completing text recognition for all text box regions, the recognized text is reassembled sequentially into complete webpage text, thus extracting the webpage text information.

[0053] Based on the web page statistical features of the target web page, the web page statistical feature information is mapped into a first web page statistical feature vector sequence; the web page statistical features include link analysis features, content heuristic features, website affiliation features, and time series features;

[0054] Based on one-hot encoding and word embedding vector pattern, the second text feature vector matrix, the second HTML text feature vector matrix, and the second HTML tag feature vector matrix are obtained according to the first text vector sequence, the first HTML text vector sequence, and the first HTML tag vector sequence. The Transformer encoder model is used to extract features from the second text feature vector matrix, the second HTML text feature vector matrix, and the second HTML tag feature vector matrix to obtain text feature vectors, HTML text feature vectors, and HTML tag feature vectors.

[0055] In this embodiment, as Figure 3 , Figure 4 and Figure 5 As shown, specifically:

[0056] Based on one-hot encoding and word embedding vector patterns, for each discrete text unit Perform a mapping, mapping it to a d-dimensional vector space representation. Final first text vector sequence Mapping to a vector matrix Matrix M text This is the second text feature vector matrix generated by feature construction; specifically, the value of d is 100.

[0057] Based on one-hot encoding and word embedding vector patterns, each discrete HTML text unit... Perform a mapping, mapping it to a d-dimensional vector space representation. Final first HTML text vector sequence Mapping to a vector matrix Matrix M htmlText This is the second HTML text feature vector matrix generated by feature construction;

[0058] Based on one-hot encoding and word embedding vector patterns, for each discrete HTML tag unit Perform a mapping, mapping it to a d-dimensional vector space representation. Final first HTML tag vector sequence Mapping to a vector matrix Matrix M htmlTag This is the second HTML tag feature vector matrix generated by feature construction.

[0059] like Figure 7 As shown, the Transformer encoder model is used to extract features from the second text feature vector matrix, the second HTML text feature vector matrix, and the second HTML tag feature vector matrix. Specifically:

[0060] The Transformer encoder model includes several encoders, which include a multi-head self-attention mechanism, a feedforward layer, and layer normalization.

[0061] The process of feature extraction using the Transformer encoder model is as follows:

[0062] Position embedding is performed on the feature vector matrix to obtain a feature vector matrix with embedded position information. The Transformer encoder model cannot utilize the order information between words. In this embodiment, position embedding is introduced before text feature extraction to record the position information of words in the sequence, so that the Transformer encoder model can distinguish words in different positions and thus obtain the ability to capture order information.

[0063] The feature vector matrix with embedded location information is passed sequentially through the multi-head self-attention mechanism, feedforward layer and layer normalization in each encoder to finally obtain the feature vector;

[0064] The feature extraction process for the second text feature vector matrix is ​​expressed as follows:

[0065]

[0066] Among them, h i Let represent the vector matrix output by the i-th encoder, and l represent the number of encoders in the Transformer encoder model; The second text feature vector matrix represents the embedded location information;

[0067] The feature extraction process for the second HTML text feature vector matrix is ​​expressed as follows:

[0068]

[0069] in, The second HTML text feature vector matrix represents the embedded location information;

[0070] The feature extraction process for the second HTML tag feature vector matrix is ​​expressed as follows:

[0071]

[0072] in, The second HTML tag feature vector matrix represents the embedded location information;

[0073]

[0074] Among them, h i Let represent the vector matrix output by the i-th encoder, and l represent the number of encoders in the Transformer encoder model; Represents the word vector matrix;

[0075] The location embedding process is as follows:

[0076]

[0077]

[0078]

[0079]

[0080] Where pos represents the position index of the word, i represents the dimension index of the word, and M... peText M peHtml and M peHtmlTag These represent the position information matrices of the second text feature vector matrix, the second HTML text feature vector matrix, and the second HTML tag feature vector matrix, respectively. Their dimensions are respectively the same as those of the second text feature vector matrix M. text The second HTML text feature vector matrix M html And the dimensions of the second HTML tag feature vector matrix are equal; + indicates matrix addition operation;

[0081] Self-attention mechanism is also known as scaling dot product attention mechanism. Multi-head self-attention mechanism repeats the calculation process of self-attention mechanism h times, and then concatenates the calculation results; the process expression of multi-head self-attention mechanism is:

[0082]

[0083] multiHead(Q,K,V)=Concat(head1,…,head h )

[0084]

[0085]

[0086]

[0087]

[0088] Among them, head i This represents the calculation result of the i-th self-attention mechanism, where h represents the number of repetitions of the self-attention mechanism, and the value of h is 10; MultiHead(Q,K,V) represents the calculation result of the multi-head self-attention mechanism. These represent different linear transformation matrices; These represent the query, key, and value, respectively; d k express The number of columns in a matrix, i.e., the vector dimension;

[0089] In any encoder, after passing through a multi-head self-attention mechanism, a vector matrix A is obtained, which is then input into a feedforward layer. This layer consists of multiple fully connected layers and is responsible for performing vector matrix mapping and affine transformation on the vector matrix A. After residual connections and layer normalization, the output vector matrix h of the encoder module is obtained. i The output vector matrix is ​​then used as the input to the next encoder module; the output h of the last encoder... l This refers to text feature vectors, HTML text feature vectors, or HTML tag feature vectors.

[0090] Based on the feature combination and concatenation model, the first webpage statistical feature vector sequence is concatenated to obtain the second webpage statistical feature vector matrix. Webpage statistical feature information is defined as input data and treated as a webpage statistical feature sequence, which is a one-dimensional numerical sequence that also includes link analysis features x. link (TrustRank, SpamRank, Truncated PageRank, etc.), content heuristic features x content (Keyword coverage, text length, etc.), webpage attribution characteristics x attribution (Domain holder, IP address, etc.) and time-series characteristics x time ;

[0091] like Figure 6 As shown, the above four features are sequentially concatenated and combined to generate a feature vector sequence x. statistics ={x link ;x content ;x attribution ;x time}; Perform a dimensionality transformation on the feature vector sequence to generate the feature vector matrix M. statistics ={M link M content M attribution M time};

[0092] Where M statistics ∈R L*1 L represents the total number of statistical features of the webpage, 1 indicates that each element in the generated feature vector matrix is ​​a 1-dimensional vector, and M represents the total number of statistical features of the webpage. statistics The matrix is ​​the second webpage statistical feature vector matrix generated by feature construction.

[0093] A DNN model is used to extract features from the statistical feature vector matrix of the second webpage to obtain the webpage statistical feature vector; the DNN model includes several fully connected networks.

[0094] The DNN model is used to extract features from the statistical feature vector matrix of the second webpage, specifically as follows:

[0095] The second webpage statistical feature vector matrix is ​​sequentially processed through each fully connected network to perform matrix operations, ultimately yielding the webpage statistical feature vector. The process expression is as follows:

[0096]

[0097] Among them, M i M represents the vector matrix output by the i-th fully connected network, where l represents the number of fully connected networks in the DNN model; statistics The second webpage's statistical feature vector matrix represents the weight matrix of the fully connected network. L i b is the number of neurons in the i-th fully connected network; i M represents the bias term of the neuron in the i-th fully connected network; the output M of the last fully connected network in the DNN model. i This is the statistical feature vector of the webpage.

[0098] The text feature vector, HTML text feature vector, HTML tag feature vector, and webpage statistical feature vector are combined to obtain a combined feature vector, specifically:

[0099] Using vector concatenation operations, the text feature vector V is converted into a single vector. textHTML text feature vector V htmlText HTML tag feature vector V htmlTag and webpage statistical feature vector V statistics By sequentially concatenating and combining the features, we obtain the concatenated feature vector V. c =[V text V htmlText V htmlTag V statistics ];

[0100] A single fully connected layer is used to perform feature combination operations on the concatenated feature vectors. The purpose of this method is to reduce the dimensionality of features, eliminate redundant variables, and find the intrinsic relationships between feature vectors. The expression for this is:

[0101] V = W c V c +b c

[0102] Where, V∈R f , represents the feature combination vector, f represents the dimension of the low-dimensional feature space; W is the weight matrix of the fully connected network. c ∈R f f is the number of neurons in the fully connected network; b c This represents the bias term of a neuron in a fully connected network.

[0103] The evaluation and grading are performed based on the combined feature vector V, generating a predicted evaluation grade for the webpage, specifically as follows:

[0104] A DNN model is used to evaluate and classify the combined feature vector V. The DNN model consists of multiple fully connected networks. Specifically, the combined feature vector V is taken as input to the DNN model, and matrix operations are performed sequentially through the multiple fully connected networks within the model. The final output is the predicted evaluation level of the combined feature vector, expressed as:

[0105]

[0106]

[0107]

[0108] Among them, V i Let l represent the vector matrix output by the i-th fully connected network. v This represents the number of fully connected networks in the DNN model; the weight matrix of the fully connected networks. f i b is the number of neurons in the i-th fully connected network; iRepresents the bias term of the neuron in the i-th fully connected network; the output of the last fully connected network in the DNN model. This is the quality evaluation vector; This represents the predicted probability of each evaluation level corresponding to the quality evaluation vector; Indicates the predicted evaluation level;

[0109] In the iterative training process of the static ranking learning method for multimodal Web information retrieval, a multi-class cross-entropy function is used as the loss function. The weight parameters of all models in the method are iteratively updated via backpropagation. The expression for the loss function is:

[0110]

[0111] Among them, y (i) This represents the true rating of the i-th webpage during the training process; This represents the predicted probability of each evaluation level for the i-th webpage during the training process; N represents the total number of samples during the evaluation training process; and i represents the index number of the current sample.

[0112] In some embodiments, y (i) ∈[0,R], that is y (i) For quality assessment values ​​in the range [0, R], this invention refers to the ECML / PKDD Discovery Challenge rules and considers maximizing the quality difference, with R being no less than 9, meaning there are no fewer than 10 quality levels.

[0113] Those skilled in the art should understand that, in the iterative training process of the static ranking learning method for multimodal Web information retrieval, the parameters in all models are backpropagated based on the difference between the predicted evaluation level and the actual evaluation level, and based on the results of the loss function. Different algorithms such as gradient descent are used to minimize the loss function, and the parameters are continuously dynamically adjusted so that the optimal parameters are obtained in the continuous iteration.

[0114] In another embodiment, a multimodal web information retrieval static ranking learning system is provided. The system employs the multimodal web information retrieval static ranking learning method described in the above embodiments, such as... Figure 8 As shown, the system includes an information extraction module, a feature extraction module, a feature combination module, and a quality evaluation module;

[0115] The information extraction module is used to obtain screenshots and information of the target webpage and to obtain webpage text information, HTML text, HTML tags, and webpage statistical feature information based on the screenshots and information.

[0116] The feature extraction module is used to map web page text information, HTML text information, HTML tag information, and web page statistical feature information into a first text vector sequence, a first HTML text vector sequence, a first HTML tag vector sequence, and a first web page statistical feature vector sequence, respectively; it is also used to extract features from the first text vector sequence, the first HTML text vector sequence, the first HTML tag vector sequence, and the first web page statistical feature vector sequence, respectively, to obtain text feature vectors, HTML text feature vectors, HTML tag feature vectors, and web page statistical feature vectors, respectively;

[0117] The feature combination module is used to combine text feature vectors, HTML text feature vectors, HTML tag feature vectors, and web page statistical feature vectors to obtain a combined feature vector.

[0118] The quality evaluation module is used to evaluate and classify web pages based on combined feature vectors, and generate a predicted evaluation level for the web page.

[0119] In this embodiment, the feature extraction module includes a DNN model and a Transformer encoder model;

[0120] In this embodiment, the feature combination module is equipped with a fully connected layer. The fully connected layer combines the text feature vector with text context feature information, the HTML text feature vector with HTML text context feature information, the HTML tag feature vector with HTML tag context feature information, and the web page statistical feature vector with web page statistical feature information to calculate the optimal fusion method and obtain feature results suitable for classification analysis.

[0121] In this embodiment, the quality evaluation module is equipped with a DNN model, which evaluates and classifies the quality of Web content based on combined feature vectors to determine the predicted evaluation level of the current webpage.

[0122] In another embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the multimodal Web information retrieval static ranking learning method as described in the above embodiments.

[0123] The hardware entity of the computer device includes a processor, a memory, and a communication interface; wherein, the processor typically controls the overall operation of the computer device; the communication interface is used to enable the computer device to communicate with other terminals or servers via a network; the memory is configured to store instructions and applications executable by the processor, and may also cache data to be processed or already processed by the processor and various modules in the computer device (including but not limited to image data, audio data, voice communication data, and video communication data), and may be implemented through flash memory or random access memory (RAM).

[0124] Data can be transferred between processors, communication interfaces, and memory via a bus. The bus can include any number of interconnecting buses and bridges, connecting various circuits of one or more processors and memory together.

[0125] In another embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the multimodal Web information retrieval static ranking learning method of the above embodiments.

[0126] The storage medium can be transient or non-transient. Exemplary examples include, but are not limited to, USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, and other media capable of storing computer program code.

[0127] For example, the processor can be a central processing unit (CPU), a microprocessor unit (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA), etc.

[0128] In another embodiment, a computer program is also provided, including computer-readable code, wherein, when the computer-readable code is executed in a computer device, a processor in the computer device performs some or all of the steps of the multimodal Web information retrieval static ranking learning method of the above embodiments.

[0129] In another embodiment, a computer program product is also provided, which can be implemented by hardware, software, or a combination thereof. As a non-limiting example, the computer program product can be embodied as a storage medium or as a software product, such as an SDK (Software Development Kit).

[0130] It should also be noted that, in this specification, terms such as "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0131] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A static ranking learning method for multimodal Web information retrieval, characterized in that, Includes the following steps: Obtain a screenshot of the target webpage, extract the webpage text information based on the screenshot, and map the webpage text information into a first text vector sequence; Obtain the HTML text of the target webpage and map the HTML text information into a first HTML text vector sequence; Obtain the HTML tags of the target webpage and map the HTML tag information to the first HTML tag vector sequence; Based on the web page statistical features of the target web page, the web page statistical feature information is mapped into a first web page statistical feature vector sequence; the web page statistical features include link analysis features, content heuristic features, website affiliation features, and time series features; Based on one-hot encoding and word embedding vector pattern, the second text feature vector matrix, the second HTML text feature vector matrix, and the second HTML tag feature vector matrix are obtained according to the first text vector sequence, the first HTML text vector sequence, and the first HTML tag vector sequence. The Transformer encoder model is used to extract features from the second text feature vector matrix, the second HTML text feature vector matrix, and the second HTML tag feature vector matrix to obtain text feature vectors, HTML text feature vectors, and HTML tag feature vectors. Based on the feature combination and concatenation model, the statistical feature vector sequence of the first webpage is concatenated to obtain the statistical feature vector matrix of the second webpage; the DNN model is used to extract features from the statistical feature vector matrix of the second webpage to obtain the webpage statistical feature vector. The text feature vector, HTML text feature vector, HTML tag feature vector and web page statistical feature vector are combined to obtain the combined feature vector. The evaluation and grading are performed based on the combined feature vectors, generating a predicted evaluation grade for the webpage.

2. The multimodal Web information retrieval static ranking learning method according to claim 1, characterized in that, Mapping webpage text information into a first text vector sequence is specifically as follows: The retrieved webpage text information is defined as input data x. text And treat it as a text sequence, x text It will be divided into a series of discrete text units, namely x text ={t1,t2,t3,…,t L }, where L is the original length of the input data, t i x represents text Each discrete text unit in the text; by setting a fixed length of L c After truncation or padding, a new sequence of character elements is obtained. That is, the first text vector sequence, where express Each discrete text unit in the text; Mapping HTML text information to a first HTML text vector sequence is specifically as follows: The obtained HTML text information of the webpage is defined as input data x. htmlText And treat it as a sequence of HTML text, x htmlText It is divided into a series of discrete HTML text units, namely x htmlText ={h1,h2,h3,…,h hL }, where hL is the original length of the input data, h i x represents htmlText Each discrete HTML text unit in the document; by setting a fixed length of L h After truncation or padding, a new sequence of HTML text elements is obtained. That is, the first HTML text vector sequence, where express Each discrete HTML text unit in; Mapping HTML tag information to a first HTML tag vector sequence is specifically as follows: The obtained HTML tag information of the webpage is defined as input data x. htmlTag And treat it as a sequence of HTML tags, x htmlTag It will be divided into a series of discrete HTML tag units, namely x htmlTag ={g1,g2,g3,…,g gL }, where gL is the original length of the input data, g i x represents htmlTag Each discrete HTML tag unit in the text; by setting a fixed length of L g After truncation or padding, a new sequence of HTML tag elements is obtained. That is, the first HTML tag vector sequence, where express Each discrete HTML tag unit in the text.

3. The multimodal Web information retrieval static ranking learning method according to claim 2, characterized in that, Based on one-hot encoding and word embedding vector pattern, the second text feature vector matrix, the second HTML text feature vector matrix, and the second HTML tag feature vector matrix are obtained according to the first text vector sequence, the first HTML text vector sequence, and the first HTML tag vector sequence, specifically as follows: Based on one-hot encoding and word embedding vector patterns, for each discrete text unit Perform a mapping, mapping it to a d-dimensional vector space representation. Final first text vector sequence Mapping to a vector matrix Matrix M text This is the second text feature vector matrix generated by feature construction; the value of d is 100. Based on one-hot encoding and word embedding vector patterns, each discrete HTML text unit... Perform a mapping, mapping it to a d-dimensional vector space representation. Final first HTML text vector sequence Mapping to a vector matrix Matrix M html This is the second HTML text feature vector matrix generated by feature construction; Based on one-hot encoding and word embedding vector patterns, for each discrete HTML tag unit Perform a mapping, mapping it to a d-dimensional vector space representation. Final first HTML tag vector sequence Mapping to a vector matrix Matrix M htmlTag This is the second HTML tag feature vector matrix generated by feature construction.

4. The static ranking learning method for multimodal Web information retrieval according to claim 1, characterized in that, For webpage statistical feature information, it is defined as input data and regarded as a webpage statistical feature sequence. This feature sequence is a one-dimensional numerical sequence, which also contains link analysis features x. link Content heuristic features x content Webpage Attribution Characteristics x attributioi and time series features x time ; The above four features are sequentially concatenated and combined to generate a feature vector sequence x. statistics ={x link ;x content ;x attribution ;x time }; Perform a dimensionality transformation on the feature vector sequence to generate the feature vector matrix M. statistics ={M link M content M attribution M time }; Where M statistics ∈R L*1 L represents the total number of statistical features of the webpage, 1 indicates that each element in the generated feature vector matrix is ​​a 1-dimensional vector, and M represents the total number of statistical features of the webpage. statistics The matrix is ​​the second webpage statistical feature vector matrix generated by feature construction.

5. The static ranking learning method for multimodal Web information retrieval according to claim 1, characterized in that, The Transformer encoder model is used to extract features from the second text feature vector matrix, the second HTML text feature vector matrix, and the second HTML tag feature vector matrix, specifically as follows: The Transformer encoder model includes several encoders, which include a multi-head self-attention mechanism, a feedforward layer, and layer normalization. The process of feature extraction using the Transformer encoder model is as follows: Position embedding is performed on the feature vector matrix to obtain a feature vector matrix with embedded position information; The feature vector matrix with embedded location information is passed sequentially through the multi-head self-attention mechanism, feedforward layer and layer normalization in each encoder to finally obtain the feature vector; The feature extraction process for the second text feature vector matrix is ​​expressed as follows: Among them, h i Let represent the vector matrix output by the i-th encoder, and l represent the number of encoders in the Transformer encoder model; The second text feature vector matrix represents the embedded location information; The feature extraction process for the second HTML text feature vector matrix is ​​expressed as follows: in, The second HTML text feature vector matrix represents the embedded location information; The feature extraction process for the second HTML tag feature vector matrix is ​​expressed as follows: in, The second HTML tag feature vector matrix represents the embedded location information; The location embedding process is as follows: Where pos represents the position index of the word, i represents the dimension index of the word, and M... peText M peHtml and M peHtmlTag These represent the position information matrices of the second text feature vector matrix, the second HTML text feature vector matrix, and the second HTML tag feature vector matrix, respectively. Their dimensions are respectively the same as those of the second text feature vector matrix M. text The second HTML text feature vector matrix M html And the second HTML tag feature vector matrix M htmlTag The dimensions are equal; + indicates matrix addition. Self-attention mechanism is also known as scaling dot product attention mechanism. Multi-head self-attention mechanism repeats the calculation process of self-attention mechanism h times, and then concatenates the calculation results; the process expression of multi-head self-attention mechanism is: MultiHead(Q,K,V)=Concat(head1,…,head h ) Among them, head i This represents the calculation result of the i-th self-attention mechanism, where h represents the number of repetitions of the self-attention mechanism, and the value of h is 10; MultiHead(Q,K,V) represents the calculation result of the multi-head self-attention mechanism. These represent different linear transformation matrices; These represent the query, key, and value, respectively; d k express The number of columns in a matrix, i.e., the vector dimension; In any encoder, after passing through a multi-head self-attention mechanism, a vector matrix A is obtained, which is then input into a feedforward layer. This layer consists of multiple fully connected layers and is responsible for performing vector matrix mapping and affine transformation on the vector matrix A. After residual connections and layer normalization, the output vector matrix h of the encoder module is obtained. i The output vector matrix is ​​then used as the input to the next encoder module; the output h of the last encoder... l This refers to text feature vectors, HTML text feature vectors, or HTML tag feature vectors.

6. The static ranking learning method for multimodal Web information retrieval according to claim 1, characterized in that, The DNN model includes several fully connected networks; The DNN model is used to extract features from the statistical feature vector matrix of the second webpage, specifically as follows: The second webpage statistical feature vector matrix is ​​sequentially processed through each fully connected network to perform matrix operations, ultimately yielding the webpage statistical feature vector. The process expression is as follows: Among them, M i M represents the vector matrix output by the i-th fully connected network, where l represents the number of fully connected networks in the DNN model; statistics The second webpage's statistical feature vector matrix represents the weight matrix of the fully connected network. L i b is the number of neurons in the i-th fully connected network; i M represents the bias term of the neuron in the i-th fully connected network; the output M of the last fully connected network in the DNN model. i This is the statistical feature vector of the webpage.

7. The multimodal Web information retrieval static ranking learning method according to claim 1, characterized in that, The text feature vector, HTML text feature vector, HTML tag feature vector, and webpage statistical feature vector are combined as follows: Using vector concatenation operations, the text feature vector V is converted into a single vector. text HTML text feature vector V htmlText HTML tag feature vector V htmlTag and webpage statistical feature vector V statistics By sequentially concatenating and combining the features, we obtain the concatenated feature vector V. c =[V text V htmlText V htmlTag V statistics ]; A single fully connected layer is used to perform feature combination operations on the concatenated feature vectors. The purpose of this method is to reduce the dimensionality of features, eliminate redundant variables, and find the intrinsic relationships between feature vectors. The expression for this is: V=W c V c +b c Where, V∈R f , represents the feature combination vector, f represents the dimension of the low-dimensional feature space; W is the weight matrix of the fully connected network. c ∈R f f is the number of neurons in the fully connected network; b c This represents the bias term of a neuron in a fully connected network; The evaluation and grading are performed based on the combined feature vector v, generating a predicted evaluation grade for the webpage, specifically as follows: A DNN model is used to evaluate and classify the combined feature vector v. The DNN model consists of multiple fully connected networks. Specifically, the combined feature vector V is taken as input to the DNN model and passed through the multiple fully connected networks in the model for matrix operations. The final output is the predicted evaluation level of the combined feature vector, expressed as: Among them, V i Let l represent the vector matrix output by the i-th fully connected network. v This represents the number of fully connected networks in the DNN model; the weight matrix of the fully connected networks. f i b is the number of neurons in the i-th fully connected network; i Represents the bias term of the neuron in the i-th fully connected network; the output of the last fully connected network in the DNN model. This is the quality evaluation vector; This represents the predicted probability of each evaluation level corresponding to the quality evaluation vector; Indicates the predicted evaluation level; In the iterative training process of the static ranking learning method for multimodal Web information retrieval, a multi-class cross-entropy function is used as the loss function. The weight parameters of all models in the method are iteratively updated via backpropagation. The expression for the loss function is: Among them, y (i) This represents the true rating of the i-th webpage during the training process; This represents the predicted probability of each evaluation level for the i-th webpage during the training process; N represents the total number of samples during the evaluation training process; and i represents the index number of the current sample. In the iterative training process of the static ranking learning method for multimodal Web information retrieval, the parameters in all models are backpropagated based on the difference between the predicted evaluation level and the actual evaluation level, according to the result of the loss function. Different algorithms such as gradient descent are used to minimize the loss function, and the parameters are dynamically adjusted continuously to obtain the optimal parameters in the continuous iteration.

8. A multimodal Web information retrieval static ranking learning system, characterized in that, The system adopts the multimodal Web information retrieval static ranking learning method according to any one of claims 1-7, and the system includes an information extraction module, a feature extraction module, a feature combination module, and a quality evaluation module; The information extraction module is used to obtain screenshots and information of the target webpage and to obtain webpage text information, HTML text, HTML tags, and webpage statistical feature information based on the screenshots and information. The feature extraction module is used to map web page text information, HTML text information, HTML tag information, and web page statistical feature information into a first text vector sequence, a first HTML text vector sequence, a first HTML tag vector sequence, and a first web page statistical feature vector sequence, respectively; it is also used to extract features from the first text vector sequence, the first HTML text vector sequence, the first HTML tag vector sequence, and the first web page statistical feature vector sequence, respectively, to obtain text feature vectors, HTML text feature vectors, HTML tag feature vectors, and web page statistical feature vectors, respectively; The feature combination module is used to combine text feature vectors, HTML text feature vectors, HTML tag feature vectors, and web page statistical feature vectors to obtain a combined feature vector. The quality evaluation module is used to evaluate and classify web pages based on combined feature vectors, and generate a predicted evaluation level for the web page. The feature extraction module includes a DNN model and a Transformer encoder model; The feature combination module has a fully connected layer. Through the fully connected layer, the text feature vector with text context feature information, the HTML text feature vector with HTML text context feature information, the HTML tag feature vector with HTML tag context feature information, and the web page statistical feature vector with web page statistical feature information are combined to calculate the best fusion method and obtain feature results suitable for classification analysis. The quality evaluation module is equipped with a DNN model, which evaluates and classifies the quality of Web content based on combined feature vectors to determine the predicted evaluation level of the current webpage.

9. A computer device, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes a computer program, it implements the multimodal Web information retrieval static ranking learning method as described in any one of claims 1-7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the multimodal Web information retrieval static ranking learning method according to any one of claims 1-7.

Images