Webpage information classification method and device and storage medium

The web page data is obtained through Scrapy and Selenium, combined with XLM-RoBERTa and convolutional graph neural network model, and the difficulties of traditional crawler tools in dynamic web page processing are solved, and the accuracy and efficiency of web page information classification are improved.

CN119003920AInactive Publication Date: 2024-11-22CETC BIGDATA RES INST CO LTD

Patent Information

Application Number
CN202411481452.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-23
Publication Date
2024-11-22
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Traditional crawler tools require complex configuration and additional processing steps when processing dynamic web pages, making it difficult to efficiently capture target information. In addition, traditional web page analysis methods are unable to handle complex and changeable web page structures, and identify web page effects with high noise density or sparse text content. Not good.

Method used

Use Python's Scrapy library to obtain static data, Selenium automation tool to obtain dynamic data, build a DOM tree structure, vectorize data through XLM-RoBERTa pre-trained language model, convert it into graph structure, and input a convolutional graph neural network model to generate feature vectors , enter the preconfigured classification model for classification.

Benefits of technology

It reduces the difficulty of dynamic web page crawling, improves the accuracy of the classifier, can effectively process complex web page structures and identify web pages with high noise density or sparse text content.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119003920A_ABST
    Figure CN119003920A_ABST
Patent Text Reader

Abstract

The invention discloses a webpage information classification method and device and a storage medium, and is applied to the field of information classification. The method comprises the following steps: acquiring static data in a webpage from a target webpage by using a Scrapy library of Python; obtaining dynamic data of the webpage from the target webpage by using a Selenium automation tool; constructing a DOM tree structure based on the static data and the dynamic data; using an XLM-RoBERTa pre-training language model to carry out vectorization on the static data and the dynamic data of the constructed DOM tree structure, and converting the static data and the dynamic data into numerical vectors; converting the DOM tree structure into a graph structure, inputting the numerical vector into a convolutional graph neural network model, and encoding the graph structure through the convolutional graph neural network model to generate a feature vector; and inputting the feature vector into a pre-configured classification model for calculation, and classifying the static data and the dynamic data through the pre-configured classification model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of information classification, and in particular to a web page information classification method, device and storage medium. Background Art

[0002] With the continuous advancement and innovation of information technology, the field of web page information classification is also constantly evolving. Crawler technology is generally used to obtain web page information from the target web page.

[0003] In the prior art, traditional crawler tools can efficiently capture target information when applied to static web pages, but for dynamically loaded web pages, traditional crawler tools require complex configuration and additional processing steps, which increases the difficulty of capturing target information.

[0004] Web pages, as the main carrier of Internet information, contain rich multimedia data such as text, images, and videos. However, web page content usually exists in the form of unstructured HTML documents, and it is difficult to directly extract effective information from them. Although traditional web page parsing methods such as regular expressions, XPath, and Beautiful Soup can solve the problem to a certain extent, they often seem to be powerless when dealing with complex and changeable web page structures or require a lot of manual maintenance and rule updates. Traditional web page content classification methods based on text and symbol density rely on the density of text and symbols (such as punctuation marks) in web pages to distinguish between the main content and other noise information in the web page (such as advertisements, navigation bars, copyright information, etc.). However, for some web pages with special formats, such as web pages with a large number of pictures, videos, or complex layouts, or web page structures with high noise density or sparse text content, the recognition effect may not be good. Summary of the invention

[0005] In order to solve the above technical problems, the present application provides a web page information classification method, which comprises the following steps: Use Python's Scrapy library to obtain static data from the target web page; Use Selenium automation tool to obtain dynamic data of web pages from target web pages; Constructing a DOM tree structure based on the static data and the dynamic data; Vectorize the static data and the dynamic data in the constructed DOM tree structure using the XLM-RoBERTa pre-trained language model and convert them into numerical vectors; The DOM tree structure is converted into a graph structure, and the numerical vector is input into the convolutional graph neural network model. The graph structure is encoded by the convolutional graph neural network model to generate a feature vector. The feature vector is input into a preconfigured classification model for calculation, and the static data and the dynamic data are classified by the preconfigured classification model.

[0006] Optionally, before constructing the DOM tree structure based on the static data and the dynamic data, the method further includes: The acquired static data and the dynamic data are preprocessed.

[0007] Optionally, preprocessing the acquired static data and the dynamic data includes: Use the BeautifulSoup4 toolkit to perform a depth-first traversal of the DOM tree for the static data and the dynamic data, and remove HTML tags and scripts.

[0008] Optionally, converting the tree structure into a graph structure includes: Define each node in the DOM tree structure and the text in the node as a node in the graph; The parent-child relationship between nodes is determined as directed edges in the graph, and each edge connects a parent node and one or more child nodes to obtain a graph structure.

[0009] Optionally, before the feature vector is input into a preconfigured classification model for calculation, and the static data and the dynamic data are classified by the preconfigured classification model; after the DOM tree structure is converted into a graph structure, and the numerical vector is input into a convolutional graph neural network model, and the graph structure is encoded by the convolutional graph neural network model, and the feature vector is generated, the method further includes: Build an initial classification model; Input the labeled sample data set into the initialization classification model; Calculate the classification score zic for each sample i and each category c through the initialized classification model; Use the cross entropy loss function to calculate the difference between the probability distribution predicted by the model and the label; Based on the difference, the initialized classification model is optimized using a preconfigured optimization algorithm until a converged classification model is obtained.

[0010] Optionally, the classification model is configured with a softmax classification algorithm, and the softmax classification algorithm is defined as: ; Among them, c represents the classification category, represents the probability that sample i belongs to category c, Represents the exponential sum of all category scores.

[0011] Optionally, the Calculate using the following formula: ; in, represents the classification score, represents the weight vector of the category, represents the category bias, The feature vector representing the sample.

[0012] Optionally, the cross entropy loss function is calculated by the following formula: ; in, is a binary indicator representing the indicator variable, represents the loss value of the sample, c represents the index or total number of categories, represents the logarithmic function, Represents the output probability of the category after classification score.

[0013] Optionally, the method of obtaining dynamic data of a webpage from a target webpage using a Selenium automation tool includes: Configure Python development environment and Selenium automation tool; In the Python environment, configure the Web Driver browser driver; A collection model is constructed, and dynamic data of the target web page is acquired through the collection model.

[0014] Optionally, the constructing of the acquisition model and obtaining the dynamic data of the target webpage through the acquisition model includes: Initializing the acquisition model; Open the configured web page start address and wait for the web page to load; Determine whether there is an input event: If it is determined that there is an input event, the input events are executed in sequence, and the confirm search button is simulated to complete the search; Determine whether the user has configured filtering conditions: If it is determined that the user has configured a filtering condition, the click operation is performed in sequence according to the position information of the click event input by the user; Determine whether the user has configured a data list item: If it is determined that the user has configured a data list item, the Selenium automation tool automatically clicks each item in the data list and loads the page after the jump; According to user configuration, the content of dynamic data of the target web page is extracted.

[0015] Optionally, the preconfigured optimization algorithm includes: Calculate the gradient of the loss with respect to the model parameters and update the weights and biases based on the gradient, where the update is performed as follows: ; ; in represents the learning rate, represents the parameter value before weight iteration, represents the parameter value before the bias iteration, represents iteration, represents the value after weight iteration, represents the value of the bias term after iteration, represents the partial derivative, represents the partial derivative of the loss with respect to the weight, represents the partial derivative of the loss with respect to the bias.

[0016] A third aspect of the present application provides a web page information classification device, the device comprising: The first acquisition unit uses the Python Scrapy library to acquire static data from the target web page; The second acquisition unit uses the Selenium automation tool to acquire dynamic data of the web page from the target web page; A first construction unit constructs a DOM tree structure based on the static data and the dynamic data; A numerical vector conversion unit, which uses an XLM-RoBERTa pre-trained language model to vectorize the static data and the dynamic data in the constructed DOM tree structure and convert them into numerical vectors; The encoding unit converts the DOM tree structure into a graph structure, and inputs the numerical vector into the convolutional graph neural network model, encodes the graph structure through the convolutional graph neural network model, and generates a feature vector; The classification unit inputs the feature vector into a preconfigured classification model for calculation, and classifies the static data and the dynamic data through the preconfigured classification model.

[0017] A fourth aspect of the present application provides a web page information classification device, the device comprising: Processor, memory, input-output unit, and bus; The processor is connected to the memory, the input and output unit, and the bus; The memory stores a program, and the processor calls the program to execute the first aspect and any optional method in the first aspect.

[0018] A fifth aspect of the present application provides a computer-readable storage medium, on which a program is stored. When the program is executed on a computer, the program executes the first aspect and any optional method in the first aspect.

[0019] It can be seen from the above technical solutions that this application has the following advantages: 1. By using Selenium automation tools to obtain dynamic data from the target web page, dynamic web pages can be crawled without complex configuration, reducing the difficulty of crawling; 2. By encoding the structural features of the DOM tree and generating samples corresponding to the feature vector, the accuracy of the classifier is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solution in the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 A schematic diagram of an embodiment of the web page information classification method provided in this application; Figure 2 A flowchart of another embodiment of the web page information classification method provided in this application; Figure 3 A flowchart of another embodiment of the web page information classification method provided in this application; Figure 4 This is a flow chart of another embodiment of the web page information classification method provided in this application; Figure 5 A schematic diagram of a specific embodiment of the web page information classification method provided in this application; Figure 6 A flowchart of another specific embodiment of the web page information classification method provided in this application; Figure 7-1 This is a flowchart of the first part of another specific embodiment of the web page information classification method provided in this application; Figure 7-2 This is a flowchart of the second part of another specific embodiment of the web page information classification method provided in this application; Figure 8 This is a schematic diagram of the structure of another embodiment of a web page information classification device provided in this application; Fig. 9 This is a schematic diagram of the structure of another embodiment of a web page information classification device provided in this application. DETAILED DESCRIPTION

[0022] It should be noted that the webpage information classification method provided by the present application can be applied to a terminal, a system, or a server. For example, the terminal can be a smart phone or a computer, a tablet computer, a smart TV, a smart watch, a portable computer terminal, or a fixed terminal such as a desktop computer. For the convenience of explanation, the present application uses the terminal as an example for explanation.

[0023] Before introducing the web page classification method provided in the embodiment of the present application, some concepts involved in the embodiment of the present application are first introduced: DOM tree structure: DOM tree structure (Document Object Model Tree) refers to a tree structure that organizes and represents documents according to the hierarchical structure of HTML, XML or XHTML documents. This structure represents the entire document as a hierarchical structure composed of nodes, each of which represents an element, attribute, text or comment in the document.

[0024] Static data: refers to web pages whose content is pre-generated on the server and statically displayed on the client (browser). The page content does not change dynamically based on user behavior or external factors. It is usually developed using technologies such as HTML, CSS, and JavaScript. The page is sent to the client after being rendered on the server, and the content is not dynamically updated through JavaScript or other technologies. After the crawler receives the HTML code, it uses an HTML parser (such as BeautifulSoup, lxml, etc.) to extract the required data. This data is usually contained in HTML tags.

[0025] Dynamic data: refers to web pages whose content changes according to user behavior, time or other dynamic factors. For example, search results page, news headlines, etc. Use JavaScript, AJAX or other technologies to load dynamic content. These technologies enable web page content to be rendered on the client (browser), and the server is only responsible for providing data. The data information is not in the HTML code. Collection requires Selenium or Puppeteer to simulate real browser behavior and execute JavaScript to achieve collection.

[0026] Embodiment 1: See also Figure 1 The present application first provides an embodiment of a web page information classification method, which includes: S101. Use Python's Scrapy library to obtain static data from the target web page; Scrapy is a fast and advanced crawling and web scraping framework for crawling websites and extracting structured data from pages. It is based on the Twisted asynchronous network framework and is written in Python. The architecture of the Scrapy framework is very flexible and contains multiple components: engine, scheduler, downloader, crawler, project pipeline, downloader middleware, and crawler middleware. When writing a program to crawl files, you can define how to crawl a website, including how to extract data and generate new requests.

[0027] The static data obtained refers to the source code of the web page, the domain name information of the web page, the script of the web page, and the text, pictures, audio, video and files contained in the web page.

[0028] S102, using Selenium automation tool to obtain dynamic data of the web page from the target web page; Selenium is an automation tool for web application testing. It supports multiple programming languages ​​and browsers, supports multiple platforms, is free and compact, can simulate user interaction behaviors, automatically execute test cases and generate test reports, and can run tests directly in the browser, just like real users are operating. Selenium can handle all types of web pages, including dynamically loaded web content. Selenium writes automated test scripts and interacts with the browser through the WebDriver API. WebDriver can directly control the browser and simulate user operations such as clicking and entering text. Selenium is completely open source and has no restrictions on commercial users. It supports distribution and has a mature community and learning documents.

[0029] S103, constructing a DOM tree structure based on the static data and the dynamic data; The DOM tree structure consists of document nodes, element nodes, attribute nodes, text nodes, comment nodes, and document type nodes. It is a hierarchical structure. Each element node can have child nodes. The element node contains tag information and plain text content. A directed edge is established between each element node and its child nodes, indicating a parent-child relationship. Each edge connects a parent node and one or more child nodes.

[0030] The hierarchical nature of the DOM tree structure makes it easy to operate and access documents. Through the DOM tree structure, developers can use various methods and attributes to locate, manipulate, and modify specific nodes in the document to achieve dynamic updates and interactions with HTML, XML, or XHTML documents. The DOM is presented in a tree structure that is easy to access and modify.

[0031] S104, using the XLM-RoBERTa pre-trained language model to vectorize the static data and the dynamic data in the constructed DOM tree structure into numerical vectors; XLM-RoBERTa is a multilingual version of the RoBERTa model that supports multiple languages ​​and can be used for a variety of natural language processing (NLP) tasks, including text classification, named entity recognition (NER), question-answering systems, etc., and supports multiple language inputs. The XLM-RoBERTa model can be used to divide and encode text at the subword level to obtain a vector representation of each subword in a specific context. The model uses a deep neural network with a Transformer architecture to divide and encode text at the subword level, ensuring that each subword vector contains rich contextual information.

[0032] S105, converting the DOM tree structure into a graph structure, and inputting the numerical vector into a convolutional graph neural network model, encoding the graph structure through the convolutional graph neural network model, and generating a feature vector; The convolutional graph neural network model is a deep learning model that applies the idea of ​​convolutional neural networks to graph structured data. When processing such data, this network model can better capture the dependencies and structural information between nodes and can handle more complex and irregular data structures. Therefore, it has a wide range of applications in computer vision, recommendation systems, traffic prediction and other fields.

[0033] By inputting the numerical vector into the convolutional neural network model, selecting the number of layers of the convolutional neural network model, selecting the dimension of node embedding, selecting the optimizer, and setting the parameters of the convolutional neural network model, the model can accurately encode the graph structure and generate a feature vector. The generated feature vector is used to input the preconfigured classifier for calculation to obtain an optimized classifier and output a more accurate classification result.

[0034] S106: Input the feature vector into a preconfigured classification model for calculation, and classify the static data and the dynamic data through the preconfigured classification model.

[0035] The feature vector converted from the DOM tree structure into a numerical form is input into the trained classification model for calculation. First, the features of the feature vector are extracted, and then the data is cleaned to remove noise, missing values ​​and irrelevant data. The processed feature vector is used as input and passed to the pre-configured classification model. The classification model may be a neural network, decision tree, support vector machine or any other machine learning algorithm that can process vector input.

[0036] Use the softmax classification algorithm to classify the data, calculate the data, get the weight of each data, and use the cross entropy loss function as the optimization target to measure the difference between the predicted probability distribution and the true label. According to the gradient of the loss function, use gradient descent or other optimization algorithms to update the weight of the model, and repeat the above process until a converged model is obtained.

[0037] The feature vector is forward propagated through the layers of the classification model, each layer transforms the input data and finally outputs the classification result.

[0038] Specifically, the feature vector is forward propagated through each layer of the classification model, each layer transforms the input data, and finally outputs the classification results such as the text and title. Specific classification categories are given as examples.

[0039] The beneficial effects of this embodiment are that by using the Selenium automation tool to obtain dynamic data from the target web page, dynamic web pages can be crawled without complex configuration, thereby reducing the difficulty of crawling; by encoding the structural features of the DOM tree and generating samples corresponding to the feature vector, the accuracy of the classifier is improved.

[0040] Embodiment 2: See also Figure 2 The present application provides another embodiment of a web page information classification method, which includes: S201, using Python's Scrapy library to obtain static data from the target web page; S202, using Selenium automation tool to obtain dynamic data of the webpage from the target webpage; S203, preprocessing the acquired static data and the dynamic data; Preprocessing data can improve data quality and help machine learning algorithms better understand and learn patterns and regularities in the data. Perform data cleaning on static data, convert the cleaned data into a unified type, integrate the data to form a comprehensive data set, and encode categorical variables so that they can be processed by the algorithm. Dynamic data changes over time, so it is necessary to process the time series of dynamic data, aggregate the data in time, use interpolation methods to fill in missing values ​​in the time series, and use methods such as difference and logarithmic transformation to process non-stationary time series data to make it more suitable for modeling.

[0041] S204, constructing a DOM tree structure based on the static data and the dynamic data; S205, using the XLM-RoBERTa pre-trained language model to vectorize the static data and the dynamic data in the constructed DOM tree structure into numerical vectors; S206, converting the DOM tree structure into a graph structure, and inputting the numerical vector into a convolutional graph neural network model, encoding the graph structure through the convolutional graph neural network model, and generating a feature vector; S207: Input the feature vector into a preconfigured classification model for calculation, and classify the static data and the dynamic data using the preconfigured classification model.

[0042] In this embodiment, steps S201 to S202 are implemented in a similar manner to steps S101 to S102 in the above embodiment, and are not described in detail here. Steps S204 to S207 are similar to steps S103 to S106 in the above embodiment, and are not described in detail here.

[0043] The beneficial effect of this embodiment is that static data and dynamic data are preprocessed when constructing the tree structure, making it faster and more convenient to construct the tree structure.

[0044] Embodiment three: See also Figure 3 This application is based on Figure 2 The corresponding embodiment provides a further embodiment, which includes a specific preprocessing method, including: S301, using Python's Scrapy library to obtain static data in the target web page; S302, using Selenium automation tool to obtain dynamic data of the webpage from the target webpage; S303, using the BeautifulSoup4 toolkit to perform a DOM tree depth-first traversal on the static data and the dynamic data, and remove HTML tags and scripts; BeautifulSoup4 is a Python library for extracting data from HTML or XML files. It can help developers efficiently navigate, search, and modify documents during web crawling and data extraction. It creates a parse tree so that users can traverse, search, and modify tags in the parse tree. Use the methods provided by the BeautifulSoup object to navigate and find the required elements in the document, and further extract attributes, text content, and other information based on the obtained elements for subsequent processing.

[0045] Before using BeautifulSoup4 to traverse the HTML DOM tree, use JavaScript to load dynamic content, then use BeautifulSoup4 to parse all static data and remove HTML tags and scripts.

[0046] S304, constructing a DOM tree structure based on the static data and the dynamic data; S305, using the XLM-RoBERTa pre-trained language model to vectorize the static data and the dynamic data in the constructed DOM tree structure into numerical vectors; S306, converting the DOM tree structure into a graph structure, and inputting the numerical vector into a convolutional graph neural network model, encoding the graph structure through the convolutional graph neural network model, and generating a feature vector; S307: Input the feature vector into a preconfigured classification model for calculation, and classify the static data and the dynamic data through the preconfigured classification model.

[0047] In this embodiment, steps S301 to S302 are implemented in a similar manner to steps S101 to S102 in the above embodiment, and are not described in detail here. Steps S304 to S307 are similar to steps S103 to S106 in the above embodiment, and are not described in detail here.

[0048] The beneficial effect of this embodiment is that static data and dynamic data are traversed first before building the DOM tree structure, HTML tags and scripts are removed, and convenience is provided for building the DOM tree structure.

[0049] Embodiment 4: See also Figure 4 The present application first provides an embodiment of a web page information classification method, which includes: S401, using Python's Scrapy library to obtain static data in the target web page; S402, using Selenium automation tool to obtain dynamic data of the webpage from the target webpage; S403, constructing a DOM tree structure based on the static data and the dynamic data; S404, using the XLM-RoBERTa pre-trained language model to vectorize the dynamic data of the constructed DOM tree structure and convert it into a numerical vector; S405, defining each node in the DOM tree structure and the text in the node as a node in the graph; The DOM tree structure is a tree structure that organizes and represents documents according to the hierarchical structure of HTML, XML or XHTML documents. It represents the entire document as a hierarchical structure composed of nodes.

[0050] Each node represents an element, attribute, text, or comment in the document.

[0051] A graph is a nonlinear data structure consisting of nodes and edges. Nodes are the basic units of a graph, and edges connect nodes to each other.

[0052] First, a depth-first search algorithm or a breadth-first search algorithm is used to traverse the DOM tree structure. During the traversal process, each node in the DOM tree structure is visited. For each node in the DOM tree structure or the text content in the node, a corresponding graph node is created.

[0053] In this embodiment, each node in the DOM tree structure and the text in the node are defined as a node in the graph, which can provide a more detailed and comprehensive representation of the web page structure. The structural information of the DOM and the text content in the node are incorporated into the graph structure, which is convenient for complex web page content analysis, information extraction or structure visualization tasks. This conversion enhances the flexibility and depth of data processing and provides a basis for further algorithm processing and application.

[0054] S406, the parent-child relationship between the nodes is determined as a directed edge in the graph, each edge connects a parent node and one or more child nodes, and a graph structure is obtained; A directed edge in a graph is a special type of edge in a graph that has a clear directionality, pointing from one vertex to another. Directed graphs have a wide range of applications, including network analysis, flow charts, social network analysis, etc.

[0055] In a tree structure, each node has a parent node (except the root node), and each parent node can have multiple child nodes. This relationship is represented by directed edges, which connect two nodes and represent the parent-child relationship between them.

[0056] For each node in the DOM tree, and it has child nodes, a directed edge from the parent node to the child node is established in the graph structure for this pair of parent and child nodes. The text content in the node creates an independent graph node, and the corresponding directed edges of the graph need to be established between these nodes to obtain a graph structure containing all DOM nodes and the nodes of the node text content and the relationship between them.

[0057] In this embodiment, by determining directed edges between nodes and connecting parent and child nodes, the original hierarchical structure of the DOM tree is maintained, so that the dependency and subordination between nodes can be clearly reflected in the graph structure. And through this process, a graph structure containing nodes and directed edges is obtained, which provides a basis for subsequent graph algorithm analysis and optimization.

[0058] S407, inputting the numerical vector into the convolutional graph neural network model, encoding the graph structure through the convolutional graph neural network model, and generating a feature vector; S408: Input the feature vector into a preconfigured classification model for calculation, and classify the static data and the dynamic data using the preconfigured classification model.

[0059] In this embodiment, steps S401 to S404 are implemented in a similar manner to steps S101 to S104 in the above embodiment, and are not described in detail here. Steps S407 to S408 are similar to steps S105 to S106 in the above embodiment, and are not described in detail here.

[0060] In this embodiment, the DOM tree structure is replaced with a graph structure, which enhances the flexibility of data processing. By treating each node and its text as an independent node in the graph, the hierarchical structure and content information of the DOM tree are retained, so that the originally copied tree structure can be presented in a more intuitive and flexible way in the graph structure. By determining directed edges between nodes and connecting parent and child nodes, the original hierarchical structure of the DOM tree is maintained, so that the dependencies and subordinate relationships between nodes can be clearly reflected in the graph structure. And through this process, a graph structure containing nodes and directed edges is obtained, which provides a basis for subsequent algorithm analysis and optimization.

[0061] Embodiment five: See also Figure 5 In practice, the classification algorithm used needs to be classified in a classification model that has completed convergence. Now we provide a training process for a convergent classification model: S501, constructing an initialization classification model; A classification model is a widely used data analysis model that can classify data into different categories. Classification models are usually trained using classification algorithms to identify different categories in the model.

[0062] Common classification models include: logistic regression model, decision tree model, support vector machine model, naive Bayes model, random forest model and gradient boosting tree model. Classification models have many uses in practical applications, such as customer classification, product classification, event classification, etc. Common classification models use classification algorithms including random forest, support vector machine SVM, neural network and softmax regression.

[0063] To build an initialized classification model, it is necessary to define the model data structure, clarify the specific structure of the classification model and the required data fields, and then use the database management system to establish the database model. After the database model is established, data initialization is required, that is, inserting the data into the database. After completing the establishment of the database model and data initialization, testing and verification are required to ensure the correctness of the model and the integrity of the data.

[0064] In this embodiment, by constructing an initialization classification model, the accuracy and efficiency of data classification can be significantly improved, providing a basis for subsequent sample data set input and classification training.

[0065] S502, inputting the sample data set with labels into the corresponding initialization classification model; A dataset is a set of data used to train and evaluate models. This data can include different types of samples such as numerical data, text, images, audio, or video.

[0066] The dataset is divided into feature vectors and category labels. Then, a programming library is used to load and preprocess these data to ensure that they are suitable for model training. Next, a classification model is initialized. Finally, the preprocessed feature vectors and category labels are passed as input data to the model for training.

[0067] In this embodiment, by inputting a sample data set with web page feature vectors and category labels into the initialization classification model, the accuracy and efficiency of web page content classification are significantly improved. The classification model can use feature information for effective learning, quickly identify and classify web page content, and provide an accurate basis for subsequent classification calculations.

[0068] S503: Calculate the classification score for each sample i and each category c through the initialized classification model ; In this embodiment, the initialization classification model processes each input sample i and calculates a classification score for each possible category c. .

[0069] The sample features are transformed based on the weights and biases within the model, and then these scores are converted into probability distributions through the softmax classification algorithm, thereby assigning a probability value to each category.

[0070] in, Calculate using the following formula: ; in, represents the classification score, represents the weight vector of the category, represents the category bias, The feature vector representing the sample; In this embodiment, the classification score is calculated , which helps to evaluate the classification model's prediction accuracy for each sample and its ability to distinguish different categories, and by calculating the classification score , the performance of the model can be further analyzed, so as to comprehensively evaluate the pros and cons of the model and provide a basis for the subsequent calculation of the loss and optimization of the classification model.

[0071] S504, using a cross entropy loss function to calculate the difference between the probability distribution predicted by the model and the label; ‌The cross entropy loss function mainly includes the cross entropy loss function of the single-label classification task and the cross entropy loss function of the multi-label classification task. The cross entropy loss function is calculated by the following formula: ; in, is a binary indicator representing the indicator variable, represents the loss value of the sample, Indicates the index or total number of categories, represents the logarithmic function, Represents the output probability of the category after classification score.

[0072] In this embodiment, the cross entropy loss function is mainly used to calculate the difference between the probability distribution predicted by the model and the true probability distribution, which provides a guarantee for the subsequent optimization of the classification model and improves the classification accuracy.

[0073] S505, based on the difference, optimizing the initialized classification model using a preconfigured optimization algorithm until a converged classification model is obtained; In this embodiment, the preconfigured optimization algorithm uses a gradient descent algorithm, which iteratively searches in the opposite direction of the gradient corresponding to the current point on the function to achieve the purpose of finding the minimum value of the function.

[0074] The gradient descent algorithm is used to adjust the model parameters according to the output of the cross entropy loss function to reduce the difference between the prediction and the true label. The loss function value is gradually reduced by calculating the gradient of the loss function with respect to the model parameters and updating the parameters in the opposite direction of the gradient. The whole process continues to cycle until the model parameters converge and reach the preset value.

[0075] The gradient of the loss with respect to the model parameters is calculated, and the weights and biases are updated according to the gradient, wherein the update is performed by the following formula: ; ; in, represents the learning rate, Wc represents the parameter value before the weight iteration, and bc represents the parameter value before the bias iteration. represents iteration, represents the value after weight iteration, represents the value of the bias term after iteration, represents the partial derivative, represents the partial derivative of the loss with respect to the weight, represents the partial derivative of the loss with respect to the bias.

[0076] In this embodiment, the classification model is optimized using a gradient descent algorithm, which can significantly promote the model to converge to the global optimal solution quickly and effectively. By continuously calculating the gradient of the loss function with respect to the model parameters and updating the parameter values ​​in the opposite direction of the gradient, the prediction error is gradually reduced. On the one hand, it can improve the training efficiency of the model and reduce the number of iterations required to achieve stable performance; on the other hand, by fine-tuning the parameters, gradient descent helps to improve the classification accuracy of the model, so that the model can make more accurate predictions for unseen data.

[0077] In this embodiment, by constructing and optimizing the initialization classification model, this process significantly improves the classification accuracy of the model for sample data. The cross entropy loss function is used to calculate the difference between the probability distribution predicted by the model and the label, ensuring the accuracy of the model optimization direction. The preconfigured optimization algorithm calculates the gradient of the loss relative to the model parameters, and updates the weights and biases according to the gradient, which accelerates the convergence speed of the model. The resulting converged classification model can output more reliable and accurate classification results when processing classified data sets, providing strong technical support in practical applications and further improving the level of automation of classification.

[0078] Embodiment six: See also Figure 6 This application is based on Figure 6 The corresponding embodiment provides a further embodiment, which includes a specific method for obtaining dynamic data, including: S601, using Python's Scrapy library to obtain static data in the target web page; S602. Configure Python development environment and Selenium automation tool; In this embodiment, before obtaining dynamic data, it is necessary to configure the Python development environment and the Selenium automation tool. Configuring the Python development environment includes setting environment variables, and the test environment configuration is successful. Configuring the Selenium automation tool includes installing the Selenium library, downloading the browser driver, and other steps.

[0079] The Python development environment is the basis for running automation tools, and has rich library and module resources, which can improve the efficiency and stability of code operation. The Selenium automation tool can help users simulate browser operations to achieve dynamic data acquisition. This embodiment does not specifically limit the configuration versions of the Python development environment and the Selenium automation tool.

[0080] S603, in the Python environment, configure the Web Driver browser driver; Web Driver is a tool for automating the control of browsers. Programming languages ​​such as Python, Java, and C# interact with browsers through the APIs provided by browsers. Web Driver controls the behavior of browsers by simulating user operations, such as clicks, input, and navigation. It is a key component for implementing automated testing and web data capture. Each mainstream browser, such as Chrome, Firefox, Edge, and Safari, has a corresponding WebDriver driver, such as ChromeDriver, GeckoDriver, etc.

[0081] To configure the Web Driver browser driver, you need to select the browser driver for download configuration. The configuration information includes: specifying the driver path, configuring the waiting time, configuring the collection frequency of the scheduled task, determining the collection target field, specifying the access address and other information.

[0082] After completing the environment setup, configure the Web Driver browser driver so that the Selenium automation tool can communicate with the browser and control the browser's behavior, establishing a communication foundation for subsequent acquisition of dynamic data on the web page.

[0083] S604: constructing a collection model, and acquiring dynamic data of the target web page through the collection model; The collection model defines the data flow and processing process from the data source to the final application, including key steps such as data acquisition, transmission, storage, processing and analysis. The collection model involves a variety of technologies and tools, such as web crawlers, API interfaces, databases, data stream processing frameworks, etc. Through the reasonable design and implementation of the collection model, timely update, high-quality processing and effective use of data can be achieved, thereby supporting decision-making and business optimization.

[0084] In the above configured environment, building a collection model requires formulating collection strategies such as page navigation, element positioning, and data extraction rules, determining the data storage method, configuring the exception handling mechanism, configuring scheduled tasks, and using automated scripts to build the collection model, simulate the interaction between users and the target web page, and obtain dynamic data from dynamically loaded pages through element positioning and data extraction strategies.

[0085] Building a collection model and obtaining dynamic data of the target web page through the model can realize the automation, efficiency and scale of data collection, and significantly improve the speed and quality of data acquisition. This model not only ensures the consistency and real-time nature of the data, but also supports large-scale data processing and deep mining, thereby providing strong support for decision-making and business analysis. The flexibility and scalability of the collection model enable it to adapt to the ever-changing data environment and enhance the compliance and security of data collection.

[0086] S605, constructing a DOM tree structure based on the static data and the dynamic data; S606, using the XLM-RoBERTa pre-trained language model to vectorize the static data and the dynamic data in the constructed DOM tree structure into numerical vectors; S607, converting the DOM tree structure into a graph structure, and inputting the numerical vector into a convolutional graph neural network model, encoding the graph structure through the convolutional graph neural network model, and generating a feature vector; S608: Input the feature vector into a preconfigured classification model for calculation, and classify the static data and the dynamic data using the preconfigured classification model.

[0087] In this embodiment, step S601 is implemented in a similar manner to step S101 in the above embodiment, and will not be described in detail here. In this embodiment, steps S605 to S608 are implemented in a similar manner to steps S103 to S106 in the above embodiment, and will not be described in detail here.

[0088] The beneficial effect of this embodiment is that the dynamic data of the target web page can be acquired through the Selenium automation tool, which can efficiently and automatically acquire the dynamic data of the target web page, and the dynamic web page can be crawled without complicated configuration, which reduces the difficulty of crawling.

[0089] Embodiment seven: When using Selenium automation tools to obtain dynamic data from the target web page, there will be different operations. When the collection model collects different operations, it is necessary to determine whether the corresponding steps have been performed and make corresponding operations.

[0090] See also Figure 7-1 , Figure 7-2 The present application provides another specific embodiment of a web page information classification method, which includes: S701, using Python's Scrapy library to obtain static data in the target web page; S702. Configure Python development environment and Selenium automation tool; S703. In the Python environment, configure the Web Driver browser driver; S704, initializing the acquisition model; S705, open the configured webpage start address, and wait for the webpage to load completely; S706: Determine whether there is an input event: S707: If it is determined that there is an input event, the input events are executed in sequence, and a confirm search button is simulated to complete the search; S708: If not, execute step S709.

[0091] After the acquisition model is initialized, the web page is opened. After the web page is loaded, the acquisition model starts to collect dynamic data of the web page information and starts to determine whether there is an input event. If the acquisition model determines that an input event has occurred, it will execute the input events in sequence and simulate the user clicking the confirm search button to complete the retrieval and then proceed to step S709; if the acquisition model determines that there is no input event, proceed to the next step.

[0092] S709: Determine whether the user has configured a screening condition: If it is determined that the user has configured a filtering condition, the click operation is performed in sequence according to the position information of the click event input by the user; S710: If not, execute step S712.

[0093] After executing step S708, the collection model determines whether there are user-configured filtering conditions. If it is determined that there are user-configured filtering conditions, it will simulate the user's click operations in sequence according to the location information of the click event input by the user; if the collection model determines that there are no user-configured filtering conditions, it will proceed to the next step.

[0094] S711. Determine whether the user has configured a data list item: S712: If it is determined that the user has configured a data list item, the Selenium automation tool automatically clicks each item in the data list and loads the redirected page; S713: If not, directly proceed to the content extraction step.

[0095] After executing step S710, the acquisition model determines whether there are data list items configured by the user. If it is determined that there are data list items configured by the user, the Selenium automation tool automatically clicks each item in the data list and loads the page after the jump; if it is determined that there are no data list items configured by the user, the data extraction step is performed.

[0096] S714: Extract the content of the dynamic data of the target web page according to the user configuration.

[0097] After the collection model completes the judgment event, it executes the corresponding steps according to the judgment result, uses the Selenium automation tool to simulate the user's operation, and the collection model collects dynamic data through the Selenium automation tool to simulate the operation.

[0098] S715, constructing a DOM tree structure based on the static data and the dynamic data; S716, using the XLM-RoBERTa pre-trained language model to vectorize the static data and the dynamic data in the constructed DOM tree structure into numerical vectors; S717, converting the DOM tree structure into a graph structure, and inputting the numerical vector into a convolutional graph neural network model, encoding the graph structure through the convolutional graph neural network model, and generating a feature vector; S718: Input the feature vector into a preconfigured classification model for calculation, and classify the static data and the dynamic data using the preconfigured classification model.

[0099] The beneficial effect of this embodiment is that the Scrapy library is used to crawl static data, which can efficiently obtain directly visible information on the web page, such as text, picture links, etc., to ensure the rapid acquisition of data. Combining the Selenium automation tool to process dynamic data, it can simulate the operation behavior of real users and effectively deal with complex web page interactions, thereby ensuring the integrity and accuracy of the data, supporting users to customize screening conditions and data list items, making the data collection process more flexible, and being able to adjust the collection strategy according to different needs to meet diverse data collection needs. It allows users to configure input events and click events during the data collection process, simulating user input and search behavior, and further enhancing the flexibility and depth of data acquisition.

[0100] Embodiment eight: See also Figure 8 , the present application provides a web page information classification device, comprising: The first acquisition unit 801 uses the Scrapy library of Python to acquire static data in the target web page from the web page; The second acquisition unit 802 uses the Selenium automation tool to acquire dynamic data of the webpage from the target webpage; A first construction unit 803 constructs a DOM tree structure based on the static data and the dynamic data; A numerical vector conversion unit 804 uses an XLM-RoBERTa pre-trained language model to vectorize the static data and the dynamic data in the constructed DOM tree structure and convert them into numerical vectors; The encoding unit 805 converts the DOM tree structure into a graph structure, and inputs the numerical vector into a convolutional graph neural network model, encodes the graph structure through the convolutional graph neural network model, and generates a feature vector; The classification unit 806 inputs the feature vector into a preconfigured classification model for calculation, and classifies the static data and the dynamic data through the preconfigured classification model.

[0101] Optionally, a pre-processing unit 807 is further included, which is specifically used for: The acquired static data and the dynamic data are preprocessed.

[0102] Optionally, the preprocessing unit 807 is specifically configured to: Use the BeautifulSoup4 toolkit to perform a depth-first traversal of the DOM tree for the static data and the dynamic data, and remove HTML tags and scripts.

[0103] Optionally, the encoding unit 805 is specifically used for: Each node in the DOM tree structure and the text in the node are defined as a node in the graph; the parent-child relationship between the nodes is determined as a directed edge in the graph, each edge connects a parent node and one or more child nodes, and a graph structure is obtained.

[0104] Optionally, the method further includes a second constructing unit 808, which is specifically used to: Build an initial classification model; Input the labeled sample data set into the initialization classification model; Calculate the classification score zic for each sample i and each category c through the initialized classification model; Use the cross entropy loss function to calculate the difference between the probability distribution predicted by the model and the label; Based on the difference, the initialized classification model is optimized using a preconfigured optimization algorithm until a converged classification model is obtained.

[0105] Optionally, the preset algorithm is a softmax classification algorithm, and the softmax classification algorithm is defined as: ; Among them, c represents the classification category, represents the probability that sample i belongs to category c,

[0106] Represents the exponential sum of all category scores.

[0107] Optionally, the Calculate using the following formula: ; in, represents the classification score, represents the weight vector of the category, represents the category bias, The feature vector representing the sample.

[0108] Optionally, the cross entropy loss function is calculated by the following formula: ; in, is a binary indicator representing the indicator variable, represents the loss value of the sample, c represents the index or total number of categories, represents the logarithmic function, Represents the output probability of the category after classification score.

[0109] Optionally, the second acquiring unit 808 is specifically configured to: Configure Python development environment and Selenium automation tool; In the Python environment, configure the Web Driver browser driver; A collection model is constructed, and dynamic data of the target web page is acquired through the collection model.

[0110] Optionally, the second acquisition unit 808 is specifically configured to: Initializing the acquisition model; Open the configured web page start address and wait for the web page to load; Determine whether there is an input event: If it is determined that there is an input event, the input events are executed in sequence, and the confirm search button is simulated to complete the search; Determine whether the user has configured filtering conditions: If it is determined that the user has configured a filtering condition, the click operation is performed in sequence according to the position information of the click event input by the user; Determine whether the user has configured a data list item: If it is determined that the user has configured a data list item, the Selenium automation tool automatically clicks each item in the data list and loads the page after the jump; According to user configuration, the content of dynamic data of the target web page is extracted.

[0111] Optionally, the preconfigured optimization algorithm includes: Calculate the gradient of the loss with respect to the model parameters and update the weights and biases based on the gradient, where the update is performed as follows: ; ; in represents the learning rate, represents the parameter value before weight iteration, represents the parameter value before the bias iteration, represents iteration, represents the value after weight iteration, represents the value of the bias term after iteration, represents the partial derivative, represents the partial derivative of the loss with respect to the weight, represents the partial derivative of the loss with respect to the bias.

[0112] Embodiment nine: See also Fig. 9 The present application also provides a web page information classification device, including: Processor 901, memory 902, input and output unit 903, bus 904; The processor 901 is connected to the memory 902, the input and output unit 903 and the bus 904; The memory 902 stores a program, and the processor 901 calls the program to execute any of the above methods. The present application also relates to a computer-readable storage medium, on which a program is stored, wherein when the program is run on a computer, the computer executes any of the above methods.

[0113] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0114] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0115] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0116] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0117] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk and other media that can store program code.

Claims

1. A web page information classification method, characterized in that: The method comprises: Use Python's Scrapy library to obtain static data from the target web page; Use Selenium automation tool to obtain dynamic data of web pages from target web pages; Constructing a DOM tree structure based on the static data and the dynamic data; Vectorize the static data and the dynamic data in the constructed DOM tree structure using the XLM-RoBERTa pre-trained language model and convert them into numerical vectors; The DOM tree structure is converted into a graph structure, and the numerical vector is input into the convolutional graph neural network model. The graph structure is encoded by the convolutional graph neural network model to generate a feature vector. Inputting the feature vector into a preconfigured classification model for calculation, and classifying the static data and the dynamic data by the preconfigured classification model; The method of using the Selenium automation tool to obtain dynamic data of a web page from a target web page includes: Configure Python development environment and Selenium automation tool; In the Python environment, configure the Web Driver browser driver; Constructing a collection model, and acquiring dynamic data of the target web page through the collection model; The step of constructing a collection model and acquiring dynamic data of the target webpage through the collection model includes: Initializing the acquisition model; Open the configured web page start address and wait for the web page to load; Determine whether there is an input event: If it is determined that there is an input event, the input events are executed in sequence, and the confirm search button is simulated to complete the search; Determine whether the user has configured filtering conditions: If it is determined that the user has configured a filtering condition, the click operation is performed in sequence according to the position information of the click event input by the user; Determine whether the user has configured a data list item: If it is determined that the user has configured a data list item, the Selenium automation tool automatically clicks each item in the data list and loads the page after the jump; According to user configuration, the content of dynamic data of the target web page is extracted.

2. The web page information classification method according to claim 1, characterized in that: Before constructing the DOM tree structure based on the static data and the dynamic data, the method further includes: The acquired static data and the dynamic data are preprocessed.

3. The web page information classification method according to claim 2, characterized in that: Preprocessing the acquired static data and the dynamic data includes: Use the BeautifulSoup4 toolkit to perform a depth-first traversal of the DOM tree for the static data and the dynamic data, and remove HTML tags and scripts.

4. The web page information classification method according to claim 1, characterized in that: The converting of the DOM tree structure into a graph structure comprises: Define each node in the DOM tree structure and the text in the node as a node in the graph; The parent-child relationship between nodes is determined as directed edges in the graph, and each edge connects a parent node and one or more child nodes to obtain a graph structure.

5. The web page information classification method according to claim 1, characterized in that: The feature vector is input into a preconfigured classification model for calculation, and before the static data and the dynamic data are classified by the preconfigured classification model; after the DOM tree structure is converted into a graph structure, and the numerical vector is input into a convolutional graph neural network model, and the graph structure is encoded by the convolutional graph neural network model to generate a feature vector, the method further includes: Build an initial classification model; Input the labeled sample data set into the initialization classification model; Calculate the classification score zic for each sample i and each category c through the initialized classification model; Use the cross entropy loss function to calculate the difference between the probability distribution predicted by the model and the label; Based on the difference, the initialized classification model is optimized using a preconfigured optimization algorithm until a converged classification model is obtained.

6. The web page information classification method according to claim 5, characterized in that: The classification model is configured with a softmax classification algorithm, which is defined as: ; Among them, c represents the classification category, represents the probability that sample i belongs to category c, Represents the exponential sum of all category scores.

7. The web page information classification method according to claim 5, characterized in that: Said Calculate using the following formula: ; in, represents the classification score, represents the weight vector of the category, represents the category bias, The feature vector representing the sample.

8. The web page information classification method according to claim 5, characterized in that: The cross entropy loss function is calculated by the following formula: ; in, is a binary indicator representing the indicator variable, represents the loss value of the sample, c represents the index or total number of categories, represents the logarithmic function, Represents the output probability of the category after classification score.

9. The web page information classification method according to claim 6, characterized in that: The preconfigured optimization algorithms include: Calculate the gradient of the loss with respect to the model parameters and update the weights and biases based on the gradient, where the update is performed as follows: ; ; in represents the learning rate, represents the parameter value before weight iteration, represents the parameter value before the bias iteration, represents iteration, represents the value after weight iteration, represents the value of the bias term after iteration, represents the partial derivative, represents the partial derivative of the loss with respect to the weight, represents the partial derivative of the loss with respect to the bias.

10. A web page information classification device, characterized in that: The device comprises: The first acquisition unit uses the Python Scrapy library to acquire static data from the target web page; The second acquisition unit uses the Selenium automation tool to acquire dynamic data of the web page from the target web page; A first construction unit constructs a DOM tree structure based on the static data and the dynamic data; A numerical vector conversion unit, which uses an XLM-RoBERTa pre-trained language model to vectorize the static data and the dynamic data in the constructed DOM tree structure and convert them into numerical vectors; The encoding unit converts the DOM tree structure into a graph structure, and inputs the numerical vector into the convolutional graph neural network model, encodes the graph structure through the convolutional graph neural network model, and generates a feature vector; A classification unit, inputting the feature vector into a preconfigured classification model for calculation, and classifying the static data and the dynamic data through the preconfigured classification model; The second acquisition unit is specifically used for: Configure Python development environment and Selenium automation tool; In the Python environment, configure the Web Driver browser driver; Constructing a collection model, and acquiring dynamic data of the target web page through the collection model; The second acquisition unit is specifically used for: Initializing the acquisition model; Open the configured web page start address and wait for the web page to load; Determine whether there is an input event: If it is determined that there is an input event, the input events are executed in sequence, and the confirm search button is simulated to complete the search; Determine whether the user has configured filtering conditions: If it is determined that the user has configured a filtering condition, the click operation is performed in sequence according to the position information of the click event input by the user; Determine whether the user has configured a data list item: If it is determined that the user has configured a data list item, the Selenium automation tool automatically clicks each item in the data list and loads the page after the jump; According to user configuration, the content of dynamic data of the target web page is extracted.

11. A web page information classification device, characterized in that: The device comprises: Processor, memory, input-output unit, and bus; The processor is connected to the memory, the input and output unit, and the bus; The memory stores a program, and the processor calls the program to execute the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having a program stored thereon, wherein the program, when executed on a computer, performs the method according to any one of claims 1 to 9.

Citation Information

Patent Citations

  • Website list page classification method and system and storage medium

    CN112287272A

  • Website list page classification method and system and storage medium

    CN112287274A

  • GCN-based significant event trend prediction method

    CN112989060A

  • Webpage graph convolution document sorting method and system based on comparative learning

    CN114817639A

  • Power equipment defect detection method and device, computer equipment and storage medium

    CN116503734A

Cited By

  • Cross-platform burying-point-free data acquisition and intelligent circle selection rule generation method and system

    CN119537158A