Page classification method, device, equipment, storage medium and product

Abstract

The application discloses a page classification method, device, equipment, storage medium and product, and belongs to the computer technical field.The method comprises the following steps: obtaining a to-be-processed page; performing text classification processing on text information to obtain a text classification result; performing page jump information detection processing on the to-be-processed page to obtain a detection result; in the case that the text classification result indicates that the text information is a target text type and the detection result indicates that the to-be-processed page comprises page jump information, determining that a page type corresponding to the to-be-processed page is a target page type.The technical scheme provided by the embodiment of the application can obtain the text classification result and the detection result by performing the text classification processing and the page jump information detection processing on the to-be-processed page, and if the text classification result indicates that the text information is the target text type and the detection result indicates that the to-be-processed page comprises the page jump information, it is determined that the to-be-processed page is the target page type, thereby effectively improving the accuracy of webpage classification.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a page classification method, apparatus, device, storage medium, and product. Background Technology

[0002] With the development of digital and internet technologies, the types of web pages and the multimedia content disseminated within them have become increasingly rich and diverse. To monitor and manage these web pages, it is typically necessary to categorize them.

[0003] In related technologies, the first step is to collect unimodal information from web pages, and then detect and identify the collected unimodal information to achieve web page classification. For example, for web pages containing text-based content such as articles or copywriting, such as advertising articles, the web page type can be determined by classifying the text within the web page.

[0004] In related technologies, the modality of web page information used to determine web page type is singular, resulting in low accuracy in web page classification. Summary of the Invention

[0005] This application provides a page classification method, apparatus, device, storage medium, and product that can effectively improve the accuracy of page classification.

[0006] According to one aspect of the embodiments of this application, a page classification method is provided, the method comprising:

[0007] Obtain the page to be processed, which includes text information;

[0008] The text information is subjected to text classification processing to obtain the text classification result;

[0009] The page to be processed is subjected to page redirection information detection processing to obtain the detection result;

[0010] If the text classification result indicates that the text information is the target text type, and the detection result indicates that the page to be processed includes page jump information, then the page type corresponding to the page to be processed is determined to be the target page type.

[0011] According to one aspect of the embodiments of this application, a page sorting apparatus is provided, the apparatus comprising:

[0012] The page acquisition module is used to acquire the page to be processed, which includes text information;

[0013] The text classification module is used to perform text classification processing on the text information to obtain the text classification result;

[0014] The jump information detection module is used to detect and process page jump information on the page to be processed, and obtain the detection result;

[0015] The page type determination module is used to determine the page type corresponding to the page to be processed as the target page type when the text classification result indicates that the text information is the target text type and the detection result indicates that the page to be processed includes page jump information.

[0016] According to one aspect of the embodiments of this application, a computer device is provided, the computer device including a processor and a memory, the memory storing at least one instruction, at least one program, code set or instruction set, the at least one instruction, the at least one program, the code set or instruction set being loaded and executed by the processor to implement the above-described page classification method.

[0017] According to one aspect of the embodiments of this application, a computer-readable storage medium is provided, wherein at least one instruction, at least one program, code set, or instruction set is stored in the storage medium, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the above-described page classification method.

[0018] According to one aspect of this application, a computer program product is provided, the computer program product including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform an action to implement the page classification method described above.

[0019] The technical solution provided in this application can bring the following beneficial effects:

[0020] By performing text classification and page redirection information detection on the page to be processed, we can obtain the text classification results corresponding to the text information in the page to be processed, as well as the detection results that indicate whether the page to be processed includes page redirection information. Then, based on the above two results, we can determine the page type of the page to be processed, enriching the analysis dimensions of page classification. When the text classification results indicate that the text information is the target text type and the detection results indicate that the page to be processed includes page redirection information, we can determine that the page to be processed is the target page type, effectively improving the accuracy of page classification. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of an application runtime environment provided in one embodiment of this application;

[0023] Figure 2 This is a flowchart of a page classification method provided in one embodiment of this application. Figure 1 ;

[0024] Figure 3 This is a flowchart of a page classification method provided in one embodiment of this application. Figure 2 ;

[0025] Figure 4 An example is shown in the network structure diagram of a text classification model;

[0026] Figure 5 An exemplary diagram illustrates a process for text classification based on a text classification model.

[0027] Figure 6 This is a flowchart of a page classification method provided in one embodiment of this application. Figure 3 ;

[0028] Figure 7 An exemplary schematic diagram of a graphic code detection model is shown;

[0029] Figure 8 This is a flowchart of a training method for a graphic code detection model provided in one embodiment of this application;

[0030] Figure 9 This is a flowchart of a page classification method provided in one embodiment of this application. Figure 4 ;

[0031] Figure 10 An exemplary schematic diagram illustrates the technical framework corresponding to a page classification method provided in one embodiment of this application;

[0032] Figure 11 An exemplary diagram illustrates an advertising article recognition process implemented based on the page classification method provided in an embodiment of this application;

[0033] Figure 12 An exemplary schematic diagram of a page image to be processed is shown;

[0034] Figure 13This is a block diagram of a page classification device provided in one embodiment of this application;

[0035] Figure 14 This is a structural block diagram of a computer device provided in one embodiment of this application. Detailed Implementation

[0036] The page classification method provided in this application involves artificial intelligence technology, which will be briefly described below to facilitate understanding by those skilled in the art.

[0037] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0038] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0039] Computer vision (CV) is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in recognizing and measuring targets, and further processes images to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, OCR (Optical Character Recognition), video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0040] Natural Language Processing (NLP) is an important field within computer science and artificial intelligence. It studies the theories and methods for enabling effective communication between humans and computers using natural language. NLP is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field involves natural language—the language people use in daily life—and thus it has a close relationship with linguistic research. NLP techniques typically include text processing, semantic understanding, machine translation, question answering, and knowledge graphs.

[0041] Machine learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning.

[0042] With the research and advancement of artificial intelligence (AI) technology, AI is being studied and applied in various fields, such as smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, autonomous driving, drones, robots, smart healthcare, and smart customer service. It is believed that with the development of technology, AI will be applied in more fields and play an increasingly important role.

[0043] The solutions provided in this application involve artificial intelligence technologies such as computer vision, natural language processing, and machine learning, which are specifically illustrated through the following embodiments.

[0044] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0045] Please refer to Figure 1 This diagram illustrates an application runtime environment provided in one embodiment of this application. The application runtime environment may include: terminal 10 and server 20.

[0046] Terminal 10 includes, but is not limited to, electronic devices such as mobile phones, computers, smart voice interaction devices, smart home appliances, in-vehicle terminals, aircraft, game consoles, e-book readers, multimedia playback devices, and wearable devices. Application clients can be installed on terminal 10.

[0047] In this embodiment, the application can be any application capable of providing web page services. Examples include news feed applications, video applications, news applications, social networking applications, interactive entertainment applications, browser applications, shopping applications, content sharing applications, virtual reality (VR) applications, and augmented reality (AR) applications. This embodiment does not limit the specific application. Furthermore, different applications may involve different web pages and have different functions, which can be pre-configured according to actual needs. This embodiment does not limit the specific application. Optionally, the terminal 10 runs a client application of the aforementioned application.

[0048] Server 20 provides background services for clients of applications in terminal 10. For example, server 20 can be a background server for the aforementioned applications. Server 20 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Optionally, server 20 provides background services for applications in multiple terminals 10 simultaneously. Optionally, server 20 has a web page management application installed. The web page classification method provided in this application embodiment can be applied to the aforementioned web page management application to classify various web pages in the server background. Optionally, the aforementioned web page management system includes an internet advertising system. The web page classification method provided in this application embodiment can be applied to the internet advertising system to identify promotional web pages on the internet.

[0049] Optionally, terminal 10 and server 20 can communicate with each other via network 30. Terminal 10 and server 20 can be directly or indirectly connected via wired or wireless communication, which is not limited herein.

[0050] Please refer to Figure 2 It illustrates the flow of a page classification method provided in one embodiment of this application. Figure 1 This method can be applied to computer devices, which refer to electronic devices with data computing and processing capabilities. For example, the entity executing each step can be... Figure 1 The application runtime environment shown is server 20. The method may include the following steps (210-240).

[0051] Step 210: Obtain the page to be processed.

[0052] In practical applications, there are various ways to obtain the page to be processed, and this application embodiment does not limit this. In an exemplary embodiment, the page to be processed can be obtained from the list of page task pairs to be processed. The aforementioned page task processing list includes at least one page address information corresponding to the page to be processed. Based on the page address information, the page to be processed can be accessed, and the page resource data (content) corresponding to the page to be processed can be obtained.

[0053] In one possible implementation, the static web crawler library requests can be used to access the webpage to be processed and obtain its content. requests is a simple and easy-to-use HTTP (Hypertext Transfer Protocol) library implemented in Python, which can be used to access webpages to obtain text and image materials.

[0054] Optionally, the page to be processed includes text information. This text information includes, but is not limited to, the page text corresponding to the page to be processed, image recognition text, and audio transcription text.

[0055] In one possible implementation, the page text can be directly obtained from the page data of the page to be processed.

[0056] In one possible implementation, by acquiring image information of the page to be processed, including at least one of page images, page videos, and page screenshots, the image information can be subjected to OCR processing to obtain the aforementioned image-recognized text.

[0057] In one possible implementation, by acquiring the audio information of the page to be processed, which includes at least one of the page's audio and audio from the video, speech recognition processing can be performed on the audio information to obtain the aforementioned audio-transcribed text.

[0058] Optionally, the page to be processed includes images. These images include, but are not limited to, the original image on the page to be processed and screenshots obtained by taking a screenshot of the page.

[0059] Step 220: Perform text classification processing on the text information to obtain the text classification result.

[0060] The above text classification results represent the text type corresponding to the text information.

[0061] In an exemplary embodiment, such as Figure 3 As shown, the implementation process of step 220 above includes the following step 221. Figure 3 The flowchart of a page classification method provided in one embodiment of this application is shown. Figure 2 。

[0062] Step 221: Input the text information into the text classification model and output the text classification result.

[0063] The text classification model is a machine learning model trained based on sample texts and label information. The label information represents the text type to which the sample text belongs, and the label information is determined based on the correspondence between the sample text and the keyword library. The keyword library stores at least one keyword related to the text type.

[0064] In a possible implementation, the sample text is tokenized to obtain a tokenization result. By comparing the words in the tokenization result with the keywords stored in the keyword library, the overlapping words between the sample text and the keyword library can be obtained. The overlapping words can represent the correspondence between the sample text and the keyword library. Obtain the text type corresponding to the overlapping words. Based on the text type corresponding to the overlapping words, the label information corresponding to the sample text can be determined.

[0065] For example, the keyword library includes at least one keyword corresponding to the target text type. If the sample text includes at least one keyword corresponding to the target text type, the sample text can be initially determined as a positive sample; otherwise, it is initially determined as a negative sample. The label information can represent the type of the sample text. For example, the label information being 1 represents that the sample text is a positive sample, and being 0 represents that the sample text is a negative sample. The initially determined positive and negative samples can also be further reviewed and verified, but they can also be directly used as the final positive and negative samples for training. This application embodiment does not limit this.

[0066] Based on the above positive and negative samples, the text classification model to be trained can be trained to obtain a trained text classification model. This text classification model can identify whether the text information is of the target text type. Optionally, the target text type includes a promotional text type, etc. The target text type can be set according to the specific application scenario. This application embodiment does not limit the specific type of the target text type.

[0067] During the training process of the text classification model, some words will add noise to the model and affect the recognition accuracy of the model. At this time, it is necessary to preprocess the original sample text and set stop words. Setting stop words can effectively reduce the interference to the model. Stop words include "of", "is", "this", "that", etc.

[0068] Optionally, by filtering out the stop words in the tokenization result, the sample noise can be reduced to obtain a filtered tokenization result.

[0069] In one possible implementation, the training process of the above text classification model is as follows: Each corpus unit in the word segmentation results is subjected to word embedding processing to obtain the word vector corresponding to each corpus unit. For example, each corpus unit is input into a word embedding model for word embedding processing to obtain the word vector corresponding to each corpus unit.

[0070] Word embedding models used to generate word vectors are generally divided into two types: CBOW (Continuous Bag-of-Words) and Skip-Gram. The CBOW model takes the word vectors of the context words of a given feature word as its training input, and outputs the word vector of that specific word. The Skip-Gram model works in reverse; its input is the word vector of a specific word, and its output is the context word vectors of that specific word. CBOW is more suitable for small databases, while Skip-Gram performs better on large corpora. The appropriate word embedding model can be selected based on the actual scenario, or a fusion model of the two can be used. The word embedding model selected in this embodiment has a word vector library of 8 million words, basically covering most Chinese character scenarios.

[0071] After obtaining the word vectors corresponding to each of the above corpus units, the word vectors are input into the text classification model to be trained, thereby outputting the text classification result corresponding to the sample text. Based on the text classification result and label information, the model loss information can be determined. Then, the parameters of the text classification model to be trained are adjusted according to the model loss information, and finally the trained text classification model is obtained.

[0072] In one possible application scenario, the aforementioned keyword library includes an advertising content keyword library. This advertising content keyword library includes keywords related to the promotional text type. For example, the advertising content keyword library includes, but is not limited to, the following keywords: direct purchase, fan special price, buy one get one free, order within 48 hours, 5 yuan coupon, 50% off limited-time offer, cost price, discreet shipping, free shipping, tax-exclusive price, no counterfeit goods, free shipping and refund if unsatisfied, no-reason return and refund if unsatisfied, subsidized price, long press the QR code to follow and purchase, long press to identify the QR code to grab a deal, long press to identify the QR code below to purchase, long press to identify the QR code below to purchase, refund guarantee, continuously hot-selling, lowest price, as low as 3.9 yuan with free shipping, as low as 33% off, click to buy, click the image to buy now, click the image below to buy, click the image below to buy, click the mini-program to buy now, multiple boxes stand Discounts, buy more get more free, buy more get more discounts, shipping location, shipping time, fan benefit price, fan price, fan special offer, fan group purchase price, fan exclusive benefit price, screenshot of successful payment, benefit price, available in all stores, free gift with purchase, how to buy, official retail price, official original price, follow to buy, welcome everyone to grab it, promotional price, buy now, buy immediately, price at the lowest, lucky draw special price, buy today, only available, please buy with confidence, huge discounts, absolutely great value for money, delivery time is not guaranteed, buy now, grab it now, order now, grab the hot item instantly, buy 2 boxes and get 10 yuan off, buy 2 boxes and get 20 yuan off, buy 2 bottles and get 20 yuan off, buy 2 get 1 free, buy 2 get 2 free. Buy two boxes and get an extra 10 yuan off, buy two get one free, the more you buy, the more you save, buy-one-get-one-free promotion, current discount price, internal purchase price, group purchase discount, platform subsidized price, grab it while you can, grab it directly, grab the lowest price on the entire network, buy it on the app, 33% off sitewide, full refund, free shipping nationwide, shipping within three months, scan to buy, scan the code or click the picture to buy, price before upgrade, subscribe to the course now, it's really too cost-effective, first purchase, free shipping nationwide for the first order, set discount, special offer price, unified price, no reason to return or exchange, no reason to return, no reason to refund, ordering users, current promotional price, in stock and ready to ship, limited time 15% off, limited time fan reward price, limited time check-in cashback activity, limited time special offer, limited Special offers, try before you buy, group purchase discounts for course enrollment now, buy now and enjoy a limited-time 50% off exclusive benefits for fans, best-selling items, sales sets, new product special prices, exclusive benefits for newcomers, genuine medicines at fair prices, subject to final release, discounted prices, priority purchase, grab discounts while supplies last, original price, further 50% off, no-reason refunds, direct discount of 20 yuan, only the price of 2 cups of coffee, main price range, exclusive mega-deals, combo discounts, maximum discounts, lowest final price, maximum cashback, save up to 80 yuan, most affordable prices, set discounts, order at your discretion, special offers, discounted prices, how to buy, buy now, cash on delivery, shipping notification, free shipping, submit your order, buy now, no-reason returns and exchanges.

[0073] After the above text classification model is trained, it can be used to classify the text information input to the text classification model. The specific process includes: segmenting the text information into words to obtain the segmentation results; embedding each corpus unit in the segmentation results, such as inputting a word embedding model, outputting word vectors corresponding to each corpus unit, and performing feature extraction on each word vector to obtain the text feature vectors corresponding to the text information; then performing pooling and logistic regression on the text feature vectors to obtain the text classification results corresponding to the text information. The text classification results represent the probability that the text information belongs to each text type, including the text type data corresponding to the target text type, which can represent the probability that the text information is the target text type.

[0074] In one example, such as Figure 4 As shown, it exemplifies the network structure diagram of a text classification model. Figure 4 This paper illustrates the network structure of a text classification model—TextCNN (Text Convolutional Neural Networks)—which includes an input layer, convolutional layers, pooling layers, and fully connected layers. The input layer receives the input text information and outputs the corresponding text representation information, such as an n×k dimensional sentence matrix representing each sentence in the text on both static and non-static channels. This sentence matrix is ​​determined based on the word vectors corresponding to each corpus unit in the sentence. Here, k represents the dimension of the corpus unit representation vector, and n represents the number of corpus units in the input sentence. The convolutional layers are convolutional layers with filters of various widths and various feature maps. Based on these convolutional layers, feature extraction processing can be performed on the text representation information to obtain text feature vectors. These text feature vectors can be input to the pooling layer for pooling processing, thereby obtaining a fixed-length representation vector corresponding to the text information. Optionally, the pooling processing includes max-over-time pooling. The fixed-length representation vector output by the pooling layer can be further input into a fully connected layer, such as a fully connected layer with dropout and softmax outputs, to obtain the text classification result described above.

[0075] In one example, such as Figure 5 As shown, this example illustrates a flowchart of text classification processing based on a text classification model. The specific process is as follows:

[0076] 1. Embedding (word embedding processing): For the input sentence "I like this movie", the input layer of the text classification model will output... Figure 5 The leftmost 7x5 matrix contains word vectors, each row of which is a word vector with a dimension of 5, analogous to the raw pixels in an image.

[0077] 2. Convolution (feature extraction): The statement matrix is ​​processed by a one-dimensional convolutional layer with kernel_sizes (operator size, region size) = (2, 3, 4). Each kernel_size corresponds to 2 filters, for a total of 6 filters. Thus, each kernel_size corresponds to two output channels, and finally, 6 feature maps corresponding to the statement matrix can be output.

[0078] 3. Max Pooling: The third layer of the text classification model is a 1-max pooling layer. This allows sentences of different lengths to be transformed into fixed-length representations after passing through the pooling layer, resulting in the fixed-length representation vectors mentioned above.

[0079] 4. FullConnection and Softmax: The last layer of the text classification model is a fully connected softmax layer, which outputs the probability that the input sentence belongs to each text type.

[0080] This embodiment of the application can directly output text classification results by inputting the text information in the webpage to be processed into a trained text classification model, which can effectively improve the efficiency and accuracy of text classification. Furthermore, the keyword library in this embodiment of the application stores keywords related to text type. Therefore, the label information of the sample text corresponding to the above text classification model can be determined based on the correspondence between the sample text and the keyword library, which effectively reduces the complexity of determining label information, thereby reducing the complexity of model training and improving the accuracy of text classification.

[0081] Step 230: Perform page redirection information detection processing on the page to be processed to obtain the detection result.

[0082] Optionally, the aforementioned page redirection information refers to the redirection information corresponding to other pages within the page to be processed, excluding the page to be processed. In many practical scenarios, page objects such as images, controls, and hyperlinks can correspond to the address information of other pages. Users can jump from the page to be processed to other pages by clicking controls, clicking hyperlinks, or recognizing the page address information in images. Therefore, these page objects that correspond to the address information of other pages constitute the page redirection information within the page to be processed.

[0083] In particular, promotional web pages often contain the aforementioned page redirection information. Therefore, by detecting whether the page to be processed contains page redirection information, the page type can be determined.

[0084] Optionally, the above detection results include detection data corresponding to page redirection information, and the detection data represents the probability that the page to be processed includes page redirection information.

[0085] In an exemplary embodiment, such as Figure 3 As shown, the implementation process of step 230 above includes the following step 231.

[0086] Step 231: Perform page object detection processing on the page to be processed to obtain page object detection data.

[0087] Among them, the page object detection data represents the probability that the target page object exists in the page to be processed. The target page object refers to the page object associated with the page jump information. The detection result includes the page object detection data.

[0088] Optionally, the target page object includes, but is not limited to, page objects such as graphic codes, page navigation controls, and hyperlink addresses. Optionally, the graphic code includes QR codes.

[0089] In one possible implementation, page structure data corresponding to the page to be processed is obtained, and it is determined whether the page structure data includes structure data corresponding to a page navigation control. If the page structure data includes structure data corresponding to a page navigation control, it can be determined that the page to be processed includes a page navigation control. Then, preset identifier data can be output as navigation control detection data corresponding to the page navigation control to indicate that the page to be processed includes a page navigation control, that is, it indicates that the page to be processed includes page navigation information.

[0090] In one possible implementation, it can also be determined whether the page structure data includes hyperlink addresses. If the page structure data includes hyperlink addresses, it can be determined that the page to be processed includes hyperlink addresses. Then, preset identifier data can be output as hyperlink detection data to indicate that the page to be processed includes hyperlink addresses, that is, to indicate that the page to be processed includes page redirection information.

[0091] In one possible implementation, it can also be determined whether the page structure data includes an image. If the page structure data includes an image, then step 231a can be performed.

[0092] This application embodiment detects whether there is a target page object associated with page jump information in the page to be processed, and obtains page object detection data that can characterize whether there is a target page object in the page to be processed, thereby realizing the detection of page jump information in the page to be processed and improving the accuracy of page jump information recognition.

[0093] In one possible implementation, the page to be processed includes an image, and the target page object includes a graphic code; accordingly, as Figure 6 As shown, the implementation process of step 231 above may include the following sub-step 231a, Figure 6 The flowchart of a page classification method provided in one embodiment of this application is shown. Figure 3 .

[0094] Step 231a: Perform graphic code detection processing on the image to obtain a graphic code detection score.

[0095] The graphic code detection score represents the probability that a graphic code exists in an image, and the page object detection data includes the graphic code detection score.

[0096] This application embodiment detects whether there is a graphic code associated with page jump information in the page to be processed, thereby improving the accuracy of page jump information recognition.

[0097] In an exemplary embodiment, an image can be input into a graphic code detection model, and the graphic code detection result can be output.

[0098] The graphic code detection result includes the graphic code detection score mentioned above. The graphic code detection model is a machine learning model trained based on sample images and label information. The sample images include synthetic images synthesized based on sample graphic codes and sample background images. The label information represents the actual location information of the sample graphic code in the sample image.

[0099] In one example, such as Figure 7 As shown, it exemplarily illustrates a schematic diagram of a graphic code detection model. Figure 7 The illustrated image code detection model is a machine learning model implemented based on Darknet-53 (a neural network framework). The Darknet-53 model is a fully convolutional network, primarily composed of a series of 1x1 and 3x3 convolutional layers (53 in total). It utilizes convolution operations instead of pooling (which has a negative gradient effect) for downsampling. The corresponding image code detection process is as follows:

[0100] 1. Darknet-53 extracts image features and performs downsampling: The input image is downsampled by 32x, 16x, and 8x respectively, and feature maps at three scales are output.

[0101] 2. Multi-scale feature fusion processing: Darknet-53 outputs feature maps at three scales to detect targets of different sizes, such as QR codes of different sizes. Detection is performed at scales of 32x, 16x, and 8x downsampling, and feature maps of different scales are stitched together using feature fusion (such as upsampling) to improve feature extraction capabilities.

[0102] 3. Predict the target using a fixed window. A fixed window is used as the central detection domain to generate nine prior boxes (i.e., image code detection boxes of different sizes). The prior boxes are used to select the target. Optionally, prior boxes of different sizes are used to detect feature maps of different scales, and the image code detection results are output, including the predicted position of the center point corresponding to the image code, the position and size information of the predicted box, the confidence score, and the type information of the predicted box.

[0103] Optionally, in the context of internet advertising, since the ad frame detection is a small target detection scenario, the prior box size can be re-adapted to 50*50 and the prior box size can be reduced to 1.

[0104] This application embodiment inputs the image from the webpage to be processed into the trained graphic code detection model, which can directly output the graphic code detection result corresponding to the image, effectively improving the efficiency and accuracy of graphic code detection; furthermore, the sample image corresponding to the above graphic code detection model includes a synthetic image synthesized based on the sample graphic code and the sample background image, which can effectively reduce the complexity of obtaining sample images, thereby reducing the complexity of model training.

[0105] In one possible implementation, such as Figure 8 The diagram shows a flowchart of a training method for a graphic code detection model provided in one embodiment of this application. The method may include the following steps (310-340).

[0106] Step 310: Obtain sample images and label information.

[0107] Obtain sample background images and sample graphic codes. Optionally, the sample background images are pre-collected background images, such as 1-2000 background images. Optionally, the sample background images include webpage background images. The purpose of collecting sample background images is to create training samples. To ensure the diversity and generalization ability of the training samples, it is necessary to collect background images that conform to the application scenario. Optionally, the sample graphic codes include pre-collected graphic code images, such as QR code images, used to expand the training data.

[0108] After obtaining the graphic code image, it is necessary to start generating training samples. Optionally, the sample graphic codes and sample background images can be synthesized to obtain a composite image. For example, the sample graphic codes can be randomly pasted onto the sample background image using an Image processing library, and the parameter information of the sample graphic codes, such as transparency, thickness, and size, can be set to generate the above-mentioned composite image, so as to ensure the diversity and fit of the training samples.

[0109] Optionally, mosaic data augmentation is performed on the synthesized image to obtain a composite image. Mosaic data augmentation refers to combining a preset number of synthesized images in a certain proportion to obtain a composite image. This allows the model to learn how to identify objects smaller than normal size. Experiments have shown that this method can improve the accuracy of small target detection.

[0110] The sample image can be obtained based on the synthesized image and the combined image described above; that is, the sample image includes the synthesized image and the combined image. During the process of generating the synthesized image and the combined image, the corresponding label information for each of the synthesized image and the combined image can be automatically generated.

[0111] The label information corresponding to the synthesized image can represent the position information of the sample graphic code in the synthesized image; the label information of the combined image can represent the position information of the sample graphic code in the combined image.

[0112] Optionally, the above sample images are normalized to obtain normalized sample images. Image normalization is a widely used technique in computer vision, pattern recognition, and other fields. Image normalization involves transforming the original image to be processed into a unique standard form through a series of transformations (this standard form image is invariant to affine transformations such as translation, rotation, and scaling). Before training the model, the training and test images need to be normalized. Depending on the actual efficiency, the image size should be set to 320*320 or 460*460. Generally, larger image sizes result in higher accuracy but also higher processing time.

[0113] Step 320: Cluster the sample images based on the label information to obtain the graphic code detection boxes corresponding to the sample images.

[0114] Optionally, based on the positional information of the graphic codes in the sample images, the sample images can be clustered to obtain at least two classes of sample images. The size of the graphic codes in each class of sample images is relatively similar. Therefore, based on the size information of the graphic codes corresponding to each class of sample images, at least two graphic code detection boxes can be determined. These at least two graphic code detection boxes reflect graphic codes of different sizes and can serve as prior information to assist subsequent graphic code detection processing, thereby improving the accuracy of graphic code detection.

[0115] Step 330: Input the sample image, label information, and graphic code detection box into the graphic code detection model to be trained, and output the sample graphic code detection result corresponding to the sample image.

[0116] Optionally, the above sample graphic code detection results include the predicted position of the center point of the graphic code corresponding to the sample image, the predicted bounding box position and size information, the confidence level, and the predicted bounding box type information.

[0117] Step 340: Based on the label information and the sample graphic code detection results, the parameters of the graphic code detection model to be trained are adjusted to obtain the graphic code detection model.

[0118] Based on the above label information and sample graphic code detection results, the model loss information can be determined. Based on the model loss information, the parameters of the graphic code detection model to be trained can be adjusted. When the model loss information meets the preset loss conditions, a well-trained graphic code detection model can be obtained.

[0119] This application embodiment uses label information that characterizes the location information of graphic codes in sample images to cluster the sample images, thereby obtaining graphic code detection boxes corresponding to the sample images. The graphic code detection boxes, sample images, and label information are then input into a graphic code detection model to be trained, outputting the sample graphic code detection results for the sample images. The model parameters of the graphic code detection model to be trained are then adjusted based on the graphic code detection results and label information to obtain a trained graphic code detection model. The graphic code detection boxes are prior information corresponding to the sample graphic codes obtained from clustering, which can assist the graphic code detection model in graphic code detection and improve the accuracy of graphic code detection.

[0120] Optionally, after inputting the above image into the trained graphic code detection model, the graphic code detection model can perform graphic code detection processing on the input image, thereby outputting the graphic code detection result. Accordingly, as... Figure 9 As shown, the implementation process of step 231a above includes the following sub-steps (231a1 to 231a3), Figure 9 The flowchart of a page classification method provided in one embodiment of this application is shown. Figure 4 .

[0121] Step 231a1: Perform feature extraction processing on the image to obtain the feature map corresponding to the image.

[0122] Optionally, by inputting the image into the convolutional layer of the image code detection model for at least two downsampling feature extraction processes, feature maps of at least two sizes corresponding to the image can be obtained.

[0123] Optionally, based on the size difference information between each pair of feature maps of at least two sizes, the upsampling factor can be determined, and then the feature maps of different sizes can be fused according to the upsampling factor to obtain the fused feature map.

[0124] Step 231a2: Obtain the graphic code detection box.

[0125] The graphic code detection box represents the graphic code detection region corresponding to the image. Optionally, the graphic code detection box is a prior box obtained by clustering based on the label information corresponding to the sample image. Optionally, the graphic code detection box includes prior boxes of at least two sizes.

[0126] Step 231a3: The feature map is processed based on the graphic code detection box to obtain the graphic code detection score.

[0127] Optionally, for the target prior box in the prior boxes of at least two sizes, a graphic code detection process is performed on the feature maps (and the fused feature maps) of the at least two sizes based on the target prior box to obtain the graphic code detection result corresponding to the image. This graphic code detection result includes the predicted position of the graphic code center point, the predicted box position and size information, the confidence score, and the predicted box type information. The confidence score can be used as the graphic code detection score, reflecting the probability that the predicted box contains the graphic code. Optionally, the target prior box is the selected prior box among the prior boxes of at least two sizes.

[0128] This application embodiment performs feature extraction processing on the image in the webpage to be processed, which can obtain the corresponding feature map, and obtain the graphic code detection box representing the graphic code detection area to detect the feature map, thereby obtaining the graphic code detection score to represent the probability that the above image contains a graphic code, thus improving the accuracy of graphic code detection.

[0129] In another possible implementation, the target page object includes a page navigation control; accordingly, such as Figure 6 As shown, the implementation process of step 231 above may include the following sub-step 231b.

[0130] Step 231b: Perform page jump control detection processing on the page to be processed and obtain the control detection score.

[0131] Among them, the control detection score represents the probability that a page jump control exists in the page to be processed, and the page object detection data includes the control detection score.

[0132] In one possible implementation, a screenshot of the page to be processed can be obtained, and then the screenshot can be input into a page navigation control detection model for page navigation control detection processing to obtain the control detection result. The control detection result may include the aforementioned control detection score.

[0133] The training of the page navigation control detection model described above is similar to that applied to the image code detection model, except that the sample images used in both differ from the input images. Similar to the image code detection model, the page navigation control detection model is also a machine learning model trained based on sample images and label information. However, the sample images used in the page navigation control detection model include synthetic images synthesized from sample page navigation controls and sample background images. The label information used in the page navigation control detection model represents the actual position of the sample page navigation control within the sample image.

[0134] In one possible implementation, the training process of the above-mentioned page navigation control detection model is as follows:

[0135] Obtain sample images and label information. First, obtain sample background images and sample page navigation controls. Optionally, the sample background image corresponding to the page navigation control detection model and the sample background image corresponding to the image code detection model can be the same or different. Optionally, the above sample background images include webpage background images. The purpose of collecting sample background images is to create training samples. To ensure the diversity and generalization ability of the training samples, it is necessary to collect background images that conform to the application scenario. Optionally, the above sample page navigation controls include pre-collected page navigation control images, such as images corresponding to mini-program controls, used to expand the training data.

[0136] After obtaining the page navigation control image, it is necessary to start generating training samples. Optionally, the sample page navigation control and the sample background image can be composited to obtain a composite image. For example, the sample page navigation control can be randomly pasted onto the sample background image using an Image processing library, and the parameter information of the sample page navigation control, such as transparency, thickness, and size, can be set to generate the above composite image, so as to ensure the diversity and fit of the training samples.

[0137] Optionally, mosaic data augmentation is performed on the synthesized image to obtain a composite image. Mosaic data augmentation refers to combining a preset number of synthesized images in a certain proportion to obtain a composite image. This allows the model to learn how to identify objects smaller than normal size. Experiments have shown that this method can improve the accuracy of small target detection.

[0138] The sample image can be obtained based on the synthesized image and the combined image described above; that is, the sample image includes the synthesized image and the combined image. During the process of generating the synthesized image and the combined image, the corresponding label information for each of the synthesized image and the combined image can be automatically generated.

[0139] The label information corresponding to the synthesized image can represent the position information of the sample page jump control in the synthesized image; the label information of the combined image can represent the position information of the sample page jump control in the combined image.

[0140] Optionally, the above sample images are normalized to obtain normalized sample images. Image normalization is a widely used technique in computer vision, pattern recognition, and other fields. Image normalization involves transforming the original image to be processed into a unique standard form through a series of transformations (this standard form image is invariant to affine transformations such as translation, rotation, and scaling). Before training the model, the training and test images need to be normalized. Depending on the actual efficiency, the image size should be set to 320*320 or 460*460. Generally, larger image sizes result in higher accuracy but also higher processing time.

[0141] After obtaining the above sample images and label information, the sample images can be clustered based on the label information to obtain the page jump control detection box corresponding to the sample images.

[0142] Optionally, based on the position information of page navigation controls in the sample images, the sample images can be clustered to obtain at least two classes of sample images. The sizes of the sample page navigation controls in each class of sample images are relatively similar. Therefore, based on the size information of the sample page navigation controls corresponding to each class of sample images, at least two types of page navigation control detection boxes can be determined. These at least two types of page navigation control detection boxes reflect page navigation controls of different sizes and can serve as prior information to assist subsequent page navigation control detection processing, thereby improving the accuracy of page navigation control detection.

[0143] Furthermore, the sample images, label information, and page navigation control detection boxes are input into the page navigation control detection model to be trained, and the sample page navigation control detection results corresponding to the sample images are output.

[0144] Optionally, the above-mentioned sample page jump control detection results include the predicted position of the center point of the page jump control corresponding to the sample image, the predicted box position and size information, the confidence level, and the predicted box type information.

[0145] Based on the label information and the detection results of sample page navigation controls, the parameters of the page navigation control detection model to be trained are adjusted to obtain the page navigation control detection model.

[0146] Based on the above label information and the sample page jump control detection results, the model loss information can be determined. Based on the model loss information, the parameters of the page jump control detection model to be trained can be adjusted. When the model loss information meets the preset loss conditions, the trained page jump control detection model can be obtained.

[0147] The trained page navigation control detection model can be deployed to relevant services. In one possible implementation, after inputting a page screenshot into the page navigation control detection model, the model can perform page navigation control detection processing on the input screenshot, thereby outputting the page navigation control detection result. The specific process includes: performing feature extraction processing on the page screenshot to obtain the feature map corresponding to the page screenshot; obtaining the page navigation control detection box; and performing detection processing on the feature map based on the page navigation control detection box to obtain the control detection score.

[0148] Optionally, by inputting the page screenshot into the convolutional layer of the page navigation control detection model for at least two downsampling feature extraction processes, feature maps of at least two sizes corresponding to the page screenshot can be obtained.

[0149] Optionally, based on the size difference information between each pair of feature maps of at least two sizes, the upsampling factor can be determined, and then the feature maps of different sizes can be fused according to the upsampling factor to obtain the fused feature map.

[0150] The page navigation control detection box represents the detection area of ​​the page navigation control corresponding to the page screenshot. Optionally, the above-mentioned page navigation control detection box is a prior box obtained by clustering based on the label information corresponding to the sample image. Optionally, the above-mentioned page navigation control detection box includes prior boxes of at least two sizes.

[0151] For target prior boxes in at least two prior bounding boxes, page jump control detection processing is performed on the feature maps (and fused feature maps) of the at least two sizes based on the target prior boxes to obtain the page jump control detection result corresponding to the page screenshot. This result includes the predicted center point of the page jump control corresponding to the page screenshot, the predicted bounding box position and size information, the confidence score, and the predicted bounding box type information. The confidence score can be used as the control detection score, reflecting the probability that the predicted bounding box corresponding to the confidence score contains the page jump control.

[0152] This application embodiment detects whether there is a page jump control associated with page jump information in the page to be processed, thereby improving the accuracy of page jump information recognition.

[0153] In another possible implementation, a page object detection model can be trained that can detect and identify multiple target page objects, including the aforementioned graphic code detection and page navigation control detection. The model structure of this page object detection model is similar to that of the graphic code detection model and the page navigation control detection model. The training samples corresponding to this page object detection model include the sample images and label information corresponding to the graphic code detection model and the page navigation control detection model, respectively.

[0154] Step 240: If the text classification result indicates that the text information is the target text type, and the detection result indicates that the page to be processed includes page jump information, then determine that the page type corresponding to the page to be processed is the target page type.

[0155] Optionally, the target page type includes promotional types, but this application embodiment does not limit this.

[0156] In one example, such as Figure 10 As shown, it exemplarily illustrates a schematic diagram of the technical framework corresponding to a page classification method provided in an embodiment of this application. Figure 10 The illustrated technical framework includes a database of web pages to be processed, a web page acquisition module, an image recognition module, a text classification module, a model fusion module, and a web page database corresponding to the target page type. The database of web pages to be processed includes at least one web page to be processed. The web page acquisition module is used to retrieve the web page to be processed from the database and to collect images and text from the web page. The text classification module performs step 220, the image recognition module performs step 230, and the model fusion module determines the web page type based on the output text classification and detection results. In the case corresponding to step 240, the web page identified as belonging to the target page type is stored in the web page database corresponding to the target page type.

[0157] In the field of internet advertising, the aforementioned webpage collection module collects webpage articles from the database of webpages to be processed, and uses crawlers to collect text and image materials from the links in the webpage articles. These text and image materials are used for subsequent recognition logic. The text recognition module recognizes the text of the webpage articles and outputs a text recognition score (i.e., the text type data mentioned above). The image recognition module is used to recognize the advertising image materials in the webpage articles and outputs an image recognition score (i.e., the detection data mentioned above). The model fusion module combines the text recognition score and the image recognition score to obtain the final result and updates the advertising advertorial database (when the target page type is promotional, the webpage database corresponding to the target page type is the advertising advertorial database), thus completing the recognition of the advertising advertorial.

[0158] In an exemplary embodiment, the text classification result includes text type data corresponding to the target text type, where the text type data represents the probability that the text information is the target text type; the detection result includes detection data corresponding to the page jump information, where the detection data represents the probability that the page to be processed includes page jump information; accordingly, as Figure 3 As shown, the specific implementation steps of step 240 above may include the following step 241.

[0159] Step 241: If the text type data is greater than the first threshold and the detection data is greater than the second threshold, determine the page type as the target page type.

[0160] The situation where the text type data exceeds the first threshold indicates that the text information is the target text type, and the situation where the detection data exceeds the second threshold indicates that the page to be processed includes page redirection information. Optionally, the first threshold is a threshold set for the text type data, and the second threshold is a threshold set for the detection data. The values ​​of the first threshold and the second threshold can be the same or different.

[0161] Optionally, the above detection data includes, but is not limited to, graphic code detection scores and control detection scores; correspondingly, the above second threshold includes the graphic code detection score threshold corresponding to the graphic code detection score and the control detection score threshold corresponding to the control detection score.

[0162] Optionally, if the text type data is greater than the first threshold and the image code detection score is greater than the image code detection score threshold, or if the text type data is greater than the first threshold and the control detection score is greater than the control detection score threshold, the page type is determined as the target page type.

[0163] The condition that the image code detection score is greater than the image code detection score threshold indicates that the page to be processed contains an image code, and the condition that the control detection score is greater than the control detection score threshold indicates that the page to be processed contains a page jump control.

[0164] In this embodiment, the text classification result includes text type data that can characterize the probability that the text information is the target text type, and the page jump information detection result includes detection data that can characterize the probability that the page to be processed includes page jump information. Therefore, by judging the relationship between the text type data in the text classification result and the first threshold, and the relationship between the detection data in the page jump information detection result and the second threshold, the page type of the page to be processed can be comprehensively judged, which effectively improves the page classification efficiency and page classification accuracy. When the text type data is greater than the first threshold and the detection data is greater than the second threshold, the page type can be determined to be the target page type.

[0165] The page classification method provided in this application embodiment will be described below with reference to specific application scenarios. The page classification method provided in this application embodiment can be applied to the field of internet advertising identification. Most internet advertisements are spread on the internet in the form of advertorials. Based on the page classification method provided in this application embodiment, advertorials existing on the internet can be identified, thereby providing basic data for the internet advertising management system. The internet advertising management system can then conduct subsequent review and processing of the identified advertorials.

[0166] In one example, such as Figure 11 As shown, it exemplarily illustrates a schematic diagram of an advertising article recognition process implemented based on the page classification method provided in the embodiments of this application. Figure 11 The illustrated advertorial recognition process first involves accessing the webpage to be processed and acquiring images and text. For the acquired text, a text classification model is called to classify the text, resulting in the text type data. For the acquired images, an object detection model is called to perform object detection, such as calling a QR code detection model to detect QR codes, thus obtaining the QR code detection score. The results from both models are then fused to comprehensively determine whether the webpage is an advertorial.

[0167] The aforementioned text classification and object detection models require prior training. For training the text classification model, firstly, positive and negative samples of advertising copy are acquired, and stop word detection is performed on these samples to filter out stop words and reduce noise. After filtering out stop words, word embedding processing is performed, and the word vectors corresponding to the positive and negative samples are input into the text classification network to train the text classification model. The trained text classification model is then deployed to the advertising copy recognition service so that it can be called within the advertising copy recognition process.

[0168] For training the object detection model (using QR codes as an example), the background image and QR code image are first acquired, and then QR code overlay is performed to obtain a synthesized image. Next, the synthesized image undergoes mosaic enhancement and image size normalization to obtain the sample image and label information corresponding to the object detection model. By downsampling the sample image features, feature maps at multiple scales can be obtained. Multi-scale feature fusion processing is then performed on these feature maps to obtain a fused feature map. This fused feature map is then used to predict the target using a fixed window (i.e., the QR code detection box), thus training the object detection model. The trained object detection model is then deployed to an advertising copy recognition service so that it can be called within the advertising copy recognition process.

[0169] The aforementioned advertorial recognition process incorporates both object detection-based QR code detection and text classification-based text classification, enabling the identification of QR code advertisements with no or irrelevant text. Furthermore, since the format of QR codes is relatively fixed, self-made sample images and automatically generated sample labels can be used for model training, significantly reducing labor costs. Finally, the model fusion combining text classification and QR code detection achieves high accuracy and coverage, both exceeding 90%.

[0170] In one example, such as Figure 12 As shown, it exemplarily illustrates a schematic diagram of a page image to be processed. Figure 12 The image shows a webpage image 120 including a QR code 121. After the page to be processed, including the webpage image 120, is classified as a promotional webpage by the page classification method described above.

[0171] In summary, the technical solution provided in this application, by performing text classification processing and page jump information detection processing on the page to be processed, can obtain the text classification results corresponding to the text information in the page to be processed, as well as the detection results that can indicate whether the page to be processed includes page jump information. Then, the page type of the page to be processed can be determined based on the above two results, enriching the analysis dimensions of page classification. When the above text classification results indicate that the text information is the target text type and the detection results indicate that the page to be processed includes page jump information, the page to be processed is determined to be the target page type, effectively improving the accuracy of page classification.

[0172] The technical solution provided in this application embodiment will be further described below with reference to specific application scenarios. Advertising copy recognition in the field of internet advertising is a typical application scenario corresponding to the technical solution provided in this application embodiment. The technical solution provided in this application embodiment simultaneously introduces text classification processing and page jump information detection processing, and also sets a fusion algorithm for the two processing results. This effectively solves the problems of low accuracy and low coverage in advertising copy recognition. Practice has proven that this algorithm combined with text classification has excellent results, with accuracy and coverage exceeding 90%. Furthermore, the technical solution provided in this application embodiment also introduces strategies such as sample data augmentation and self-made training samples. A large amount of training data can be automatically generated through QR code mapping and image enhancement transformation, achieving a recognition accuracy of over 90% under low-cost conditions.

[0173] Based on the technical solution provided in this application, advertising advertorials can be automatically detected in the mobile internet advertising system every day, with an accuracy and recall rate of over 90%, which greatly saves labor costs.

[0174] The following are embodiments of the apparatus of this application, which can be used to execute embodiments of the method of this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method of this application.

[0175] Please refer to Figure 13 This diagram illustrates a block diagram of a page classification apparatus according to an embodiment of this application. The apparatus has the function of implementing the page classification method described above; this function can be implemented in hardware or by hardware executing corresponding software. The apparatus can be a computer device or can be installed within a computer device. The apparatus 1300 may include: a page acquisition module 1310, a text classification module 1320, a jump information detection module 1330, and a page type determination module 1340.

[0176] Page acquisition module 1310 is used to acquire a page to be processed, the page to be processed including text information;

[0177] The text classification module 1320 is used to perform text classification processing on the text information to obtain text classification results;

[0178] The jump information detection module 1330 is used to perform page jump information detection processing on the page to be processed and obtain the detection result;

[0179] The page type determination module 1340 is used to determine the page type corresponding to the page to be processed as the target page type when the text classification result indicates that the text information is the target text type and the detection result indicates that the page to be processed includes page jump information.

[0180] In an exemplary embodiment, the jump information detection module 1330 includes a page object detection submodule.

[0181] The page object detection submodule is used to perform page object detection processing on the page to be processed and obtain page object detection data.

[0182] The page object detection data represents the probability that a target page object exists in the page to be processed. The target page object refers to the page object associated with the page jump information. The detection result includes the page object detection data.

[0183] In an exemplary embodiment, the page to be processed includes an image, the target page object includes a graphic code, and the page object detection submodule includes: a graphic code detection unit.

[0184] A graphic code detection unit is used to perform graphic code detection processing on the image to obtain a graphic code detection score. The graphic code detection score represents the probability that the graphic code exists in the image. The page object detection data includes the graphic code detection score.

[0185] In an exemplary embodiment, the graphic code detection unit includes: a feature map extraction subunit, a detection box acquisition subunit, and a graphic code detection subunit.

[0186] The feature map extraction subunit is used to perform feature extraction processing on the image to obtain the feature map corresponding to the image.

[0187] The detection box acquisition subunit is used to acquire the graphic code detection box, which represents the graphic code detection area corresponding to the image.

[0188] The graphic code detection subunit is used to perform detection processing on the feature map based on the graphic code detection box to obtain the graphic code detection score.

[0189] In an exemplary embodiment, the graphic code detection unit is specifically used to input the image into the graphic code detection model and output the graphic code detection result;

[0190] The graphic code detection result includes the graphic code detection score. The graphic code detection model is a machine learning model trained based on sample images and label information. The sample images include synthetic images synthesized based on sample graphic codes and sample background images. The label information represents the actual location information of the sample graphic code in the sample image.

[0191] In an exemplary embodiment, the training process of the graphic code detection model includes:

[0192] Obtain the sample image and the label information;

[0193] Based on the label information, the sample image is clustered to obtain the graphic code detection box corresponding to the sample image;

[0194] The sample image, the label information, and the graphic code detection box are input into the graphic code detection model to be trained, and the sample graphic code detection result corresponding to the sample image is output.

[0195] Based on the label information and the sample graphic code detection results, the parameters of the graphic code detection model to be trained are adjusted to obtain the graphic code detection model.

[0196] In an exemplary embodiment, the target page object includes a page navigation control, and the page object detection submodule includes a control detection unit.

[0197] The control detection unit is used to perform page jump control detection processing on the page to be processed and obtain a control detection score. The control detection score represents the probability that the page jump control exists on the page to be processed. The page object detection data includes the control detection score.

[0198] In an exemplary embodiment, the text classification result includes text type data corresponding to the target text type, the text type data representing the probability that the text information is the target text type, and the detection result includes detection data corresponding to the page jump information, the detection data representing the probability that the page to be processed includes the page jump information;

[0199] The page type determination module 1340 is specifically used to determine the page type as the target page type when the text type data is greater than a first threshold and the detection data is greater than a second threshold.

[0200] In an exemplary embodiment, the text classification module 1320 is specifically used to input the text information into a text classification model and output the text classification result;

[0201] The text classification model is a machine learning model trained on sample text and label information. The label information represents the text type to which the sample text belongs. The label information is determined based on the correspondence between the sample text and a keyword library. The keyword library stores at least one keyword related to the text type.

[0202] In summary, the technical solution provided in this application, by performing text classification processing and page jump information detection processing on the page to be processed, can obtain the text classification results corresponding to the text information in the page to be processed, as well as the detection results that can indicate whether the page to be processed includes page jump information. Then, the page type of the page to be processed can be determined based on the above two results, enriching the analysis dimensions of page classification. When the above text classification results indicate that the text information is the target text type and the detection results indicate that the page to be processed includes page jump information, the page to be processed is determined to be the target page type, effectively improving the accuracy of page classification.

[0203] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when implementing its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0204] Please refer to Figure 14 This diagram illustrates a structural block diagram of a computer device according to an embodiment of this application. The computer device may be a server for executing the page classification method described above. Specifically:

[0205] Computer device 1400 includes a central processing unit (CPU) 1401, a system memory 1404 including random access memory (RAM) 1402 and read-only memory (ROM) 1403, and a system bus 1405 connecting the system memory 1404 and the CPU 1401. Computer device 1400 also includes a basic input / output system (I / O system) 1406 that facilitates information transfer between various devices within the computer, and a mass storage device 1407 for storing the operating system 1413, application programs 1414, and other program modules 1415.

[0206] The basic input / output system 1406 includes a display 1408 for displaying information and an input device 1409 for user input, such as a mouse or keyboard. Both the display 1408 and the input device 1409 are connected to the central processing unit 1401 via an input / output controller 1410 connected to the system bus 1405. The basic input / output system 1406 may also include the input / output controller 1410 for receiving and processing input from multiple other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 1410 also provides output to a display screen, printer, or other types of output devices.

[0207] Mass storage device 1407 is connected to central processing unit 1401 via a mass storage controller (not shown) connected to system bus 1405. Mass storage device 1407 and its associated computer-readable media provide non-volatile storage for computer device 1400. That is, mass storage device 1407 may include computer-readable media (not shown) such as hard disk or CD-ROM (Compact Disc Read-Only Memory) drive.

[0208] Without loss of generality, computer-readable media can include computer storage media and communication media. Computer storage media includes volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes RAM, ROM, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), flash memory or other solid-state storage technologies, CD-ROM, DVD (Digital Video Disc) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that computer storage media are not limited to the above-mentioned types. The system memory 1404 and mass storage device 1407 described above can be collectively referred to as memory.

[0209] According to various embodiments of this application, the computer device 1400 can also be connected to a remote computer on a network, such as the Internet. That is, the computer device 1400 can be connected to the network 1412 via the network interface unit 1411 connected to the system bus 1405, or the network interface unit 1411 can be used to connect to other types of networks or remote computer systems (not shown).

[0210] The memory also includes a computer program stored in the memory and configured to be executed by one or more processors to implement the page classification method described above.

[0211] In an exemplary embodiment, a computer-readable storage medium is also provided, the storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set, when executed by a processor, implements the page classification method described above.

[0212] Optionally, the computer-readable storage medium may include: ROM (Read Only Memory), RAM (Random Access Memory), SSD (Solid State Drives), or optical disc, etc. The random access memory may include ReRAM (Resistance Random Access Memory) and DRAM (Dynamic Random Access Memory).

[0213] In an exemplary embodiment, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the page classification method described above.

[0214] It should be understood that "multiple" as used herein refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. Furthermore, the step numbers described herein are merely illustrative of one possible execution order. In some other embodiments, the steps may not be executed in numerical order, such as two steps with different numbers being executed simultaneously, or two steps with different numbers being executed in the reverse order of the illustration. This application does not limit this.

[0215] In addition, in the specific embodiments of this application, data such as user information are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0216] The above description is merely an exemplary embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A page classification method, characterized in that, The method includes: Obtain the page to be processed, which includes text information; The text information is subjected to text classification processing to obtain the text classification result; The page to be processed is subjected to page redirection information detection processing to obtain the detection result; The step of performing page jump information detection processing on the page to be processed to obtain detection results includes: performing page object detection processing on the page to be processed to obtain page object detection data; wherein, the page object detection data represents the probability that a target page object exists in the page to be processed, the target page object refers to the page object associated with the page jump information, and the detection result includes the page object detection data; the page to be processed includes an image, and the target page object includes a graphic code; The step of performing page object detection processing on the page to be processed to obtain page object detection data includes: performing graphic code detection processing on the image to obtain a graphic code detection score, wherein the graphic code detection score represents the probability that the graphic code exists in the image, and the page object detection data includes the graphic code detection score; If the text classification result indicates that the text information is the target text type, and the detection result indicates that the page to be processed includes page jump information, then the page type corresponding to the page to be processed is determined to be the target page type.

2. The method according to claim 1, characterized in that, The step of performing graphic code detection processing on the image to obtain a graphic code detection score includes: The image is subjected to feature extraction processing to obtain the feature map corresponding to the image; Obtain a graphic code detection box, wherein the graphic code detection box represents the graphic code detection area corresponding to the image; The feature map is processed based on the graphic code detection box to obtain the graphic code detection score.

3. The method according to claim 1, characterized in that, The step of performing graphic code detection processing on the image to obtain a graphic code detection score includes: The image is input into the graphic code detection model, and the graphic code detection result is output. The graphic code detection result includes the graphic code detection score. The graphic code detection model is a machine learning model trained based on sample images and label information. The sample images include synthetic images synthesized based on sample graphic codes and sample background images. The label information represents the actual location information of the sample graphic code in the sample image.

4. The method according to claim 3, characterized in that, The training process of the graphic code detection model includes: Obtain the sample image and the label information; Based on the label information, the sample image is clustered to obtain the graphic code detection box corresponding to the sample image; The sample image, the label information, and the graphic code detection box are input into the graphic code detection model to be trained, and the sample graphic code detection result corresponding to the sample image is output. Based on the label information and the sample graphic code detection results, the parameters of the graphic code detection model to be trained are adjusted to obtain the graphic code detection model.

5. The method according to claim 1, characterized in that, The target page object includes a page navigation control. The step of performing page object detection processing on the page to be processed to obtain page object detection data includes: The page to be processed is subjected to page jump control detection processing to obtain a control detection score. The control detection score represents the probability that the page jump control exists in the page to be processed. The page object detection data includes the control detection score.

6. The method according to claim 1, characterized in that, The text classification result includes text type data corresponding to the target text type, and the text type data represents the probability that the text information is the target text type. The detection result includes detection data corresponding to the page jump information, and the detection data represents the probability that the page to be processed includes the page jump information. The step of determining the page type corresponding to the page to be processed as the target page type when the text classification result indicates that the text information is the target text type and the detection result indicates that the page to be processed includes page jump information includes: If the text type data is greater than a first threshold and the detection data is greater than a second threshold, the page type is determined to be the target page type.

7. The method according to claim 1, characterized in that, The text classification process for the text information, to obtain the text classification result, includes: The text information is input into the text classification model, and the text classification result is output. The text classification model is a machine learning model trained on sample text and label information. The label information represents the text type to which the sample text belongs. The label information is determined based on the correspondence between the sample text and a keyword library. The keyword library stores at least one keyword related to the text type.

8. A page sorting device, characterized in that, The device includes: The page acquisition module is used to acquire the page to be processed, which includes text information; The text classification module is used to perform text classification processing on the text information to obtain the text classification result; The jump information detection module is used to detect and process page jump information on the page to be processed, and obtain the detection result; The page type determination module is used to determine the page type corresponding to the page to be processed as the target page type when the text classification result indicates that the text information is the target text type and the detection result indicates that the page to be processed includes page jump information. The redirection information detection module includes: a page object detection submodule; the page object detection submodule is used to perform page object detection processing on the page to be processed to obtain page object detection data; wherein, the page object detection data represents the probability that a target page object exists in the page to be processed, the target page object refers to the page object associated with the page redirection information, and the detection result includes the page object detection data; the page to be processed includes an image, and the target page object includes a graphic code; The page object detection submodule includes: a graphic code detection unit; the graphic code detection unit is used to perform graphic code detection processing on the image to obtain a graphic code detection score, the graphic code detection score representing the probability that the graphic code exists in the image, and the page object detection data includes the graphic code detection score.

9. The apparatus according to claim 8, characterized in that, The graphic code detection unit includes: a feature map extraction subunit, a detection box acquisition subunit, and a graphic code detection subunit; The feature map extraction subunit is used to perform feature extraction processing on the image to obtain the feature map corresponding to the image; The detection box acquisition subunit is used to acquire the graphic code detection box, and the graphic code detection box represents the graphic code detection area corresponding to the image. The graphic code detection subunit is used to perform detection processing on the feature map based on the graphic code detection box to obtain the graphic code detection score.

10. The apparatus according to claim 8, characterized in that, The graphic code detection unit is used to input the image into the graphic code detection model and output the graphic code detection result. The graphic code detection result includes the graphic code detection score. The graphic code detection model is a machine learning model trained based on sample images and label information. The sample images include synthetic images synthesized based on sample graphic codes and sample background images. The label information represents the actual location information of the sample graphic code in the sample image.

11. The apparatus according to claim 10, characterized in that, The training process of the graphic code detection model includes: Obtain the sample image and the label information; Based on the label information, the sample image is clustered to obtain the graphic code detection box corresponding to the sample image; The sample image, the label information, and the graphic code detection box are input into the graphic code detection model to be trained, and the sample graphic code detection result corresponding to the sample image is output. Based on the label information and the sample graphic code detection results, the parameters of the graphic code detection model to be trained are adjusted to obtain the graphic code detection model.

12. The apparatus according to claim 8, characterized in that, The target page object includes a page navigation control, and the page object detection submodule includes: a control detection unit; The control detection unit is used to perform page jump control detection processing on the page to be processed and obtain a control detection score. The control detection score represents the probability that the page jump control exists on the page to be processed. The page object detection data includes the control detection score.

13. The apparatus according to claim 8, characterized in that, The text classification result includes text type data corresponding to the target text type, and the text type data represents the probability that the text information is the target text type. The detection result includes detection data corresponding to the page jump information, and the detection data represents the probability that the page to be processed includes the page jump information. The page type determination module is used to determine the page type as the target page type when the text type data is greater than a first threshold and the detection data is greater than a second threshold.

14. The apparatus according to claim 8, characterized in that, The text classification module is used to input the text information into the text classification model and output the text classification result; The text classification model is a machine learning model trained on sample text and label information. The label information represents the text type to which the sample text belongs. The label information is determined based on the correspondence between the sample text and a keyword library. The keyword library stores at least one keyword related to the text type.

15. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one instruction, at least one program, a code set, or an instruction set, the at least one instruction, the at least one program, the code set, or the instruction set being loaded and executed by the processor to implement the page classification method as described in any one of claims 1 to 7.

16. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or instruction set is loaded and executed by a processor to implement the page classification method as described in any one of claims 1 to 7.

17. A computer program product, characterized in that, The computer program product includes computer instructions stored in a computer-readable storage medium, a processor of a computer device reading the computer instructions from the computer-readable storage medium, and the processor executing the computer instructions to cause the computer device to perform the page classification method as described in any one of claims 1 to 7.

Images