Station group management method and system based on low code

Abstract

The invention relates to the technical field of station group management methods, in particular to a station group management method based on low codes. According to the method, the scrolling behavior of the user is dynamically simulated through the headless browser, the problem that a traditional static crawler cannot capture JavaScript rendering content and hidden texts is solved in combination with the random speed and a waiting mechanism, and the scrolling speed is dynamically adjusted, for example, scrolling parameters are calculated based on DOM node density and AJAX request frequency, the randomness of human browsing is simulated, so that the user experience is improved. The method comprises the steps of effectively avoiding the recognition of an anti-crawling mechanism on a fixed rolling mode, ensuring that page contents are completely rendered through a display waiting mechanism, capturing a complete DOM structure and hidden texts including lazy loading resources and sensitive elements generated by an AJAX dynamic request, and avoiding the omission of key risk information; the technical problem of relatively low identification efficiency in the process of identifying the station group risk filliform in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of site group management methods, and in particular to a low-code-based site group management method and system. Background Art

[0002] Current low-code processing technologies for identifying risky web pages in site clusters mainly rely on static HTML source code parsing and keyword matching. For example, they use rule engines or traditional machine learning models to perform sensitive word frequency statistics or URL feature matching on web page text content. Some methods combine simple crawler tools to obtain basic web page information. However, existing technologies have significant limitations: First, by relying on JavaScript to load content or asynchronously requesting AJAX to dynamically render web pages, key risk information cannot be fully obtained due to the inability to fully obtain the rendered DOM structure and hidden text. Second, existing methods are mostly based on single-modal data, such as plain text or static screenshots, and lack deep integration of visual features such as page layout and image content with structural features. As a result, the model is unable to identify disguised risk content, such as images embedded in illegal information. In addition, traditional crawlers use fixed scrolling speeds and intervals to simulate browsing behavior, which can easily be identified as robot operations by anti-crawling mechanisms and restricted access. In addition, the simple block processing of static screenshots makes it difficult to focus on key areas and introduces a large amount of redundant information. Summary of the Invention

[0003] In response to the shortcomings of the existing technology, the present invention provides a low-code based site group management method and system, which solves the technical problem of relatively low identification efficiency in the process of identifying site group risk factors in the existing technology.

[0004] To solve the above technical problems, the present invention provides the following technical solution: a low-code-based site group management method, which specifically includes the following steps: S1. Obtain the webpage URL and webpage attribute characteristics of each site in the site group; S2, simulating user scrolling through a headless browser to obtain a rendered complete web page corresponding to the web page URL, and then extracting the complete HTML source code and hidden text data of the web page; S3, obtaining a full-page screenshot of the rendered complete web page and segmenting it, and inputting the segmented image into the first convolutional neural network model to obtain the image with the highest attention score, and marking it as a preprocessed image; S4, converting the HTML source code, hidden text data and pre-processed image into source code vector, text vector and image vector respectively; S5. Concatenate the source code vector, text vector, image vector, and web page vector into a fusion vector, and input it into the second convolutional neural network model to obtain the risk probability.

[0005] Preferably, in step S2, the following steps are specifically included: S21. Calling a headless browser driver to establish a connection with the browser; S22, input the site webpage URL, and drive the browser to execute the JavaScript script embedded in the webpage corresponding to the webpage URL; S23, controlling the browser through a JavaScript script to simulate a user's manual scrolling operation within the webpage, and gradually obtaining the rendered webpage through a display waiting mechanism; S24. Obtain the complete HTML source code and DOM elements of the rendered complete web page; S25. Parse the body tag text in the DOM tree through DOM, and parse the hidden text data from the body tag text.

[0006] Preferably, in step S23, the following steps are specifically included: S231, controlling the browser through a JavaScript script to simulate a user's manual scrolling operation within the web page; S232. Obtain webpage data of the rendered webpage through DOM parsing, where the webpage data includes headless browser window data and a DOM structure and AJAX request data of an area where the headless browser window is located; S233. Calculate the DOM node density based on the web page data. The calculation formula for the DOM node density is: In the above formula, Indicates the density of DOM nodes, Indicates the total number of DOM nodes in the area where the headless browser window is located. is the area of ​​the headless browser window, which is equal to the product of the length and width of the headless browser window; S234. Calculate the AJAX request frequency based on the web page data. The calculation formula for the AJAX request frequency is: In the above formula, Indicates the AJAX request frequency, Indicates the total number of AJAX requests triggered during the rendering process of the area where the headless browser window is located. Indicates the initial loading time of the area where the headless browser window is located during the rendering process; S235, calculating the scrolling time of each scroll according to the DOM node density and the AJAX request frequency; S236: Preset the maximum scrolling speed and calculate the scrolling speed of each scroll according to the DOM node density; the calculation formula of the scrolling speed is: In the above formula, Indicates the scroll speed, Indicates the maximum scroll speed, is a random number that changes randomly over time; S237, after each scrolling is completed, wait for the corresponding web page area to be completely rendered through the display waiting mechanism; S238. Calculate the product of the scrolling time and the scrolling speed of each scroll to obtain the scrolling distance of each scroll; When the scroll distance reaches 0, the scrolling ends and the rendered complete web page is obtained.

[0007] Preferably, in step S3, the following steps are specifically included: S31, obtaining several full-page screenshots of the rendered complete web page corresponding to the web page URL in a preset image acquisition cycle; S32. Obtain the lengths of the long side and the wide side of the full-page screenshot, mark the longer side as the sliding side, mark the sliding direction on the sliding side from left to right and from top to bottom, and mark the length of the shorter side as the sliding distance; S33, along the sliding direction, select the first boundary line perpendicular to the sliding direction on the full-page screenshot and mark it as the initial segmentation edge, and mark another boundary line perpendicular to the sliding direction as the final segmentation edge; S34, setting a distance threshold according to the sliding distance, and sequentially setting a plurality of segmentation edges within the distance threshold between the initial segmentation edge and the final segmentation edge of the full-page screenshot along the sliding direction, until the full-page screenshot is segmented to obtain a plurality of sub-screenshots; S35. Compress the aspect ratio of the sub-screenshot to 1:1 and input it into a neural network including a transposed convolution layer and a self-attention module to obtain several sub-screenshots with the highest attention scores and mark them as preprocessed images.

[0008] Preferably, in step S34, the following steps are specifically included: S341, setting a distance threshold according to the sliding distance, and marking the segmentation edge farthest from the initial segmentation edge as the current segmentation edge; S342: Setting a pixel value fusion interval, and establishing a nine-square grid with each pixel point on the full-page screenshot on one side of the current segmentation edge along the sliding direction as the center; S343, sequentially determining whether there is a pixel point in each nine-square grid whose pixel value difference with the center pixel point of the nine-square grid is within the pixel value fusion interval; If they exist, the two are marked as the same connecting line to obtain several connecting lines; If it does not exist, no connection will be made; S344, calculating the distance between the final segmentation edge and the current segmentation edge, and determining whether to extract the sub-screenshot based on whether the distance is less than the maximum value of the distance threshold; If yes, then the image areas between adjacent segmentation edges are output to obtain a number of sub-snipple images, and the process goes to step S348; If not, proceed to the next step; S345: setting a plurality of edges to be segmented that are parallel to the current segmentation edge on the full-page screenshot along the sliding direction, wherein the distances between the edges to be segmented and the current segmentation edge are within a distance threshold; S346, extracting the skeleton line of each connecting line, and sequentially counting the number of intersections between each edge to be segmented and all connecting lines; S347, marking the edge to be split that has the smallest number of intersections and is closest to the sliding distance from the current split edge as a split edge, and then returning to step S341; S348: Calculate the area of ​​each sub-screenshot and determine whether the area of ​​the sub-screenshot with the smallest area is less than the area threshold; If yes, the segmentation is terminated and the process goes to step S35; If not, the sub-screenshot with the smallest area is marked as a full-page screenshot, and then the process returns to step S32.

[0009] Preferably, in step S4, the following steps are specifically included: S41, inputting the hidden text data into the BERT model to obtain a first text vector; S42. Match the sensitive word library according to the Trie tree and count the frequency of sensitive words in the hidden text data to obtain a second text vector: S43, delete the third-party library tags in the HTML source code, then parse it into a DOM tree structure, and generate a node sequence vector through a pre-order traversal algorithm; S44, input the node sequence vector into the FastText model and output the source code vector; S45, input each preprocessed image into the EfficientNet model and output the image vector; S46. Obtain the record information, access location, and URL length of the web page corresponding to the web page URL and perform One-Hot encoding to generate a web page vector.

[0010] Preferably, in step S5, the following steps are specifically included: S51. Concatenate the first text vector and the second text vector to obtain a text vector; the expression of the text vector is: In the above formula, represents the fused text vector, and represent the first text vector and the second text vector respectively; S52, calculating the value matrix, key matrix, and value matrix of the attention mechanism according to the text vector and the image vector, and performing semantic alignment and fusion through the attention mechanism to generate a text-image fusion vector; S53, converting the HTML source code and the DOM tree into a DOM graph, wherein each vertex in the vertex set of the DOM graph corresponds to each node in the DOM tree, and establishing edges at positions in the DOM graph corresponding to positions between parent nodes and child nodes in the DOM tree to obtain an adjacency matrix; S54, connecting the text image fusion vector with each node in the DOM graph to obtain updated node features; the expression of the updated node features is: In the above formula, represents the updated i-th node feature, represents the i-th node feature before updating, Represents the text image fusion vector; S55, input the updated node features into several layers of graph convolutional networks (GCN) to obtain aggregated features; S56. Perform average pooling based on the aggregated features to obtain a second fusion feature. The calculation formula for the second fusion feature is: In the above formula, represents the second fusion feature, N represents the total number of nodes in the DOM graph, Represents the aggregated features obtained after the L-layer graph convolutional network GCN; S57. Map the second fusion feature and the web page vector to a unified dimension through a fully connected layer and then concatenate them to obtain a fusion vector. Input the fusion vector into the DNN model and output the risk probability.

[0011] Preferably, in step S235, the calculation formula of the scrolling time is: In the above formula, Indicates the scroll time, 、 and are the first, second, and third regression coefficients obtained through historical data training.

[0012] Preferably, in step S52, the expression of the text image fusion vector is: In the above formula, represents the text-image fusion vector, represents the attention mechanism, Q, K, and V represent the query matrix, key matrix, and value matrix respectively. 、 and represent the query matrix parameter matrix, key matrix parameter matrix, and value matrix parameter matrix respectively.

[0013] The present invention also provides a low-code-based site cluster management system, including a processor and a memory, wherein the memory is used to store a computer program, and the computer program implements the site cluster management method when executed by the processor.

[0014] Through the above technical solution, the present invention provides a low-code-based site group management method and system, which has at least the following beneficial effects: 1. The present invention dynamically simulates user scrolling behavior through a headless browser, and combines random speed and waiting mechanisms to solve the problem that traditional static crawlers cannot capture JavaScript rendered content and hidden text, significantly improving the integrity and accuracy of dynamic web page risk identification. By dynamically adjusting the scrolling speed, such as calculating scrolling parameters based on DOM node density and AJAX request frequency, the randomness of human browsing is simulated, and the anti-crawling mechanism's recognition of fixed scrolling patterns is effectively avoided. At the same time, the display waiting mechanism ensures that the page content is fully rendered, and captures the complete DOM structure and hidden text including lazy-loaded resources and sensitive elements generated by AJAX dynamic requests, avoiding the omission of key risk information, and reducing the missed detection rate by more than 60% compared with traditional methods.

[0015] 2. This invention introduces a multimodal data fusion framework that combines text semantics, visual features, source code structure, and web page attributes to significantly improve the ability to identify disguised risky content. The BERT model is used to extract deep semantic features of hidden text, and the Trie tree is used to count the frequency of sensitive words to supplement explicit rules. FastText encodes the hierarchical relationship of the HTML source code, EfficientNet extracts high-order visual features of images, and One-Hot encodes the structural attributes of web pages to form complementary features. Furthermore, the attention mechanism is used to achieve semantic alignment of text and images. The graph convolutional network (GCN) is used to aggregate DOM graph node features and capture code topological associations. Finally, the DNN model is used to fuse multimodal features to output risk probabilities, improving the recognition accuracy of complex attack patterns such as embedded illegal information in images and hidden sensitive words in CSS by 45%.

[0016] 3. The present invention adopts adaptive image segmentation technology, reduces redundant information interference through nine-square pixel fusion and dynamic boundary adjustment, focuses on high-risk visual areas, dynamically segments full-page screenshots based on sliding direction and distance threshold, uses the nine-square pixel fusion algorithm to identify content areas, avoids the breakage of text areas or interference in advertising areas caused by traditional uniform blocking, combines the transposed convolution layer to restore image details, and the self-attention module selects high-attention sub-graphs, thereby improving image preprocessing efficiency by 30% and reducing the waste of subsequent neural network computing resources.

[0017] 4. This invention enhances the modeling capability of DOM structured features through the deep integration of DOM graph construction and graph convolutional networks (GCN), effectively identifying distributed dark links and dynamic content injection attacks: the DOM tree is converted into a graph structure, and neighborhood node features are aggregated through the GCN layer to capture code hierarchical relationships and event propagation paths; at the same time, the text image fusion vector is spliced ​​with DOM node features and the node representation is updated, enabling the model to identify dynamically bound malicious scripts and cross-node attack patterns, reducing the false alarm rate by 25%.

[0018] 5. The present invention meets the real-time analysis needs of massive station cluster data through efficient computing resource allocation and parallel processing. It adopts a phased pipeline design, such as parallel execution of scrolling rendering, image segmentation, vectorization, and fusion computing, combined with lightweight models (such as EfficientNet-B4) and feature compression technology. While ensuring accuracy, the single-page processing time is controlled within 2 seconds. At the same time, it supports distributed deployment, such as DNN model sharding computing, which can be horizontally expanded to synchronize monitoring of thousands of sites, reducing resource utilization by 50%, and is suitable for high-frequency risk monitoring scenarios of large station clusters. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 This is a flow chart of the low-code-based site group management method of the present invention; Figure 2 Schematic diagram of the screenshot of the present invention. DETAILED DESCRIPTION

[0020] To make the above-mentioned objectives, features, and advantages of the present invention more clearly understood, the present invention is further described below in detail with reference to the accompanying drawings and specific embodiments. This will enable a full understanding of how this application uses technical means to solve technical problems and achieve technical effects, and to implement the invention accordingly.

[0021] In order to solve the technical problem of relatively low recognition efficiency in the process of identifying risky website groups in the existing technology, the present invention provides a low-code-based website group management method, further improves the method of obtaining dynamically rendered web pages, so as to extract the complete rendered DOM structure and hidden text, and combines multimodal data such as plain text, static screenshots and website attributes to improve the recognition accuracy of risky web pages, such as Figure 1 As shown, the specific steps include: S1. Obtain the web page URL and web page attribute characteristics of each site in the station group. The web page attribute characteristics include the registration information, access location and URL length of the web page. Export URLs in batches through the station group management background or log system, and obtain real-time attribute characteristics in combination with a web crawler or API interface.

[0022] S2. Use a headless browser to simulate user scrolling to obtain the rendered complete webpage corresponding to the webpage URL. Then extract the complete HTML source code and hidden text data of the webpage. Modern webpages rely on JavaScript to dynamically load content. Traditional crawlers cannot capture the rendered DOM structure. By simulating user scrolling behavior, triggering AJAX requests and lazy loading, we ensure that the page content is fully rendered. The specific steps include the following: S21. Call the headless browser driver to establish a connection with the browser. Use tools such as Selenium or Puppeteer to start the browser process. Configure the proxy and user agent to simulate a real device. Establish a communication channel with the headless browser by initializing a browser instance (such as Chrome Headless).

[0023] S22. Enter the site web page URL and drive the browser to execute the JavaScript script embedded in the web page corresponding to the web page URL. After the browser navigates to the URL, it automatically executes the page script and waits for the initial DOM loading to be completed. By loading the target page and executing the JavaScript code in the page, dynamic content such as AJAX requests are ensured to be triggered.

[0024] S23. Control the browser through JavaScript scripts to simulate the user's manual scrolling operation within the web page, and gradually obtain the rendered web page through the display waiting mechanism. The regularity of the fixed scrolling operation, such as a fixed scrolling operation time of 500ms, is easily identified as robot behavior. In order to reduce the risk of anti-crawling, the scrolling speed and interval time are randomly changed through a dynamic strategy. That is, by calculating the scrolling speed, time and distance, the scrolling parameters are dynamically adjusted in combination with the DOM node density and AJAX request frequency to simulate the randomness of human browsing and simulate human browsing behavior. Specifically, the steps include the following: S231. Control the browser through JavaScript script to simulate the user's manual scrolling operation within the web page.

[0025] S232. Obtain webpage data of the rendered webpage through DOM parsing, where the webpage data includes headless browser window data and a DOM structure and AJAX request data of an area where the headless browser window is located; S233. Calculate the DOM node density based on the web page data. A high density indicates complex content and requires a longer scrolling wait time. The calculation formula for DOM node density is: In the above formula, Indicates the density of DOM nodes, Indicates the total number of DOM nodes in the area where the headless browser window is located. is the area of ​​the headless browser window, which is equal to the product of the length and width of the headless browser window; S234. Calculate the AJAX request frequency based on the web page data. A high frequency indicates that the page is highly dynamic and the scrolling interval needs to be shortened. The calculation formula for the AJAX request frequency is: In the above formula, Indicates the AJAX request frequency, Indicates the total number of AJAX requests triggered during the rendering process in the area where the headless browser window is located. It can be captured through the Network panel of the browser developer tools. Indicates the initial loading time of the area where the headless browser window is located during the rendering process; S235. Calculate the scrolling time for each scroll according to the DOM node density and the AJAX request frequency. The calculation formula for the scrolling time is: In the above formula, Indicates the scroll time, 、 and are the first, second, and third regression coefficients obtained through historical data training; S236. Preset the maximum scrolling speed and calculate the scrolling speed for each scroll according to the DOM node density. When the node density is high, reduce the speed to ensure that the content is fully rendered. The calculation formula for the scrolling speed is: In the above formula, Indicates the scroll speed, Indicates the maximum scroll speed, It is a random number that changes randomly over time, simulating the speed instability of a person rolling. The random number is randomly generated periodically through a random number function.

[0026] S237, after each scrolling is completed, wait for the corresponding web page area to be completely rendered through the display waiting mechanism; S238. Calculate the product of the scrolling time and the scrolling speed of each scroll to obtain the scrolling distance of each scroll, determine the pixel displacement of a single scroll, and avoid content skipping caused by excessive scrolling. When the scrolling distance is 0, the scrolling ends, and the rendered complete web page is obtained.

[0027] S24. Obtain the complete HTML source code and DOM elements of the rendered complete web page, export the complete HTML source code of the rendered complete web page through a browser developer tool API (such as Chrome DevTools Protocol), and obtain the final DOM structure after rendering, where the DOM tree structure includes dynamically generated HTML elements.

[0028] S25, parse the body tag text in the DOM element through DOM, and parse the hidden text data from the body tag text, extract all text nodes by traversing the tags of the DOM tree, and filter invalid tags, such as <script>，以提取通过CSS隐藏（如display: none）或JavaScript动态插入的文本内容，从而获得隐藏文本数据。

[0029] S3、获取渲染后的完整网页的全页面截图并分割，并将分割后的图像输入到第一卷积神经网络模型以获得注意力分数最高的图像，并标记为预处理图像，全页面截图包含大量无关区域，如页眉和广告等，需聚焦高风险内容区域，如正文和评论区等，因此需要将截取的完整页面图像分割为关键区域并筛选高注意力的子截图，减少冗余信息干扰，如图2所示，具体包括如下步骤：S31、以预设的图像采集周期，获取与网页URL相对应的渲染后的完整网页的若干个全页面截图；S32、获取全页面截图的长边和宽边的长度，将长度较大的边标记为滑动边，按照从左往右和从上往下的方向在滑动边上标记滑动方向，将长度较小的边的长度标记为滑动距离，比如对于一般网页，会向下滚动以逐步获取网页内容，因此将从上往下的方向标记为滑动方向，全页面截图最顶部的边的长度标记为滑动距离。

[0030] S33、沿着滑动方向，选取全页面截图上与滑动方向垂直的第一条边界线标记为初始的分割边，将与滑动方向垂直的另一条边界线标记为最终的分割边，即对于全页面截图，若宽度较小，则其最上方的长边为与滑动方向垂直的第一条边界线，即初始的分割边，而最下方的长边为与滑动方向垂直的另一条边界线，即最终的分割边。

[0031] S34、根据滑动距离设置距离阈值，并沿着滑动方向在全页面截图初始的分割边和最终的分割边之间依次设置若干条处于距离阈值内的分割边，直至分割完全页面截图以获得若干个子截图，获取的子截图需要再后续压缩为1:1的比例，为了避免压缩后的图像中的关键信息在后续识别时被遗漏或者丢失，需要保证子截图的长宽比与1:1接近，通过一种利用九宫格像素融合的算法识别相邻区域的连续性，动态调整分割边界，具体包括如下步骤：S341、根据滑动距离设置距离阈值，距离阈值通常设置为滑动距离的80%-120%，避免分割过密或过疏，并将与初始的分割边距离最远的分割边标记为当前分割边；S342、设置像素值融合区间，分别以当前分割边沿着滑动方向一侧的全页面截图上的各像素点为中心建立九宫格；S343、依次判断各九宫格内是否存在与九宫格中心像素点的像素值之差处于像素值融合区间内的像素点，比如设置融合区间为RGB差值<20，若九宫格内相邻像素值差在区间内，则标记为同一连接线；若存在，则将二者标记为同一条连接线，以获得若干条连接线，识别图像中连续区域；若不存在，则不连接；S344、计算最终的分割边与当前分割边的距离，并根据该距离是否小于距离阈值的最大值判定是否提取子截图；若是，则输出各相邻的分割边之间的图像区域以获得若干个子截图，并进入步骤S348；若否，则进入下一步；S345、沿着滑动方向在全页面截图上设置与当前分割边平行的若干条待分割边，所述待分割边与当前分割边的距离处于距离阈值内；S346、提取各连接线的骨架线，并依次统计各待分割边与所有连接线的交点的数量；S347、将交点的数量最小且与当前的分割边的距离最接近滑动距离的待分割边标记为分割边，量化分割边与内容区域的交叠程度，选择干扰最小的分割线，之后返回步骤S341，进行下一轮迭代，直至满足终止条件。

[0032] S348、计算各子截图的面积，并判断面积最小的子截图的面积是否小于面积阈值，如50×50像素，以避免无限迭代，同时通过设置较小的面积阈值，避免最终终止时面积最小的子截图内含有重要信息；若是，则结束分割，并进入步骤S35；若否，则将面积最小的子截图标记为全页面截图，之后返回步骤S32。

[0033] S35、将子截图的长宽比压缩为1：1后输入到包含转置卷积层和自注意力模块的神经网络，以获得注意力分数最高的若干个子截图，并标记为预处理图像，使用转置卷积层可以恢复图像细节，自注意力模块可以聚焦关键区域，并获得子截图的注意力分数。

[0034] S4、将HTML源码和隐藏文本数据和预处理图像分别转化为源码向量、文本向量和图像向量，在步骤S4中，具体包括如下步骤：S41、将隐藏文本数据输入到BERT模型以获得第一文本向量，使用预训练的BERT-base模型，对文本进行Tokenization和Embedding，以捕捉文本深层语义（如情感倾向、敏感话题），解决传统关键词匹配的局限性。

[0035] S42、根据Trie树匹配敏感词库，统计隐藏文本数据中的敏感词频以获得第二文本向量，通过构建敏感词Trie树，统计匹配频次并生成One-Hot向量，以快速检测显式敏感词，如政治违禁词，补充BERT的语义分析。

[0036] S43、删除HTML源码中的第三方库标签，之后解析为DOM树结构，并通过前序遍历算法生成节点序列向量，通过删除第三方库标签，如<script>，并通过前序遍历生成节点序列，保留父子节点关系，以将HTML源码转化为反映DOM层级结构的节点序列向量。

[0037] S44、将节点序列向量输入到FastText模型，并输出源码向量，通过将节点序列输入FastText模型，输出基于N-gram的源码向量，以捕捉HTML标签的上下文语义，如与的嵌套关系。

[0038] S45、将各个预处理图像输入到EfficientNet模型，并输出图像向量，使用EfficientNet-B4预训练模型，输出ImageNet分类层前的特征向量，以从子截图中提取高阶视觉特征，如颜色分布和纹理模式等。

[0039] S46、获取网页URL所对应网页的备案信息、接入地和URL长度进行One-Hot编码，以生成网页向量，对于接入地，境内 / 境外编码为[0,1]或[1,0]，从而将URL长度、备案信息等结构化特征转化为数值向量。

[0040] S5、将源码向量、文本向量、图像向量和网页向量拼接为融合向量，并输入第二卷积神经网络模型以获得风险概率，但是对于不同的向量，若直接进行拼接融合，可能会导致特征空间混乱，模型难以捕捉跨模态的复杂关联，比如高维向量（如图像特征）可能主导融合结果，掩盖低维但关键的信号，因此提供一种分步骤拼接融合策略，从低层的特征到中层的DOM结构特征，再到高层的风险概率，逐层提炼信息，避免一次性融合的维度爆炸，避免某些模态因维度差异被边缘化，具体包括如下步骤：S51、将第一文本向量和第二文本向量进行拼接以获得文本向量，融合语义向量与敏感词统计向量，兼顾隐式与显式风险特征；文本向量的表达式为：上式中，表示融合后的文本向量，和分别表示第一文本向量和第二文本向量；S52、根据文本向量和图像向量计算注意力机制的值矩阵、键矩阵、值矩阵，并通过注意力机制进行语义对齐和融合，生成文本图像融合向量，通过多头注意力计算文本与图像的关联权重，实现跨模态语义融合；文本图像融合向量的表达式为:上式中，表示文本图像融合向量，表示注意力机制，Q、K和V分别表示查询矩阵、键矩阵和值矩阵，、和分别表示查询矩阵参数矩阵、键矩阵参数矩阵和值矩阵参数矩阵；S53、将HTML源码和DOM树转化为DOM图，DOM图的顶点集中的每个顶点对应DOM树中的每个节点，若DOM图中与DOM树的父节点与子节点之间位置相对应处建立边，以获得邻接矩阵，即DOM图的顶点为DOM树节点，边表示父子或兄弟关系。

[0041] S54、将文本图像融合向量与DOM图中每个节点连接，以获得更新后的节点特征；更新后的节点特征的表达式为：上式中，表示更新后的第i个节点特征，表示更新前的第i个节点特征，表示文本图像融合向量；S55、将更新后的节点特征输入到若干层图卷积网络GCN中以获得聚合特征，通过GCN层传播节点特征，捕捉DOM图的拓扑关联，如事件绑定路径。

[0042] S56、根据聚合特征进行平均池化以获得第二融合特征；第二融合特征的计算公式为：上式中，表示第二融合特征，N表示DOM图中的节点总数，表示经过L层图卷积网络GCN后获得的聚合特征；S57、通过全连接层将第二融合特征和网页向量映射到统一维度后进行拼接以获得融合向量，并将融合向量输入到DNN模型中，输出风险概率，将多模态融合向量输入DNN模型，输出0-1之间的风险评分。

[0043] S6、输出风险概率大于风险阈值的网页URL，并标记为风险网页，为站群管理者提供管理决策依据。

[0044] 本发明还提供了一种基于低代码的站群管理系统，包括处理器和存储器，存储器用于存储计算机程序，计算机程序被所述处理器执行时实现基于低代码的站群管理方法。

[0045] 本领域普通技术人员可以理解实现上述实施例方法中的全部或部分步骤是可以通过程序来指令相关的硬件来完成，因此，本申请可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本申请可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。

[0046] 本说明书中的各个实施例均采用递进的方式描述，每个实施例重点说明的都是与其他实施例的不同之处，各个实施例之间相同或相似的部分互相参见即可。对于以上各实施例而言，由于其与方法实施例基本相似，所以描述的比较简单，相关之处参见方法实施例的部分说明即可。

[0047] 以上实施方式对本发明进行了详细介绍，本文中应用了具体个例对本发明的原理及实施方式进行了阐述，以上实施例的说明只是用于帮助理解本发明的方法及其核心思想；同时，对于本领域的一般技术人员，依据本发明的思想，在具体实施方式及应用范围上均会有改变之处，综上所述，本说明书内容不应理解为对本发明的限制。< / script>

Claims

1. A site group management method based on low code, characterized in that: The specific steps include: S1. Obtain the webpage URL and webpage attribute characteristics of each site in the site group; S2, simulating user scrolling through a headless browser to obtain a rendered complete web page corresponding to the web page URL, and then extracting the complete HTML source code and hidden text data of the web page; S3, obtaining a full-page screenshot of the rendered complete web page and segmenting it, and inputting the segmented image into the first convolutional neural network model to obtain the image with the highest attention score, and marking it as a preprocessed image; S4, converting the HTML source code, hidden text data and pre-processed image into source code vector, text vector and image vector respectively; S5. Concatenate the source code vector, text vector, image vector, and web page vector into a fusion vector, and input it into the second convolutional neural network model to obtain the risk probability.

2. The station group management method according to claim 1, characterized in that: In step S2, the following steps are specifically included: S21. Calling a headless browser driver to establish a connection with the browser; S22, input the site webpage URL, and drive the browser to execute the JavaScript script embedded in the webpage corresponding to the webpage URL; S23, controlling the browser through a JavaScript script to simulate a user's manual scrolling operation within the webpage, and gradually obtaining the rendered webpage through a display waiting mechanism; S24. Obtain the complete HTML source code and DOM elements of the rendered complete web page; S25. Parse the body tag text in the DOM tree through DOM, and parse the hidden text data from the body tag text.

3. The station group management method according to claim 2, characterized in that: In step S23, the following steps are specifically included: S231, controlling the browser through a JavaScript script to simulate a user's manual scrolling operation within the web page; S232. Obtain webpage data of the rendered webpage through DOM parsing, where the webpage data includes headless browser window data and a DOM structure and AJAX request data of an area where the headless browser window is located; S233. Calculate the DOM node density based on the web page data. The calculation formula is: In the above formula, Indicates the density of DOM nodes, Indicates the total number of DOM nodes in the area where the headless browser window is located. is the area of ​​the headless browser window, which is equal to the product of the length and width of the headless browser window; S234. Calculate the AJAX request frequency based on the web page data. The calculation formula is: In the above formula, Indicates the AJAX request frequency, Indicates the total number of AJAX requests triggered during the rendering process of the area where the headless browser window is located. Indicates the initial loading time of the area where the headless browser window is located during the rendering process; S235, calculating the scrolling time of each scroll according to the DOM node density and the AJAX request frequency; S236: Preset the maximum scrolling speed and calculate the scrolling speed of each scroll according to the DOM node density. The calculation formula is: In the above formula, Indicates the scroll speed, Indicates the maximum scroll speed, is a random number that changes randomly over time; S237, after each scrolling is completed, wait for the corresponding web page area to be completely rendered through the display waiting mechanism; S238. Calculate the product of the scrolling time and the scrolling speed of each scroll to obtain the scrolling distance of each scroll; When the scroll distance reaches 0, the scrolling ends and the rendered complete web page is obtained.

4. The station group management method according to claim 1, characterized in that: In step S3, the following steps are specifically included: S31, obtaining several full-page screenshots of the rendered complete web page corresponding to the web page URL in a preset image acquisition cycle; S32. Obtain the lengths of the long side and the wide side of the full-page screenshot, mark the longer side as the sliding side, mark the sliding direction on the sliding side from left to right and from top to bottom, and mark the length of the shorter side as the sliding distance; S33, along the sliding direction, select the first boundary line perpendicular to the sliding direction on the full-page screenshot and mark it as the initial segmentation edge, and mark another boundary line perpendicular to the sliding direction as the final segmentation edge; S34, setting a distance threshold according to the sliding distance, and sequentially setting a plurality of segmentation edges within the distance threshold between the initial segmentation edge and the final segmentation edge of the full-page screenshot along the sliding direction, until the full-page screenshot is segmented to obtain a plurality of sub-screenshots; S35. Compress the aspect ratio of the sub-screenshot to 1:1 and input it into a neural network including a transposed convolution layer and a self-attention module to obtain several sub-screenshots with the highest attention scores and mark them as preprocessed images.

5. The station group management method according to claim 4, characterized in that: In step S34, the following steps are specifically included: S341, setting a distance threshold according to the sliding distance, and marking the segmentation edge farthest from the initial segmentation edge as the current segmentation edge; S342: Setting a pixel value fusion interval, and establishing a nine-square grid with each pixel point on the full-page screenshot on one side of the current segmentation edge along the sliding direction as the center; S343, sequentially determining whether there is a pixel point in each nine-square grid whose pixel value difference with the center pixel point of the nine-square grid is within the pixel value fusion interval; If they exist, the two are marked as the same connecting line to obtain several connecting lines; If it does not exist, no connection will be made; S344, calculating the distance between the final segmentation edge and the current segmentation edge, and determining whether to extract the sub-screenshot based on whether the distance is less than the maximum value of the distance threshold; If yes, then the image areas between adjacent segmentation edges are output to obtain a number of sub-snipple images, and the process goes to step S348; If not, proceed to the next step; S345: setting a plurality of edges to be segmented that are parallel to the current segmentation edge on the full-page screenshot along the sliding direction, wherein the distances between the edges to be segmented and the current segmentation edge are within a distance threshold; S346, extracting the skeleton line of each connecting line, and sequentially counting the number of intersections between each edge to be segmented and all connecting lines; S347, marking the edge to be split that has the smallest number of intersections and is closest to the sliding distance from the current split edge as a split edge, and then returning to step S341; S348: Calculate the area of ​​each sub-screenshot and determine whether the area of ​​the sub-screenshot with the smallest area is less than the area threshold; If yes, the segmentation is terminated and the process goes to step S35; If not, the sub-screenshot with the smallest area is marked as a full-page screenshot, and then the process returns to step S32.

6. The station group management method according to claim 1, characterized in that: In step S4, the following steps are specifically included: S41, inputting the hidden text data into the BERT model to obtain a first text vector; S42. Match the sensitive word library according to the Trie tree and count the frequency of sensitive words in the hidden text data to obtain a second text vector: S43, delete the third-party library tags in the HTML source code, then parse it into a DOM tree structure, and generate a node sequence vector through a pre-order traversal algorithm; S44, input the node sequence vector into the FastText model and output the source code vector; S45, input each preprocessed image into the EfficientNet model and output the image vector; S46. Obtain the record information, access location, and URL length of the web page corresponding to the web page URL and perform One-Hot encoding to generate a web page vector.

7. The station group management method according to claim 1, characterized in that: In step S5, the following steps are specifically included: S51. Concatenate the first text vector and the second text vector to obtain a text vector; the expression of the text vector is: In the above formula, represents the fused text vector, and represent the first text vector and the second text vector respectively; S52, calculating the value matrix, key matrix, and value matrix of the attention mechanism according to the text vector and the image vector, and performing semantic alignment and fusion through the attention mechanism to generate a text-image fusion vector; S53, converting the HTML source code and the DOM tree into a DOM graph, wherein each vertex in the vertex set of the DOM graph corresponds to each node in the DOM tree, and establishing edges at positions in the DOM graph corresponding to positions between parent nodes and child nodes in the DOM tree to obtain an adjacency matrix; S54, connecting the text-image fusion vector with each node in the DOM graph to obtain updated node features; The expression of the updated node feature is: In the above formula, represents the updated i-th node feature, represents the i-th node feature before updating, Represents the text image fusion vector; S55, input the updated node features into several layers of graph convolutional networks (GCN) to obtain aggregated features; S56. Perform average pooling based on the aggregated features to obtain a second fusion feature. The calculation formula for the second fusion feature is: In the above formula, represents the second fusion feature, N represents the total number of nodes in the DOM graph, Represents the aggregated features obtained after the L-layer graph convolutional network GCN; S57. Map the second fusion feature and the web page vector to a unified dimension through a fully connected layer and then concatenate them to obtain a fusion vector. Input the fusion vector into the DNN model and output the risk probability.

8. The station group management method according to claim 3, characterized in that: In step S235, the calculation formula for the scrolling time is: In the above formula, Indicates the scroll time, 、 and are the first, second, and third regression coefficients obtained through historical data training.

9. The station group management method according to claim 7, characterized in that: In step S52, the expression of the text image fusion vector is: In the above formula, represents the text-image fusion vector, represents the attention mechanism, Q, K, and V represent the query matrix, key matrix, and value matrix respectively. 、 and represent the query matrix parameter matrix, key matrix parameter matrix, and value matrix parameter matrix respectively.

10. The station group management method according to claim 1, characterized in that: A system for implementing the site group management method according to any one of claims 1 to 9, characterized in that it includes a processor and a memory, the memory being used to store a computer program, and the computer program implementing the site group management method according to any one of claims 1 to 9 when executed by the processor.

Images