A method, system, device, and storage medium for detecting and assessing isolated web resources based on natural language.

Abstract

This invention relates to the field of network security technology, specifically a method, system, device, and storage medium for detecting and assessing isolated web resources based on natural language. The method uses a semantic task conversion mechanism to parse the natural language detection requirements of operations and maintenance personnel into structured task instructions, initiating a multi-source detection fusion strategy to scan the target website. This multi-source detection strategy includes access log tracing, website link structure detection, and content semantic association. Cross-validation is used to identify sets of suspected isolated resources. An evolutionary trend prediction model is established, collecting historical page behavior data to predict state transition trends, and using the prediction results as a preliminary warning for isolation determination. A layered confirmation mechanism performs dual verification of suspected isolated resources at both the structural and semantic levels, outputting a list of isolated resources.

Description

Technical Field

[0001] This invention relates to the field of network security technology, and in particular to a method, system, device, and storage medium for detecting and assessing isolated web resources based on natural language. Background Technology

[0002] Enterprise web application systems generate a large number of unnecessary resources during operation, such as isolated web resources that are no longer referenced by the main navigation structure but can still be accessed directly via URLs. These isolated resources arise from various reasons, including legacy pages left over from website redesigns, temporary files not cleaned up during development and testing, plugin components disabled but not removed after content management system upgrades, and obsolete functional modules after business adjustments. The existence of isolated web resources poses a serious threat to enterprise information security. These resources are often outside the scope of operational monitoring, meaning security vulnerabilities cannot be patched in a timely manner, and outdated codebases and third-party components continue to expose known vulnerabilities. Some isolated resources retain sensitive data such as database configuration information and API access keys, providing attackers with a convenient entry point for information gathering.

[0003] Existing technologies for detecting isolated web resources have significant shortcomings. Web crawler-based methods compare the set of reachable pages on a website with the server's file system, but crawlers struggle to cover pages requiring authentication and dynamically generated content, limiting detection coverage. Server log analysis methods identify resources that have been inaccessible for extended periods by counting URL access frequency, but they struggle to distinguish between low-frequency normal access and true isolation, and are limited by log retention periods. Metadata-based methods identify content that hasn't been updated in a long time by checking file modification times, but they cannot determine if the page is still linked. Machine learning classification methods rely on the quality of labeled data; these methods are independent, and existing detection tools are complex to operate, requiring maintenance personnel to master specialized commands and configuration syntax. They cannot understand detection requirements described in natural language, resulting in low human-computer interaction efficiency. Existing methods are primarily passive, only discovering resources after they are completely isolated, lacking the ability to predict page lifecycles and missing opportunities for proactive protection. Risk assessment is limited in scope, mainly focusing on technical vulnerabilities, with insufficient consideration for content sensitivity and business impact, making it difficult to accurately quantify the degree of security threat. Summary of the Invention

[0004] In view of the problems existing in the prior art, the present invention is proposed.

[0005] Therefore, the problem to be solved by this invention is how to address the issues of semantic understanding, multi-source feature fusion, proactive prediction, and risk quantification in existing isolated Web resource detection methods.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, embodiments of the present invention provide a method for detecting and assessing the risks of isolated Web resources based on natural language, which includes receiving natural language detection requirements input from operation and maintenance personnel, and parsing the natural language description into structured detection task instructions through a semantic task conversion mechanism. The structured detection task instructions include the target website to be detected, the feature description of the isolated resource, and the execution constraints. Based on structured detection task instructions, a multi-source detection fusion strategy is initiated to scan the target website and identify a set of suspected isolated resources. Establish an evolution trend prediction model, collect historical behavior data of the page and extract state evolution characteristics, predict the trend of the page changing from active to decay, use the prediction results as the basis for early warning of isolation judgment, verify suspected isolated resources through hierarchical confirmation, and output a list of isolated resources after the hierarchical confirmation results are merged. Establish a risk quantification assessment system to score isolated resources from four assessment dimensions: technical vulnerability, exposure of sensitive information, cumulative effect over time, and business-related impact, and generate risk level reports and disposal priority recommendations.

[0007] As a preferred embodiment of the natural language-driven isolated Web resource detection and risk assessment method described in this invention, the step of parsing natural language descriptions into structured detection task instructions through a semantic task conversion mechanism includes receiving natural language text input by operations and maintenance personnel, performing word segmentation and part-of-speech tagging on the text, and identifying professional terms and key entities in the operations and maintenance field. The intent recognition component determines the type of detection intent of the operation and maintenance personnel. The detection intent types include full scan request, specified area detection request, historical result query request, and risk assessment request. Key detection elements are extracted from natural language text using an entity extraction component. These key detection elements include target website identifier, URL path range, resource type constraints, and time range limitations. The identified intent type and extracted detection elements are combined to generate a structured detection task instruction. The accuracy of the task understanding is confirmed with the operation and maintenance personnel through an interactive interface. The task parameters are adjusted based on the confirmation feedback before the detection process is started.

[0008] As a preferred embodiment of the natural language-driven isolated web resource detection and risk assessment method described in this invention, the step of launching a multi-source detection fusion strategy to scan the target website includes: tracing the path through access logs, reading the server's web access logs, statistically analyzing the changes in the number of accesses and the distribution of access sources for each page within a set observation period, and identifying pages whose access frequency is continuously decreasing and lacks normal referencing sources. By probing the website's link structure, starting from the homepage, we recursively traverse the website according to the link relationships, record the link path and depth information of each visited page, construct a complete website topology diagram, and mark pages that cannot be reached through normal navigation. By using the semantic association pathway, the text content features of each page on the website are extracted, and the similarity between the page content and the core theme content of the website is compared to identify pages with high thematic deviation. The results of the three detection paths are cross-validated. When a page is marked as abnormal in multiple paths, it is included in the suspected isolated resource set, and the specific abnormal characteristics given by each path are recorded.

[0009] The beneficial effects of this preferred technical solution are as follows: the access log tracing path can capture the decay trajectory of page access frequency from server logs and identify those pages that are gradually losing user attention. This path is good at discovering resources with abnormal access patterns but still existing in the file system; the website link structure detection path can draw a complete website topology diagram and directly locate those isolated nodes that are disconnected from the main navigation system. This path is good at discovering pages with structurally broken links; the content semantic association path can identify pages that deviate from the theme and discover those resources whose content is no longer related to the core business of the website. This path is good at discovering semantically isolated pages.

[0010] As a preferred embodiment of the natural language-driven isolated web resource detection and risk assessment method described in this invention, the step of establishing an evolution trend prediction model includes collecting behavioral data of each page in the website within a historical time period. The behavioral data includes page update operation records, user access statistics, page content change records, and external link changes. State evolution features are extracted from behavioral data, including a decreasing trend in update frequency, a rate of decline in access volume, a content freshness index, and a link failure rate. By using time-series analysis methods to learn the changing patterns of page state evolution characteristics, we can predict the likelihood of a page transitioning from an active state to a decaying state in the future. Set a state transition warning threshold. When the decay transition probability of a page exceeds the threshold, mark the page as a high-risk monitoring object in advance and give it higher attention weight in subsequent isolation judgments.

[0011] As a preferred embodiment of the natural language-driven isolated web resource detection and risk assessment method described in this invention, the layered confirmation includes structural layer confirmation and semantic layer confirmation. The structural layer confirmation verifies the connection relationship between the page and the website navigation system, and the semantic layer confirmation verifies the degree of relevance between the page content and the website theme. The verification of suspected isolated resources through layered confirmation includes: performing first-level confirmation at the structural layer, checking whether the suspected isolated page has a link reference from the website's main navigation menu, checking whether the page is included in the site map file, checking whether the page can be reached from the website homepage through limited redirects, and counting the number of valid structural links. Second-level confirmation is performed at the semantic layer to analyze the relevance of the content theme of the suspected isolated page to the overall content theme of the website, check whether the page contains outdated information that is unrelated to the website's business, and assess the actual value of the page content to the website's users. A comprehensive isolation determination score is calculated based on the confirmation results of the structural layer and the semantic layer. When structural connections are missing and semantic associations are weak, the determination score is high, and the resource is identified as an isolated resource. Resources identified as orphaned are categorized and marked, including abandoned test pages, expired content pages, legacy configuration files, and broken functional modules. Different types of orphaned resources will be subject to differentiated risk assessment strategies.

[0012] The beneficial effects of this preferred technical solution are as follows: When performing first-level confirmation at the structural layer, by checking three structural indicators—main navigation menu references, site map inclusion, and homepage accessibility—the connection status of the page in the website architecture can be determined, and resources that have been physically broken can be identified; when performing second-level confirmation at the semantic layer, by analyzing the relevance of the page content theme to the overall theme of the website, checking for outdated information, and evaluating the actual value of the content, the page's belonging relationship in business logic can be determined, and resources that, although linked, no longer conform to the website's positioning can be identified.

[0013] As a preferred embodiment of the natural language-driven method for detecting and assessing the risks of isolated web resources described in this invention, the establishment of a risk quantification assessment system includes: assessing the vulnerability dimension of technology, scanning for security vulnerabilities in isolated resources, identifying outdated technology components and known security flaws, and calculating the technology risk score based on the severity and exploitability of the vulnerabilities. Assess the exposure dimensions of sensitive information, detect whether isolated resources contain sensitive data such as database configuration information, API access keys, internal system addresses, and user credential information, and calculate the exposure risk score based on the category and quantity of sensitive information.

[0014] As a preferred embodiment of the natural language-driven method for detecting and assessing the risks of isolated web resources described in this invention, the establishment of a risk quantification assessment system further includes assessing the cumulative effect of time, and statistically analyzing the duration of isolated resources in an isolated state. The longer the duration, the greater the risk of the resource being attacked, as it has been out of operation and maintenance monitoring for a long time. Assess the business-related impact dimensions, analyze the degree of correlation between isolated resources and core business systems, and evaluate the possible paths and scope of impact for attackers to penetrate the core system through the isolated resource. By combining the scores of the four assessment dimensions, the overall risk score of isolated resources is calculated through a weighted allocation mechanism. Based on the scores, resources are divided into three risk levels: high-risk, medium-risk, and low-risk. Corresponding disposal recommendations are then formulated for each risk level.

[0015] Secondly, embodiments of the present invention provide a natural language-driven isolated web resource detection and risk assessment system, which includes a semantic parsing module for receiving natural language detection requirements input from operation and maintenance personnel and parsing the natural language description into structured detection task instructions through a semantic task conversion mechanism. The multi-source detection module is used to initiate a multi-source detection fusion strategy to scan the target website based on the structured detection task instructions, and obtain a set of suspected isolated resources. The trend prediction module is used to establish an evolution trend prediction model, predict the evolution trend of page state, and use the prediction results as a preliminary warning basis for isolated judgment. The hierarchical verification module is used to verify suspected orphaned resources through a hierarchical verification mechanism and output a list of orphaned resources. The risk assessment module is used to establish a quantitative risk assessment system, score isolated resources in multiple dimensions, and generate risk level reports and disposal priority recommendations. The monitoring and early warning module is used to establish a regular scanning mechanism and a dynamic early warning mechanism to achieve continuous monitoring and risk tracking.

[0016] Thirdly, embodiments of the present invention provide a computer device, including a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program instructions are executed by the processor, they implement the steps of the natural language-driven isolated web resource detection and risk assessment method described in the first aspect of the present invention.

[0017] Fourthly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program instructions are executed by a processor, they implement the steps of the natural language-driven isolated web resource detection and risk assessment method described in the first aspect of the present invention.

[0018] The beneficial effects of this invention are as follows: This invention receives natural language detection requests and parses them into structured task instructions through a semantic task conversion mechanism, freeing maintenance personnel from complex command-line operations. Detection requirements are directly expressed in colloquial language, lowering the operational threshold and shortening task configuration time. A multi-source detection fusion strategy simultaneously initiates three parallel scanning paths: access log tracing, website link structure detection, and content semantic association. Each path has its own focus and complements the others. The access log path captures abnormal access patterns, the link structure path locates broken link nodes, and the semantic association path identifies topic deviations. A cross-validation mechanism requires a page to show abnormalities in multiple paths before being included in the suspected set, improving detection coverage and accuracy. Furthermore, an evolutionary trend prediction model analyzes the page's historical behavior. To predict state transition trends based on data, pages that are about to decay are identified in advance and marked as high-risk monitoring targets, achieving a shift from passive detection to proactive early warning, and giving the operations team a window of opportunity to handle the situation. A layered verification mechanism performs dual verification at both the structural and semantic levels. The structural layer focuses on the page's connection status within the website architecture, while the semantic layer focuses on the business affiliation of the page's content. This dual verification and cross-verification reduces the false positive rate and ensures the reliability of the output list of isolated resources. A risk quantification assessment system scores isolated resources across four dimensions: technical vulnerability, exposure of sensitive information, cumulative effects over time, and business-related impacts. A comprehensive overall risk score is calculated and categorized into levels, with handling recommendations and priorities developed for each level, enabling the operations team to focus on high-risk resources for priority handling. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A flowchart for a natural language-driven method for detecting and assessing isolated web resources; Figure 2 A computer device diagram for a natural language-driven method for detecting and assessing isolated web resources. Detailed Implementation

[0021] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0022] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0023] Secondly, the term "an embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places throughout this specification does not necessarily refer to the same embodiment, nor is it a single embodiment or an embodiment selectively excluded from other embodiments.

[0024] Example 1 Reference Figure 1 - Figure 2 This is the first embodiment of the present invention, which provides a method for detecting and assessing the risks of isolated web resources based on natural language, including: S100: Receives natural language detection requirements from operations and maintenance personnel, and parses the natural language description into structured detection task instructions through a semantic task conversion mechanism. The structured detection task instructions include the target website, isolated resource feature descriptions, and execution constraints.

[0025] S200: Based on structured detection task instructions, initiate a multi-source detection fusion strategy to scan the target website and identify suspected isolated resource sets.

[0026] S300: Establish an evolution trend prediction model, collect historical behavior data of the page and extract state evolution characteristics, predict the trend of the page changing from active to decaying, use the prediction results as a preliminary warning basis for isolation judgment, verify suspected isolated resources through hierarchical confirmation, and output an isolated resource list after the hierarchical confirmation results are merged.

[0027] S400: Establish a risk quantification assessment system to score isolated resources from four assessment dimensions: technical vulnerability, exposure of sensitive information, cumulative effect over time, and business-related impact, and generate risk level reports and disposal priority recommendations.

[0028] It should be noted that this invention addresses the challenge of detecting isolated resources in enterprise web application systems during long-term operation. Isolated resources refer to web content that continues to run on the server, can be directly accessed via URL, but is no longer referenced by the website's main navigation structure. These resources often contain unpatched security vulnerabilities, exposed sensitive information, and outdated technical components, providing attackers with entry points for penetration.

[0029] In step S100, a semantic task conversion mechanism enables operations personnel to submit detection requests in natural language, lowering the operational threshold. The system automatically converts colloquial descriptions into structured task instructions, clarifying detection objectives and execution constraints. In step S200, three parallel scanning paths—access log tracing, link structure detection, and content semantic association—are initiated. These three paths complement and collaborate, improving detection coverage through cross-validation and obtaining a set of suspected isolated resources. In step S300, an evolutionary trend prediction model analyzes historical page behavior data to predict page state transition trends, identifying pages about to enter a decay period in advance, thus shifting from passive detection to proactive warning. Suspected resources are then verified through dual confirmation at the structural and semantic layers. Only when both layers of verification result in isolation is the resource finally confirmed, and an accurate list of isolated resources is output. In step S400, a quantitative assessment system is established based on four dimensions: technical vulnerability, sensitive information exposure, time cumulative effect, and business-related impact. A comprehensive risk score is calculated and classified, and handling recommendations are formulated for each risk level.

[0030] Example 2 Reference Figure 1 - Figure 2 This is the second embodiment of the present invention.

[0031] In this embodiment, step S100 receives the natural language detection request input from the operation and maintenance personnel, and parses the natural language description into a structured detection task instruction through a semantic task conversion mechanism. The structured detection task instruction includes the target website to be detected, the feature description of isolated resources, and the execution constraints, including the following step A1: A1: The semantic task conversion mechanism parses natural language descriptions into structured detection task instructions, including receiving natural language text input by operations and maintenance personnel, performing word segmentation and part-of-speech tagging on the text, and identifying professional terms and key entities in the operations and maintenance field. The intent recognition component determines the type of detection intent of the operation and maintenance personnel. The types of detection intent include full scan request, specified area detection request, historical result query request, and risk assessment request. Key detection elements are extracted from natural language text using an entity extraction component. These key detection elements include target website identifier, URL path range, resource type constraints, and time range limitations. The identified intent type and extracted detection elements are combined to generate a structured detection task instruction. The accuracy of the task understanding is confirmed with the operation and maintenance personnel through an interactive interface. The task parameters are adjusted based on the confirmation feedback before the detection process is started.

[0032] Specifically, the construction process of the word vector space in the operations and maintenance (O&M) domain is as follows: First, a dedicated O&M corpus is collected. This corpus includes network security announcements, vulnerability disclosure reports, server operation and maintenance logs, web security technical documents, and historical detection task records, with a total size of 5 million sentences covering approximately 8,000 O&M professional terms. A word segmentation tool is used to preprocess the corpus, dividing long texts into word sequences and performing part-of-speech tagging. The tagging categories include nouns, verbs, adjectives, professional terms, and entity names.

[0033] After the model is trained, semantically related operational terms in the word vector space are close in distance. For example, the vector cosine similarity between "SQL injection" and "code injection" reaches 0.82, and the similarity between "configuration file" and "environment variable" reaches 0.76. URL pattern terms such as " / admin", " / backup", and " / test" form clusters in the vector space, while security vulnerability terms such as "XSS", "CSRF", and "file inclusion" form another cluster.

[0034] Specifically, the vectorization encoding process for natural language text is as follows: When maintenance personnel input a natural language detection requirement, such as "check if the company website has a backend management page that has not been accessed in the past month," the system first performs word segmentation on the text, resulting in the word segmentation result being "check / company / website / whether / exists / within / a / month / without / being / accessed / by / the / backend / management / page". For each word, the system queries the word vector table to obtain the corresponding 300-dimensional vector representation. For rare words not found in the vocabulary, a character-level encoding method is used, splitting the word into a character sequence. Each character is mapped to a predefined character embedding vector, with a character vector dimension of 50. The character sequences are then fused through convolution operations to obtain a word-level representation.

[0035] The segmented word vector sequence is input into a pre-trained language model for context encoding. This pre-trained model employs a bidirectional encoding architecture, containing 12 transformer layers, each layer including a multi-head self-attention module and a feedforward neural network module. The multi-head self-attention module has 12 heads, each with a 64-dimensional dimension, for a total attention dimension of 768. During the self-attention calculation, for each word in the input sequence, its attention weight with all words in the sequence is calculated using the following formula: in, This represents the query vector for the i-th word. Let d represent the key vector of the j-th word, and d be the attention dimension. This represents the attention weight of the i-th word to the j-th word. The value vector is weighted and summed using these attention weights to obtain the word representation that takes into account contextual information.

[0036] After 12 layers of transformer encoding, each word in the sequence obtains a 768-dimensional vector representation that incorporates contextual information. This vector simultaneously encodes the semantic information of the word and its positional information in the sentence. In the example sentence, the encoded vectors of the words "back-end" and "management" are closely related in the context, and their mutual influence allows the model to understand "back-end management" as a holistic concept.

[0037] Specifically, the construction process of the intent recognition component is as follows: Define a set of detection intent categories, including four intent types: full scan request, specified area detection request, historical result query request, and risk assessment request. Collect labeled samples for each intent type. Full scan request samples include "perform isolated resource detection on the entire website" and "scan all possible isolated pages". Specified area detection request samples include "check isolated resources in the backend management directory" and "scan only the upload folder". Historical result query request samples include "which isolated resources were found in the last detection" and "view this month's detection report". Risk assessment request samples include "assess the security risks of found isolated resources" and "calculate the number of high-risk resources".

[0038] The sentence representation vector is input into a classifier for intent determination. The classifier uses a three-layer fully connected neural network structure, and its output is converted into a probability distribution using a softmax function. in, The output score for class c. This represents the predicted probability for that class.

[0039] Specifically, the construction process of the entity extraction component is as follows: Define an entity type annotation system, including four categories: target website identifier entity, URL path entity, resource type entity, and time range entity; the target website identifier entity includes domain name and IP address, the URL path entity includes specific path string and path pattern expression, the resource type entity includes file type and page type description, and the time range entity includes absolute time and relative time expression.

[0040] Entity identification is performed using sequence labeling, with the BIO (Browser-Indexed Identifier) ​​tagging scheme. B indicates the start of an entity, I indicates an internal entity position, and O indicates a non-entity position. Each entity type corresponds to a unique B and I tag; for example, B-DOMAIN indicates the start of a domain entity, and I-DOMAIN indicates an internal domain entity, resulting in a total of nine tags.

[0041] A sequence labeling model is constructed, with the input being a 768-dimensional word vector sequence encoded by a pre-trained language model. This vector sequence is then further encoded into a bidirectional Long Short-Term Memory (LSTM) network, which consists of two layers, each with 256 hidden units. The forward and backward hidden states are concatenated to obtain a 512-dimensional context representation. The LTM units control information flow through a gating mechanism: the forget gate determines which historical information is discarded, the input gate determines which new information is received, and the output gate determines which information is output.

[0042] The output sequence of a bidirectional long short-term memory network is input into a conditional random field (CRF) layer for global optimization. The CRF layer maintains a transition matrix that records the transition probabilities between labels; for example, B-DOMAIN is usually followed by I-DOMAIN instead of B-PATH. Given the input sequence and its corresponding label sequence, the CRF layer calculates the score for the entire sequence. in, Let be the hidden representation of the i-th position. Let be the label at the i-th position. This is the weight vector corresponding to the label. For from the tag Transferred to The score, where n is the sequence length.

[0043] In this embodiment, step S200, based on structured detection task instructions, initiates a multi-source detection fusion strategy to scan the target website and identifies a set of suspected isolated resources, including the following step B1: B1: Launch a multi-source detection and fusion strategy to scan the target website, including tracing the path through access logs, reading the server's web access logs, statistically analyzing the changes in the number of accesses and the distribution of access sources for each page within a set observation period, and identifying pages with continuously decreasing access frequency and lacking normal referencing sources. By probing the website's link structure, starting from the homepage, we recursively traverse the website according to the link relationships, record the link path and depth information of each visited page, construct a complete website topology diagram, and mark pages that cannot be reached through normal navigation. By using the semantic association pathway, the text content features of each page on the website are extracted, and the similarity between the page content and the core theme content of the website is compared to identify pages with high thematic deviation. The results of the three detection paths are cross-validated. When a page is marked as abnormal in multiple paths, it is included in the suspected isolated resource set, and the specific abnormal characteristics given by each path are recorded.

[0044] Specifically, the implementation process of the access log tracing path is as follows: First, the web access log file is read from the server, supporting Apache's Combined log format, Nginx's default log format, and IIS's W3C extended log format. The log file records detailed information about each HTTP request in chronological order, including the request timestamp, client IP address, request method, request URL, response status code, response byte count, and referrer.

[0045] Calculate the decay trend of access frequency, and determine the page's activity level by comparing the changes in access frequency over three time periods; for pages showing a continuous downward trend, calculate the access frequency decay rate: in, For early access frequency, The recent access frequency is calculated by adding 0.01 to the denominator to prevent division by zero errors. λ is the decay rate; the larger the decay rate, the faster the access frequency decreases. When the decay rate exceeds 0.7 and the recent access frequency is less than twice a day, the page is considered to have access abnormalities.

[0046] Analyze the access source distribution for each URL, extract the Referer field from the logs, which records which page the user navigated from; categorize the sources into three types: direct access sources (indicated by an empty Referer field or the browser's default value), internal referral sources (indicated by the domain name in the Referer field being the same as the current website), and external referral sources (indicated by the domain name in the Referer field being different from the current website).

[0047] An anomaly score is calculated based on both the frequency of access decreases and the anomaly in the source distribution. The anomaly score is calculated using a weighted summation method: The weighting coefficients 0.6 and 0.4 are determined through historical data statistics. When the log anomaly score exceeds 0.6, the URL is marked as a suspected isolated resource in the access log tracing path, and the URL, decay rate, direct access ratio, and anomaly score are output.

[0048] Specifically, the implementation process of website link structure detection is as follows: The homepage URL of the target website is used as the seed URL and added to the crawling queue. A URL is retrieved from the queue and an HTTP request is made to obtain the page's HTML content. The HTML content is parsed to extract hyperlink tags, including the href attribute of the tag, the src attribute of the <frame> tag, and the src attribute of the <iframe> tag.

[0049] Filter out links pointing to external sites, links pointing to static resources, and links containing special parameters. Add the filtered new links to the crawling queue, and record the parent-child relationship between the link and the current page. Maintain a set of inbound links for each URL, recording all parent pages pointing to that URL. Repeat the above crawling process until the queue is empty or the number of crawled pages reaches the preset limit.

[0050] A breadth-first search strategy is used to control the crawling order, prioritizing pages with shallower crawling depths. For each URL, the shortest path length from the homepage is recorded as its depth value, with the homepage having a depth of 0, pages directly linked from the homepage having a depth of 1, and so on. For multiple URLs with the same depth, they are sorted according to their position within the parent page, with links in the navigation menu having higher priority than links in the main content.

[0051] Concurrent crawling is implemented using an asynchronous coroutine framework to improve crawling efficiency. A coroutine pool of 50 concurrent coroutines is created, each independently executing URL request and parsing tasks. Semaphores are used to control the number of concurrent requests, preventing excessive pressure on the target server. The request interval is set to 100 milliseconds, adhering to the Robots Exclusion Protocol crawling rules. For requests returning status codes 429 or 503, exponential backoff retries are performed, with retry intervals of 1 second, 2 seconds, and 4 seconds respectively.

[0052] After crawling, construct a website link graph. The nodes of the graph represent all crawled URLs, and the edges represent the links between those URLs. Calculate the length of each node, i.e., the number of edges pointing to that node. A node with an in-degree of 0 indicates that no page links to that node; these nodes cannot be reached through normal navigation. Check if each node with an in-degree of 0 appears in the website's main navigation menu. The navigation menu is located by identifying the `nav` tag in HTML and `div` tags containing the class names "navigation" or "menu". Check if the node appears in the `sitemap.xml` file, located in the website's root directory, which contains a list of URLs that the website wants search engines to index.

[0053] For nodes with an in-degree of 0 that are not in the navigation menu or site map, mark them as suspected orphaned resources in the website link structure detection path. For nodes with an in-degree greater than 0 but an abnormally low in-degree value, such as an in-degree of 1 whose only inbound link comes from a deprecated page, also mark them as suspected orphaned resources. Output the URL, in-degree value, depth value, and a list of inbound links for each suspected resource.

[0054] Specifically, the implementation process of the content semantic association path is as follows: Extract the content from each crawled page, remove HTML tags, JavaScript scripts, and CSS style codes, and retain the plain text content. Perform word segmentation and stop word filtering on the text content. Stop words include high-frequency meaningless words such as "de", "le", "zai", etc. Calculate the word frequency of each word in the document and the document frequency in the entire website.

[0055] Use the term frequency-inverse document frequency method to calculate the importance weight of the word. The calculation formula is: Among them, is the weight of word i in document j, is the word frequency of word i in document j, N is the total number of documents in the website, is the number of documents containing word i. Words with high word frequency and appearing in a small number of documents obtain higher weights. Such words are usually the key topic words of the document.

[0056] Construct a term frequency-inverse document frequency vector for each page. The vector dimension is the size of the vocabulary. The vocabulary contains all words that appear more than 5 times in the website. The size of the vocabulary is usually between 5000 and 10000. Each component of the vector corresponds to the weight value of a word, and the weight of words that do not appear is 0. Perform L2 normalization on the vector to make the vector length 1.

[0057] Calculate the topic center vector of the website. This vector represents the core content theme of the website. The topic center vector is obtained by weighted averaging all page vectors, and the weights are determined according to the importance of the pages; pages with high authority such as the home page, navigation page, and column page contribute greater weights when calculating the topic center vector.

[0058] Calculate the cosine similarity between each page vector and the topic center vector: Among them, is the i-th component of the page vector, is the i-th component of the topic center vector, m is the vector dimension, and sim is the similarity value. The similarity value ranges from 0 to 1, and the larger the value, the more relevant the page content is to the website theme. [[ID=**********]]

[0059] For pages with a similarity lower than 0.3, they are determined to be pages with a high degree of topic deviation. Further analyze the content characteristics of these pages to identify pages containing test text, Lorem Ipsum placeholders, and abnormal character encodings. Identify pages with obviously outdated content, judged by the time markers in the pages, such as the publication date of news pages being earlier than 3 years ago. Identify content unrelated to the website's business, such as personal blog content appearing on the enterprise official website.

[0060] The topic deviation is used as a semantic anomaly score, outputting the URL, similarity value, topic deviation, and content feature tag for each suspected resource. Pages with a similarity between 0.3 and 0.5 are marked as having moderate semantic deviation, requiring a comprehensive judgment based on results from other pathways.

[0061] Specifically, the cross-validation process for the results of the three detection pathways is as follows: Collect the lists of suspected isolated resources marked by the access log tracing pathway, the website link structure detection pathway, and the content semantic association pathway, respectively. Merge and deduplicate the URLs in the three lists to obtain a set of all URLs marked by at least one pathway.

[0062] A feature vector is created for each URL, containing three components corresponding to the detection results of the three pathways. The first component is the log anomaly score, ranging from 0 to 1; URLs not marked by the log pathway have a value of 0 for this component. The second component is the structural orphanage score, obtained by normalizing the in-degree value; URLs with an in-degree of 0 have a value of 1 for this component, and the larger the in-degree, the smaller the value. The third component is the semantic deviation score, which is 1 minus the similarity value; URLs not marked by the semantic pathway have a value of 0 for this component.

[0063] In this embodiment, step S300 establishes an evolution trend prediction model, collects historical behavior data of the page and extracts state evolution features, predicts the trend of the page changing from active to decaying, uses the prediction results as a preliminary warning basis for isolation determination, verifies suspected isolated resources through hierarchical confirmation, and outputs an isolated resource list after the hierarchical confirmation results are merged, including the following C1-C2 steps: C1: Establish an evolution trend prediction model, including collecting behavioral data of each page on the website within a historical time period. The behavioral data includes page update operation records, user access statistics, page content change records, and external link changes. State evolution features are extracted from behavioral data, including the decreasing trend of update frequency, the rate of decline in access volume, content freshness index, and the proportion of broken links. By using time-series analysis methods to learn the changing patterns of page state evolution characteristics, we can predict the likelihood of a page transitioning from an active state to a decaying state in the future. Set a state transition warning threshold. When the decay transition probability of a page exceeds the threshold, mark the page as a high-risk monitoring object in advance and give it higher attention weight in subsequent isolation judgments.

[0064] Specifically, the process of collecting historical page behavior data is as follows: The system maintains a page behavior database, which is indexed according to the page URL and records the historical behavior trajectory of each page. Page update operation records are derived from the modification timestamps of the file system and the version history of the content management system; the modification time is recorded each time a page file is modified. The timeline is divided into weeks, and the number of updates each week is counted to form an update frequency time series.

[0065] External link changes are obtained by parsing links on the page and performing availability checks. All links pointing to external websites are extracted from the page, and an HTTP HEAD request is sent to each external link to check its availability. A 200 status code indicates a link is broken, while 404, 500, or other error codes indicate a broken link. The total number of links and the number of broken links for each page are counted, and the link failure rate is calculated. Link checks are performed weekly to create a time series of link failure rates.

[0066] The four types of data were aligned with a uniform time granularity, and the week was chosen as the basic time unit. For each page, data was collected at 52 points in time over the past 52 weeks, forming a time series of length 52. The four types of data formed four time series: update frequency series, traffic series, content change series, and link failure rate series.

[0067] Specifically, the process of extracting state evolution features is as follows: Extract the decreasing trend of update frequency from the update frequency time series. Calculate the change in update frequency between adjacent weeks to form a first-order difference sequence. Perform linear regression fitting on the difference sequence; the slope of the regression line reflects the changing trend of the update frequency. A negative slope indicates a decreasing trend in update frequency, and a larger absolute value of the slope indicates a faster rate of decrease. Simultaneously, calculate the ratio of the average update frequency of the most recent 12 weeks to the average update frequency of earlier time periods; a ratio less than 1 indicates a recent decrease in updates.

[0068] Extract the visitor decline rate feature from the visitor time series. Divide the 52 weeks into two periods: the first 26 weeks and the last 26 weeks. Calculate the average visitor count for each period. Calculate the ratio of the average visitor count for the last 26 weeks to the average visitor count for the first 26 weeks. A ratio less than 0.5 indicates a significant decline in visitor count. Perform exponential smoothing on the visitor series with a smoothing coefficient of 0.3 to obtain the trend component. Calculate the rate of change of the trend component. A negative rate of change with an absolute value greater than 0.1 indicates a continuous decline in visitor count.

[0069] Content freshness metrics are extracted from the content change sequence. The number of weeks with content changes within the last 12 weeks is counted; the percentage of weeks with changes reflects the activity level of content updates. The number of weeks since the last content change is calculated; a change more than 12 weeks ago indicates that the content has not been updated for a long time. A content freshness score is calculated by combining the change frequency and the last change date; a freshness score below 0.3 indicates that the content is outdated.

[0070] Extract link health features from the time series of link failure rates. Calculate the failure rate of the most recently detected links; a failure rate exceeding 0.3 indicates poor link health. Count the number of weeks the failure rate has been continuously rising; a continuous rise for more than 8 weeks indicates poor page maintenance. Calculate the growth rate of failed links; a high growth rate indicates rapid failure of external dependencies.

[0071] C2: Layered confirmation includes structural layer confirmation and semantic layer confirmation. Structural layer confirmation verifies the connection between the page and the website navigation system, while semantic layer confirmation verifies the relevance of the page content to the website theme. Verification of suspected orphaned resources is carried out through layered confirmation, including first-level confirmation at the structural layer, checking whether the suspected orphaned page has links from the website's main navigation menu, checking whether the page is included in the sitemap file, checking whether the page can be reached from the website homepage through limited redirects, and counting the number of valid structural links. Second-level confirmation is performed at the semantic layer to analyze the relevance of the content theme of the suspected isolated page to the overall content theme of the website, check whether the page contains outdated information that is unrelated to the website's business, and assess the actual value of the page content to the website's users. A comprehensive isolation determination score is calculated based on the confirmation results of the structural layer and the semantic layer. When structural connections are missing and semantic associations are weak, the determination score is high, and the resource is identified as an isolated resource. Resources identified as orphaned are categorized and marked, including abandoned test pages, expired content pages, legacy configuration files, and broken functional modules. Different risk assessment strategies are then applied to different types of orphaned resources.

[0072] Specifically, the implementation process for the first-level structural layer confirmation is as follows: For each page URL in the suspected orphaned resource set, first check if the URL appears in the website's main navigation menu. The main navigation menu is usually located at the top or side of the website's homepage and can be identified through HTML structural features. Look for the `nav` tag in the HTML; this tag is a semantic markup tag specifically used in HTML5 to mark navigation areas. Look for `div` or `ul` tags with class attributes containing keywords such as "navigation", "navbar", "menu", or "nav". Extract all links within the identified navigation area and determine if the URL of the suspected orphaned page is among these links.

[0073] Check if the URLs are included in the sitemap.xml file, which is a list of URLs provided by the website to search engines and is usually located in the website's root directory. Send an HTTP request to retrieve the content of the sitemap.xml file, parse the XML format, and extract the url tags. Compare the URLs of suspected orphaned pages with the URLs in the sitemap one by one to determine if there are any matches. For large websites that use sitemap index files, recursively parse all sub-sitemap files.

[0074] Check if a URL can be reached from the website homepage via a limited number of redirects, using the previously constructed website link graph for path searching. Use a breadth-first search algorithm, starting from the homepage node and traversing it, recording the path length to each node. Set a maximum search depth of 5, meaning pages reachable from the homepage in 5 clicks or less are considered reachable. For suspected orphaned pages, query their path length in the link graph; paths exceeding 5 or nonexistent are considered unreachable.

[0075] Count the number of structural links to suspected orphaned pages, including navigation menu links, sitemap inclusions, homepage reachable paths, and references from other active pages. Extract all inbound links pointing to this page from the link graph, filtering out inbound links from confirmed orphaned resources and retaining inbound links from active pages. Calculate the ratio of the number of inbound links to the website's average number of inbound links; a ratio less than 0.1 indicates a severe shortage of structural links.

[0076] The structural isolation degree is calculated based on the above inspection results. The calculation formula takes into account the combined influence of multiple factors: in, The navigation menu is marked with a flag: 1 for inclusion and 0 for exclusion. Include markers for the site map; depth is the path length from the homepage, taking 5 if it exceeds 5; This represents the structural isolation level, with a value ranging from 0 to 1. A higher isolation level indicates that the page is more isolated within the website structure.

[0077] Specifically, the implementation process of the semantic layer 2 verification is as follows: Deep content analysis is performed on suspected isolated pages to extract the complete text content, including the title, body text, meta description, and keyword tags. A pre-trained language model is used to encode the text. This language model employs the same architecture as the S100 step and is capable of capturing the semantic information of the text. The encoded vectors are then subjected to average pooling to obtain the semantic vector representation of the page.

[0078] The cosine similarity between the page's semantic vector and the website's topic center vector is calculated; a similarity below 0.3 indicates topic deviation. Further analysis of the page's topic distribution is conducted using a topic model implemented through a Latent Dirichlet distribution. This model assumes that each page consists of a mixture of multiple topics, with each topic represented by a probability distribution of a set of words.

[0079] After training, the topic model can infer the topic distribution vector for any page. The vector length is 20, and each component represents the probability that the page belongs to the corresponding topic. The overall topic distribution of the website is calculated by averaging the topic distributions of all active pages. KL divergence is used to measure the difference between the topic distribution of suspected orphaned pages and the overall topic distribution. in, The distribution of topics on the page, DKL represents the overall theme distribution of the website, and KL represents the KL divergence value. The larger the divergence value, the greater the deviation between the page theme and the website theme.

[0080] Checking for outdated information on a page is achieved by identifying timestamps within the page. This involves extracting time tags, date-formatted text (e.g., 2020-01-15), and text containing keywords such as "published on" or "updated on" from the HTML. The timestamps are then parsed to obtain the specific date, and the number of days since that date is calculated. For news and blog pages, pages published more than 1095 days (3 years) are considered outdated; for product introduction and technical document pages, pages updated more than 730 days (2 years) are considered outdated.

[0081] Assess the actual value of a page's content using a comprehensive evaluation across multiple dimensions. Check for placeholder text such as "LoremIpsum," as this indicates the page is incomplete. Count the page; fewer than 100 Chinese characters or 300 English words suggest thin content. Include meaningful structured information such as tables, lists, images, and videos; pages lacking structured elements have lower value. Check if the page is indexed by search engines using the search syntax `site:domain.com intitle:pagetitle`; failure to index the page indicates a low search engine value.

[0082] Semantic isolation is calculated based on the above analysis results, taking into account topic deviation, content obsolescence, and actual value. Pages with high topic deviation, outdated content, and low actual value receive high semantic isolation. The formula for calculating semantic isolation is: in, The terms "topic similarity," "outdated" indicates obsolescence, and "value" represents the content's value score. This represents semantic isolation.

[0083] Specifically, the process of comprehensive isolation determination and resource classification is as follows: The comprehensive isolation probability is calculated by combining structural isolation degree, semantic isolation degree, and evolutionary trend prediction results. The three factors are weighted and fused together. in, , , These are the weighting coefficients. The probability of inactive states is predicted based on the evolutionary trend. For pages marked as high-risk monitoring targets, the weighting coefficient is adjusted to... =0.45、 =0.35、 =0.20; for ordinary pages, the weight coefficient is =0.4、 =0.3、 =0.3.

[0084] An orphanage threshold of 0.65 is set; pages with a comprehensive orphanage probability exceeding 0.65 are identified as orphaned resources. Orphaned resources are then categorized based on URL and content characteristics.

[0085] Discarded test pages are identified by the following characteristics: the URL path contains keywords such as "test", "demo", "example", and "sandbox"; the page title contains words such as "test", "demonstration", and "example"; the page content contains test data or fake data tags; and the page's robots meta tag is set to noindex.

[0086] Characteristics of expired content pages: The page was published more than 3 years ago; the events, products, and technologies mentioned on the page are outdated; the page has had zero visits recently; the topic discussed on the page is no longer the focus of the website.

[0087] Identifying characteristics of legacy configuration files: The file extensions are .conf, .ini, .yaml, .json, and .xml; the file contains configuration information such as database connection strings and API keys; the file is located in a non-public directory but can be directly accessed; the file does not have access permission protection.

[0088] Identification characteristics of failed functional modules: The page implements a business function that has been discontinued; the backend interface that the page depends on has become invalid and returned an error; the page's JavaScript has an error and cannot run normally; the page style is messed up and indicates that resource loading has failed.

[0089] Each isolated resource is labeled with a corresponding type. These labels are used to select differentiated assessment strategies during subsequent risk assessment. The final list of isolated resources is output, which includes the URL, overall isolation probability, structural isolation degree, semantic isolation degree, predicted state probability, resource type label, and detailed feature information.

[0090] In this embodiment, step S400 establishes a risk quantification assessment system, scoring isolated resources from four assessment dimensions: technical vulnerability, exposure of sensitive information, cumulative effect over time, and business-related impact, generating a risk level report and disposal priority recommendations, including the following D1 step: D1: Establish a risk quantification assessment system, including assessing the dimension of technical vulnerability, scanning for security vulnerabilities in isolated resources, identifying outdated technical components and known security flaws, and calculating technical risk scores based on the severity and exploitability of vulnerabilities. Assess the exposure dimensions of sensitive information, detect whether isolated resources contain sensitive data such as database configuration information, API access keys, internal system addresses, and user credential information, and calculate the exposure risk score based on the category and quantity of sensitive information.

[0091] To assess the cumulative effect over time, we need to count the duration during which isolated resources remain isolated. The longer the duration, the more the resource has been out of operation and maintenance monitoring for a long time, and the greater the cumulative risk of being attacked. Assess the business-related impact dimensions, analyze the degree of correlation between isolated resources and core business systems, and evaluate the possible paths and scope of impact for attackers to penetrate the core system through the isolated resource. By combining the scores of the four assessment dimensions, the overall risk score of isolated resources is calculated through a weighted allocation mechanism. Based on the scores, resources are divided into three risk levels: high-risk, medium-risk, and low-risk. Corresponding disposal recommendations are then formulated for each risk level.

[0092] Specifically, the implementation process of the technical vulnerability assessment is as follows: Security vulnerability scans are initiated for each isolated resource. The scanner uses an active probing approach to detect common web vulnerabilities. SQL injection vulnerability detection involves injecting special characters and SQL statement fragments into URL parameters and observing whether database error messages or abnormal data appear in the returned results. Cross-site scripting vulnerability detection involves injecting JavaScript code and checking whether it is executed. File inclusion vulnerability detection involves attempting to include sensitive system files such as / etc / passwd and observing the returned content.

[0093] Command injection vulnerability detection checks for the ability to execute arbitrary commands by injecting system command delimiters and common commands. XML external entity injection vulnerability detection involves constructing a malicious XML payload and observing whether it can read files or initiate SSRF requests. Server request forgery vulnerability detection involves specifying an internal network address in the parameters and observing whether the server will send requests to that address.

[0094] Parse the HTML content of isolated resources to extract referenced third-party JavaScript libraries. Identify the library file URL using the `src` attribute of the `<script>` tag, and extract the library name and version number from the URL. Common libraries include jQuery, Bootstrap, Angular, React, and Vue. Compare the extracted library name and version number with a CVE vulnerability database, which contains publicly disclosed software vulnerabilities.

[0095] The system queries the known vulnerability list for each database. Vulnerability information includes the CVE number, CVSS score, vulnerability type, and affected version range. The CVSS score uses version 3.0 and assesses vulnerability severity across three dimensions: base score, time score, and environment score. The base score considers factors such as attack vector, attack complexity, required privileges, user interaction, and scope of impact. The score range is 0 to 10, with 0-3.9 being low risk, 4.0-6.9 being medium risk, 7.0-8.9 being high risk, and 9.0-10 being severe.

[0096] For each detected vulnerability, its CVSS baseline score is recorded. The technical vulnerability score for isolated resources is calculated, taking into account the combined impact of the number and severity of vulnerabilities. Where n is the total number of vulnerabilities detected, c For the CVSS score of the i-th vulnerability, The number of high-risk and critical vulnerabilities, This is a score for technical vulnerability. The first item calculates the average of the vulnerability scores and normalizes them. The second item adds a penalty based on the percentage of high-risk vulnerabilities. The score ranges from 0 to 1.2, with values ​​exceeding 1 truncated to 1.

[0097] Specifically, the implementation process for sensitive information exposure assessment is as follows: Sensitive information is identified in the content of isolated resources using a combination of regular expression matching and keyword detection. The identification patterns for database configuration information include keywords such as "host=", "database=", "username=", and "password="; connection string formats for databases such as MySQL, PostgreSQL, and MongoDB; and JDBC connection URL formats.

[0098] API access key identification patterns include: AWS access key format (20-character string starting with AKIA); GitHub personal access token format (36-character string starting with ghp_); API key format (32 or 64-bit hexadecimal string); and JWT token format (three-segment Base64 encoded string).

[0099] The identification patterns for internal system addresses include: private IP address ranges such as 10.0.0.0 / 8, 172.16.0.0 / 12, and 192.168.0.0 / 16; internal domain names such as *.local, .internal, and .corp; and combinations of server hostnames and ports.

[0100] User credential information identification patterns include: plaintext passwords appearing in HTML comments or JavaScript variables; hard-coded username and password combinations; and common usernames and simple passwords such as "admin", "root", and "test".

[0101] The identification patterns for sensitive business data include: personal information such as ID card number, mobile phone number, and email address; financial information such as credit card number and bank account number; and business information such as internal company documents, contracts, and quotations.

[0102] Each type of sensitive information identified is counted and statistically analyzed, and a severity weight is defined for each type. Database credentials have a weight of 0.3, API keys have a weight of 0.25, internal addresses have a weight of 0.15, user credentials have a weight of 0.2, and business data has a weight of 0.1. Sensitive information exposure scores are calculated: Among them, w The weight of the c-th type of sensitive information, The number of times this type of information appears. This is a score for the exposure of sensitive information. (Used...) c / ( +1) Normalization is performed so that even if the same type of information appears multiple times, the score will not increase indefinitely. The score range is 0 to 1.

[0103] D2: Specifically, the implementation process for assessing the cumulative effect of time is as follows: Determine the time point when an isolated resource first enters an isolated state. This time point is inferred in the following ways: Query the historical snapshot of the website's link relationship graph to find the time point when the page last had an inbound link; query the web server access log to find the time point when the page was last accessed normally; query the time point when the page's content was last updated. If there are no references for a long time after the update, the update time is used as the starting point for isolation.

[0104] Take the most recent of the three time points as the isolation start time, and calculate the difference in days from the isolation start time to the current time to obtain the isolation duration. The isolation duration reflects the length of time that the resource has been out of operation and maintenance monitoring. The longer the duration, the longer the security issues have accumulated, and the greater the possibility that they can be discovered and exploited by attackers.

[0105] Using an exponential function to model the time-cumulative effect makes the risk grow non-linearly over time: in, The isolation duration is 180, which is the time sensitivity parameter. This represents the score for the cumulative effect over time. The score is 0 when the number of isolated days is 0, and approaches 1 as the number of isolated days approaches infinity. The score is approximately 0.632 for 180 days of isolation and approximately 0.865 for 360 days of isolation, reflecting the characteristic of risk accumulating rapidly over time.

[0106] Specifically, the implementation process for business impact assessment is as follows: The importance level of the server hosting the isolated resource is analyzed. Asset levels are obtained through configuration data from the operations and maintenance management system. Assets are divided into four levels according to importance: Level 1 is for testing and development environments, Level 2 is for non-core business systems, Level 3 is for core business support systems, and Level 4 is for core business systems. The business impact of an attack on assets of different levels varies.

[0107] Assess possible paths an attacker might take to penetrate the core system from an isolated resource. Path analysis is based on network topology and system architecture. Query the network connectivity between the server hosting the isolated resource and other system servers, including firewall rules, network ACLs, and VPC subnetting. If the isolated resource server and the core business server are on the same network segment and there is no access control, the path reachability is high.

[0108] Examine other applications and services running on the server. If the server is also running core business applications, an attacker can directly access these core applications after compromising the server through isolated resources, indicating high path reachability. Also, examine the server's permission configuration. If an isolated resource application is running with a high-privilege user, an attacker can further escalate privileges after gaining code execution permissions, again indicating high path reachability.

[0109] Calculate the path reachability score, which comprehensively considers the degree of network isolation, application isolation, and access control isolation. A completely isolated path has a reachability score of 0, while a completely unisolated path has a reachability score of 1. Calculate the business impact coefficient based on asset level and path reachability. in, For asset importance level, For path reachability score, t represents the business impact coefficient. The formula ensures that even if the path reachability is 0, high-grade assets still have a base impact score of 0.7.

[0110] In summary, by constructing a word vector space specifically for the operations and maintenance domain, security terms in the vector space satisfy semantic similarity constraints, achieving a high level of similarity between SQL injection and code injection, and establishing semantic association between configuration files and environment variables. Through multi-head self-attention mechanisms and bidirectional encoding architecture, contextual encoding of input text enables complex concepts such as "backend management" to be understood as a whole by the model, achieving high levels of intent recognition accuracy and entity recognition metrics. The access log tracing path quantitatively identifies pages with abnormal access patterns by calculating access frequency decay rates and the proportion of direct access. The website link structure detection path achieves concurrent crawling and constructs a complete topology structure through an asynchronous coroutine framework. The diagram illustrates how content semantic association pathways identify topics deviating from the page using the term frequency-inverse document frequency method and cosine similarity calculation. Cross-validation of the three pathways is weighted and fused to obtain a comprehensive detection score. A long short-term memory network is used to learn the evolutionary patterns of temporal features and predict the probability of page state transitions. When the cumulative probability of inactive states exceeds a threshold, an alert is triggered and higher attention weight is given. Structural isolation is confirmed by checking navigation menus, site maps, and homepage accessibility to calculate structural isolation. Semantic isolation is confirmed by using topic models and KL divergence to calculate semantic isolation. By integrating isolation probabilities across structural, semantic, and predictive dimensions, the confirmed isolated resources are categorized to provide a basis for differentiated risk assessment.

[0111] Example 3 The above is an illustrative scheme of a natural language-driven method for detecting and assessing isolated web resources. It should be noted that the technical solution of this natural language-driven system for detecting and assessing isolated web resources belongs to the same concept as the aforementioned method. Details not described in detail in the natural language-driven system in this embodiment can be found in the description of the aforementioned method.

[0112] This embodiment also provides a natural language-driven isolated web resource detection and risk assessment system, including: The semantic parsing module is used to receive natural language detection requirements input from operations and maintenance personnel, and parse the natural language description into structured detection task instructions through a semantic task conversion mechanism. The multi-source detection module is used to scan the target website based on structured detection task instructions and initiate a multi-source detection fusion strategy to obtain a set of suspected isolated resources. The trend prediction module is used to establish an evolution trend prediction model, predict the evolution trend of page state, and use the prediction results as a preliminary warning basis for isolated judgment. The hierarchical verification module is used to verify suspected orphaned resources through a hierarchical verification mechanism and output a list of orphaned resources. The risk assessment module is used to establish a quantitative risk assessment system, score isolated resources in multiple dimensions, and generate risk level reports and disposal priority recommendations. The monitoring and early warning module is used to establish a regular scanning mechanism and a dynamic early warning mechanism to achieve continuous monitoring and risk tracking.

[0113] This embodiment also provides an electronic device suitable for the detection and risk assessment of isolated Web resources based on natural language, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the method for detecting and assessing isolated Web resources based on natural language as proposed in the above embodiment.

[0114] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, it implements the method for detecting and assessing isolated Web resources based on natural language as proposed in the above embodiments.

[0115] The storage medium proposed in this embodiment and the method for detecting and assessing isolated web resources based on natural language driving proposed in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0116] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0117] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for detecting and assessing the risks of isolated Web resources based on natural language, characterized in that: This includes receiving natural language detection requirements from operations and maintenance personnel, and parsing the natural language description into structured detection task instructions through a semantic task conversion mechanism. The structured detection task instructions include the target website to be detected, isolated resource feature descriptions, and execution constraints. Based on structured detection task instructions, a multi-source detection fusion strategy is initiated to scan the target website and identify a set of suspected isolated resources. An evolution trend prediction model is established, historical behavior data of the page is collected and state evolution characteristics are extracted, the trend of the page changing from active to decay is predicted, the prediction results are used as the basis for early warning of isolation judgment, suspected isolated resources are verified through hierarchical confirmation, and the hierarchical confirmation results are merged to output a list of isolated resources. Establish a risk quantification assessment system to score isolated resources from four assessment dimensions: technical vulnerability, exposure of sensitive information, cumulative effect over time, and business-related impact, and generate risk level reports and disposal priority recommendations.

2. The method for detecting and assessing the risks of isolated Web resources based on natural language as described in claim 1, characterized in that: The process of parsing natural language descriptions into structured detection task instructions through a semantic task conversion mechanism includes receiving natural language text input by operations and maintenance personnel, performing word segmentation and part-of-speech tagging on the text, and identifying professional terms and key entities in the operations and maintenance field. The intent recognition component determines the type of detection intent of the operation and maintenance personnel. The detection intent types include full scan request, specified area detection request, historical result query request, and risk assessment request. Key detection elements are extracted from natural language text using an entity extraction component. These key detection elements include target website identifier, URL path range, resource type constraints, and time range limitations. The identified intent type and extracted detection elements are combined to generate a structured detection task instruction. The accuracy of the task understanding is confirmed with the operation and maintenance personnel through an interactive interface. The task parameters are adjusted based on the confirmation feedback before the detection process is started.

3. The method for detecting and assessing the risks of isolated Web resources based on natural language as described in claim 2, characterized in that: The multi-source detection and fusion strategy is initiated to scan the target website, including tracing the path through access logs, reading the server's web access logs, statistically analyzing the changes in the number of accesses and the distribution of access sources for each page within a set observation period, and identifying pages whose access frequency is continuously decreasing and lacks normal referencing sources. By probing the website's link structure, starting from the homepage, we recursively traverse the website according to the link relationships, record the link path and depth information of each visited page, construct a complete website topology diagram, and mark pages that cannot be reached through normal navigation. By using the semantic association pathway, the text content features of each page on the website are extracted, and the similarity between the page content and the core theme content of the website is compared to identify pages with high thematic deviation. The results of the three detection paths are cross-validated. When a page is marked as abnormal in multiple paths, it is included in the suspected isolated resource set, and the specific abnormal characteristics given by each path are recorded.

4. The method for detecting and assessing the risks of isolated Web resources based on natural language as described in claim 3, characterized in that: The establishment of the evolution trend prediction model includes collecting behavioral data of each page in the website within a historical time period. The behavioral data includes page update operation records, user access statistics, page content change records, and external link changes. State evolution features are extracted from behavioral data, including a decreasing trend in update frequency, a rate of decline in access volume, a content freshness index, and a link failure rate. By using time-series analysis methods to learn the changing patterns of page state evolution characteristics, we can predict the likelihood of a page transitioning from an active state to a decaying state in the future. Set a state transition warning threshold. When the decay transition probability of a page exceeds the threshold, mark the page as a high-risk monitoring object in advance and give it higher attention weight in subsequent isolation judgments.

5. The method for detecting and assessing the risks of isolated Web resources based on natural language as described in claim 4, characterized in that: The layered confirmation includes structural layer confirmation and semantic layer confirmation. Structural layer confirmation verifies the connection relationship between the page and the website navigation system, while semantic layer confirmation verifies the degree of relevance between the page content and the website theme. The verification of suspected isolated resources through layered confirmation includes: performing first-level confirmation at the structural layer, checking whether the suspected isolated page has a link reference from the website's main navigation menu, checking whether the page is included in the site map file, checking whether the page can be reached from the website homepage through limited redirects, and counting the number of valid structural links. Second-level confirmation is performed at the semantic layer to analyze the relevance of the content theme of the suspected isolated page to the overall content theme of the website, check whether the page contains outdated information that is unrelated to the website's business, and assess the actual value of the page content to the website's users. A comprehensive isolation determination score is calculated based on the confirmation results of the structural layer and the semantic layer. When structural connections are missing and semantic associations are weak, the determination score is high, and the resource is identified as an isolated resource. Resources identified as orphaned are categorized and marked, including abandoned test pages, expired content pages, legacy configuration files, and broken functional modules. Different types of orphaned resources will be subject to differentiated risk assessment strategies.

6. The method for detecting and assessing the risks of isolated Web resources based on natural language as described in claim 5, characterized in that: The establishment of the risk quantification assessment system includes assessing the dimension of technical vulnerability, scanning for security vulnerabilities in isolated resources, identifying outdated technical components and known security flaws, and calculating technical risk scores based on the severity and exploitability of the vulnerabilities. Assess the exposure dimensions of sensitive information, detect whether isolated resources contain sensitive data such as database configuration information, API access keys, internal system addresses, and user credential information, and calculate the exposure risk score based on the category and quantity of sensitive information.

7. The method for detecting and assessing the risks of isolated Web resources based on natural language as described in claim 6, characterized in that: The establishment of the risk quantification assessment system also includes assessing the cumulative effect over time, and statistically analyzing the duration of isolated resources in an isolated state. The longer the duration, the greater the risk of the resource being unattended from operation and maintenance monitoring for an extended period. Assess the business-related impact dimensions, analyze the degree of correlation between isolated resources and core business systems, and evaluate the possible paths and scope of impact for attackers to penetrate the core system through the isolated resource. By combining the scores of the four assessment dimensions, the overall risk score of isolated resources is calculated through a weighted allocation mechanism. Based on the scores, resources are divided into three risk levels: high-risk, medium-risk, and low-risk. Corresponding disposal recommendations are then formulated for each risk level.

8. A natural language-driven isolated web resource detection and risk assessment system, based on the natural language-driven isolated web resource detection and risk assessment method according to any one of claims 1 to 7, characterized in that: It also includes a semantic parsing module, which receives natural language detection requirements input from operations and maintenance personnel and parses the natural language description into structured detection task instructions through a semantic task conversion mechanism; The multi-source detection module is used to initiate a multi-source detection fusion strategy to scan the target website based on the structured detection task instructions, and obtain a set of suspected isolated resources. The trend prediction module is used to establish an evolution trend prediction model, predict the evolution trend of page state, and use the prediction results as a preliminary warning basis for isolated judgment. The hierarchical verification module is used to verify suspected orphaned resources through a hierarchical verification mechanism and output a list of orphaned resources. The risk assessment module is used to establish a quantitative risk assessment system, score isolated resources in multiple dimensions, and generate risk level reports and disposal priority recommendations. The monitoring and early warning module is used to establish a regular scanning mechanism and a dynamic early warning mechanism to achieve continuous monitoring and risk tracking.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the natural language-driven isolated web resource detection and risk assessment method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the natural language-driven isolated web resource detection and risk assessment method according to any one of claims 1 to 7.

Images