Data processing method, device and equipment

By structurally decomposing and segmenting structured text data, and combining semantic saliency information and boundary sparsity factors, abnormal word segmentation is identified and a whitelist template is generated. This solves the accuracy and robustness problems of content recognition and risk detection in existing technologies, and achieves efficient content recognition and risk detection.

CN121118042APending Publication Date: 2025-12-12ALIPAY (HANGZHOU) INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202511171567.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-08-20
Publication Date
2025-12-12

AI Technical Summary

Technical Problem

Existing technologies lack a robust, interpretable, and highly accurate automated content recognition and risk detection mechanism in areas such as API security, web attack defense, intelligent log analysis, and content recognition. They cannot effectively identify abnormal content in structured or semi-structured paths, and existing methods suffer from poor generalization ability, high maintenance costs, high false positive rates, and insufficient semantic support.

Method used

By acquiring structured text data, performing structured decomposition processing, determining the contextual neighborhood of each word segment, and identifying abnormal word segmentation based on semantic saliency information, local contrast loss information, and boundary sparsity factor, a whitelist template is generated, achieving interpretable and highly accurate content recognition and risk detection.

Benefits of technology

It improves the accuracy and robustness of content recognition and risk detection, enhances the semantic generalization ability of business systems, has the ability to generalize across different types of structural content, reduces maintenance costs, and improves multimodal adaptability and evolutionary maintenance efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121118042A_ABST
    Figure CN121118042A_ABST
Patent Text Reader

Abstract

The embodiment of the invention discloses a data processing method, device and equipment, and the method comprises the steps: obtaining a plurality of different structural text data which can be decomposed into a plurality of different levels or substructures with position dependence; performing structural disassembly processing on each piece of structural text data to obtain a structural node corresponding to each piece of structural text data, and performing word segmentation processing on each structural node; for each segmented word, determining each segmented word and a corresponding context neighborhood, and further respectively determining semantic significance information between each segmented word and the corresponding context neighborhood, local contrast loss information corresponding to each segmented word and a boundary sparsification factor corresponding to each segmented word; further determining semantic anomaly significance information corresponding to each segmented word; and determining generalizable white list information and / or abnormal structural text data from a plurality of different structural text data based on the determined semantic anomaly significance information.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This document relates to the field of computer technology, and in particular to a data processing method, apparatus, and device. Background Technology

[0002] In multiple technical fields such as API security, web attack defense, intelligent log analysis, content recognition, and privacy leakage detection and protection, there are a large number of structured or semi-structured path and parameter types of content (such as URLs, path parameters, interface definitions, configuration fields, etc.). These structured paths often contain dynamic content, user input, or potential outliers. Therefore, there is a need to provide a robust, interpretable, highly accurate, and automated content recognition and risk detection mechanism to help the system identify abnormal paths, generate whitelist templates, or use them for further modeling and filtering. Summary of the Invention

[0003] The purpose of the embodiments in this specification is to provide a robust, interpretable, highly accurate, and automated content recognition and risk detection mechanism to help the system identify abnormal paths, generate whitelist templates, or use them for further modeling and filtering.

[0004] To achieve the above technical solution, the embodiments in this specification are implemented as follows: This specification provides a data processing method, comprising: acquiring multiple different structured text data, wherein the structured text data is text data that can be decomposed into multiple different hierarchical or substructures with positional dependencies; performing structural decomposition processing on each structured text data to obtain a structure node corresponding to each structured text data, and performing word segmentation processing on each structure node to obtain the words contained in each structure node; for each word segment, determining the context neighborhood corresponding to each word segment and the corresponding word segment, and based on each word segment and the corresponding word segment's context neighborhood, determining the context corresponding to each word segment and the corresponding word segment. The system includes: semantic saliency information between neighborhoods, local contrast loss information corresponding to each word, and boundary sparsity factor corresponding to each word; based on the semantic saliency information between each word and its corresponding context neighborhood, the local contrast loss information corresponding to each word, and the boundary sparsity factor corresponding to each word, the system determines semantic anomalous saliency information corresponding to each word, wherein the boundary sparsity factor is used to characterize the semantic consistency degree of the context neighborhood corresponding to each word; based on the semantic anomalous saliency information corresponding to each word, the system determines corresponding generalizable whitelist information and / or anomalous structured text data from the multiple different structured text data.

[0005] This specification provides a data processing apparatus, comprising: a data acquisition module for acquiring multiple different structured text data, wherein the structured text data is text data that can be decomposed into multiple different hierarchical or substructured structures with positional dependencies; a structure navigation encoding module for structurally decomposing each structured text data to obtain a structure node corresponding to each structured text data, and performing word segmentation on each structure node to obtain the words contained in each structure node; and a semantic saliency determination module for determining the context neighborhood of each word and the corresponding word for each word, and determining the semantic saliency of each word in relation to the context neighborhood of the corresponding word. The system includes: semantic saliency information between the context neighborhoods corresponding to each word segment, local contrast loss information corresponding to each word segment, and boundary sparsity factor corresponding to each word segment; based on the semantic saliency information between each word segment and its corresponding context neighborhood, the local contrast loss information corresponding to each word segment, and the boundary sparsity factor corresponding to each word segment, the system determines the semantic anomalous saliency information corresponding to each word segment, wherein the boundary sparsity factor is used to characterize the semantic consistency degree of the context neighborhood corresponding to each word segment; and an identification module, based on the semantic anomalous saliency information corresponding to each word segment, determines the corresponding generalizable whitelist information and / or anomalous structured text data from the multiple different structured text data.

[0006] This specification provides an embodiment of a data processing device, comprising: a processor; and a memory arranged to store computer-executable instructions, wherein the executable instructions, when executed, cause the processor to: acquire multiple different structured text data, wherein the structured text data is text data capable of being decomposed into multiple different hierarchical or substructured text data with positional dependencies; perform structural decomposition processing on each structured text data to obtain a structure node corresponding to each structured text data, and perform word segmentation processing on each structure node to obtain the words contained in each structure node; for each word segment, determine the context neighborhood corresponding to each word segment and the corresponding context neighborhood, and based on each word segment and the corresponding context neighborhood, determine the context neighborhood corresponding to each word segment and the context neighborhood corresponding to each word segment. The following steps determine the semantic saliency information between each word segment and its corresponding context neighborhood, the local contrast loss information for each word segment, and the boundary sparsity factor for each word segment. Based on the semantic saliency information between each word segment and its corresponding context neighborhood, the local contrast loss information for each word segment, and the boundary sparsity factor for each word segment, semantic anomalous saliency information for each word segment is determined. The boundary sparsity factor is used to characterize the semantic consistency degree of the context neighborhood for each word segment. Based on the semantic anomalous saliency information for each word segment, corresponding generalizable whitelist information and / or anomalous structured text data are determined from the multiple different structured text data.

[0007] This specification also provides a storage medium for storing computer-executable instructions. When executed by a processor, these instructions implement the following process: acquiring multiple different structured text data, wherein the structured text data is text data that can be decomposed into multiple different hierarchical or substructured structures with positional dependencies; performing structural decomposition processing on each structured text data to obtain a corresponding structure node, and performing word segmentation processing on each structure node to obtain the words contained in each structure node; for each word segment, determining the context neighborhood corresponding to each word segment, and based on each word segment and its corresponding context neighborhood, determining... The semantic saliency information between each word and its corresponding context neighborhood, the local contrast loss information for each word, and the boundary sparsity factor for each word are analyzed. Based on the semantic saliency information between each word and its corresponding context neighborhood, the local contrast loss information for each word, and the boundary sparsity factor for each word, semantic anomalous saliency information for each word is determined. The boundary sparsity factor is used to characterize the semantic consistency degree of the context neighborhood for each word. Based on the semantic anomalous saliency information for each word, corresponding generalizable whitelist information and / or anomalous structured text data are determined from the multiple different structured text data.

[0008] This specification also provides a computer program product, including a computer program that, when executed by a processor, performs the following process: acquiring multiple different structured text data, wherein the structured text data is text data that can be decomposed into multiple different hierarchical or substructured structures with positional dependencies; performing structural decomposition processing on each structured text data to obtain the structure nodes corresponding to each structured text data, and performing word segmentation processing on each structure node to obtain the words contained in each structure node; for each word segment, determining the context neighborhood corresponding to each word segment, and based on each word segment and the context neighborhood corresponding to the corresponding word segment, determining the context neighborhood of each word segment and the context neighborhood of the corresponding word segment. The system includes: semantic saliency information between the context neighborhoods corresponding to each word segment, local contrast loss information corresponding to each word segment, and boundary sparsity factor corresponding to each word segment; based on the semantic saliency information between each word segment and its corresponding context neighborhood, the local contrast loss information corresponding to each word segment, and the boundary sparsity factor corresponding to each word segment, the system determines semantic anomalous saliency information corresponding to each word segment, wherein the boundary sparsity factor is used to characterize the semantic consistency degree of the context neighborhood corresponding to each word segment; based on the semantic anomalous saliency information corresponding to each word segment, the system determines corresponding generalizable whitelist information and / or anomalous structured text data from the multiple different structured text data. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Figure 1 This is a schematic diagram of the structure of a data processing system described in this specification; Figure 2 This is a schematic diagram of a data processing procedure described in this specification; Figure 3 This is a schematic diagram of the structure of an NLASR-NLP model in this specification; Figure 4 This is a schematic diagram of the structure of an NLASR-NLP model combined with the DED-A module as described in this specification; Figure 5 This is a schematic diagram of the structure of an NLASR-NLP model that combines the DED-A module and the saliency-driven positive example voting fusion module as described in this specification; Figure 6 This is a schematic diagram illustrating a URL-based data processing procedure as described in this specification. Figure 7 This is a schematic diagram of a data processing device according to the present specification; Figure 8 This is a schematic diagram of a data processing device described in this specification. Detailed Implementation

[0010] This specification provides a data processing method, apparatus, and device through its embodiments.

[0011] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0012] This specification provides a content recognition mechanism based on the fusion of structural paths and context. Typically, content recognition can be achieved in many ways, such as through regular expression matching. This method relies on predefined URL templates or path whitelists, using regular expressions to statically match structural paths. Representative techniques include URL path template extraction and maintenance, and static whitelist comparison and blocking strategies. However, this method has poor generalization ability; regular expressions cannot effectively cover long-tail paths and dynamically changing parameter structures, and are unaware of new variants. Furthermore, this method has high maintenance costs, requiring continuous manual updates to the rule set, which can easily lead to lag, especially in systems with frequent business iterations. This method also cannot handle semantic relationships; regular expressions are essentially character-level operations and cannot understand the semantic dependencies and contextual logic between path fields.

[0013] In addition, it can also be implemented based on word segmentation entropy analysis. Specifically, it can detect structural or content anomalies by calculating the Shannon entropy or character frequency information of each field in the path, and is often used for data cleaning and intrusion detection. However, this method has a high false positive rate, meaning it is prone to false positives when dealing with high-information-entropy but reasonable fields such as URL encoding, hash strings, or natural language phrases. Moreover, this method lacks structure awareness, meaning it cannot distinguish between important and non-core fields in the same path hierarchy, leading to noise affecting the model's judgment. This method also has weak robustness, meaning it is not robust to malicious interference (such as inserting special characters or using spaces to bypass).

[0014] Alternatively, location-based semantic modeling can be used. Some solutions attempt to incorporate the positional preferences of word segmentation within the URL, such as prioritizing fields at the beginning of the path or key locations, using a "location scoring strategy" to improve recognition capabilities. However, this approach lacks sufficient semantic support; the location information itself is only statistically significant, lacking contextual semantic understanding and failing to identify logically "key fields." Furthermore, this approach has poor scenario adaptability; for systems with significantly different path structures or dynamically generated path mechanisms, the location features are unstable. Finally, this approach lacks interpretability, failing to clearly explain why a particular location is judged as "abnormal" or "critical," thus affecting credibility.

[0015] Another approach is to use clustering scoring based on semantic embedding. In this method, semantic embedding models (such as Word2Vec and BERT) are used to vectorize the path, and then the input samples are scored through clustering, distance calculation, and other methods. However, this approach is highly dependent on static training, meaning the model is very sensitive to the distribution of training data and is difficult to adapt to business evolution after deployment. Moreover, this approach lacks fine-grained control mechanisms, meaning it usually models the entire path and cannot accurately depict the structural relationships and importance weights between fields. Furthermore, this approach has poor interpretability and tuning capabilities, meaning that changes in the distribution in the vector space are difficult to interpret intuitively, which is not conducive to business adjustments.

[0016] Based on the above methods, there are several drawbacks: First, reliance on static rules leads to high maintenance costs and weak generalization ability. Second, excessive model training results in high deployment costs and poor tunability. Third, structural and semantic information are often modeled separately, leading to both false positives and false negatives. Fourth, a lack of dynamic adaptation mechanisms prevents effective support for business iterations and the evolution of abnormal patterns. Fifth, the use of single-dimensional features, such as entropy values, location, or keyword dictionaries, easily leads to false positives or false negatives. Therefore, a multi-dimensional fusion mechanism integrating structural and semantic features is urgently needed to achieve interpretable, highly accurate, and automatically adaptable path-level content recognition and anomaly detection. This specification provides a robust, interpretable, highly accurate, and automated content recognition and risk detection mechanism to help the system identify abnormal paths, generate whitelist templates, or provide technical solutions for further modeling and filtering. Specific processing details can be found in the following embodiments.

[0017] The data processing methods provided in one or more embodiments of this specification are applicable to the implementation environment of data processing. Figure 1 The implementation environment includes at least: Client 100 and server 200. Furthermore, server 200 can include various models, strategies, algorithms, and rules, among which: Client 100 can run on terminal devices, which can be mobile phones, personal computers, tablets, e-book readers, wearable devices, devices that interact with information based on AR (Augmented Reality) and VR (Virtual Reality), and laptop computers, etc. Client 100 can be installed on terminal devices. Client 100 can be an application, a browser, or a subroutine embedded in an application, etc.

[0018] Server 200 can run on a server, which can be one or more servers, a server cluster consisting of several servers, or a cloud server on a cloud computing platform. Server 200 can be installed on the server. Server 200 can be an application or a subroutine embedded in an application. Various models, strategies, algorithms, and rules can be integrated into server 200. Alternatively, server 200 can call any one or more of various models, strategies, algorithms, and rules to perform corresponding operations.

[0019] In addition, it may include a database 300, which may be set in the server on which the server 200 runs or outside the server on which the server 200 runs. The database 300 may store generalizable whitelist information (such as whitelist templates) and abnormal structured text data and other related information.

[0020] In this implementation environment, server 200 can obtain multiple different structured text data from users (users via client 100). Structured text data is text data that can be decomposed into multiple different hierarchical or substructured data with positional dependencies. Server 200 can perform structural decomposition processing on each structured text data to obtain the corresponding structure nodes, and perform word segmentation processing on each structure node to obtain the words contained in each structure node. For each word, the server determines the contextual neighborhood of each word and its corresponding contextual neighborhood, and based on each word and its corresponding contextual neighborhood, determines the semantic saliency information between each word and its corresponding contextual neighborhood, and the local contrast loss for each word. Based on the semantic saliency information between each word and its corresponding context neighborhood, the local contrast loss information of each word, and the boundary sparsity factor of each word, the semantic anomaly saliency information corresponding to each word is determined. The boundary sparsity factor is used to characterize the semantic consistency degree of the context neighborhood corresponding to each word. Finally, based on the semantic anomaly saliency information corresponding to each word, the corresponding generalizable whitelist information and / or abnormal structured text data can be determined from multiple different structured text data. This enables an interpretable, high-accuracy, and automated content recognition and risk detection mechanism, helping the system identify abnormal paths, generate whitelist templates, or be used for further modeling and filtering.

[0021] like Figure 2As shown in the embodiments of this specification, a data processing method is provided. The execution subject of this method can be a terminal device or a server, etc. The terminal device can be a mobile terminal device such as a mobile phone or tablet computer, a computer device such as a laptop or desktop computer, or an IoT device (specifically, a smartwatch, in-vehicle device, etc.). The server can be a single server or a server cluster composed of multiple servers. The server can be a backend server for financial business or online shopping business, or a backend server for an application, etc. This embodiment uses a server as the execution subject for detailed description. For the case where the execution subject is a terminal device, please refer to the following server case processing, which will not be repeated here. The method may specifically include the following steps: In step S202, multiple different structured text data are obtained. Structured text data is text data that can be decomposed into multiple different levels or substructures with positional dependencies.

[0022] Structured text data can contain various levels or substructures that have positional dependencies. Furthermore, structured text data can also be text data with explicit hierarchical relationships, in which case positional dependencies between different levels are implicitly implied. Structured text data can include various types, such as URLs or file download addresses. A URL can consist of a string of characters, including letters, numbers, and special symbols. The general syntax of a URL is protocol: / / hostname[:port] / path / [:parameters][?query]#fragment, where protocol specifies the transport protocol used, such as FTP, HTTP, HTTPS, MMS, ED2K, etc. Hostname is the hostname, referring to the Domain Name System (DNS) hostname or IP address of the server hosting the resources. Sometimes, the username and password required to connect to the server may be included before the hostname. `Port` represents the port number, usually an integer. It's optional in the URL; omitting it indicates the use of the default port. Various transport protocols have default port numbers; for example, the default port for HTTP is 80. If omitted, the default port number is used. Sometimes, for security or other reasons, the port can be redefined on the server, using a non-standard port number. In this case, the port number cannot be omitted from the URL. `Path` represents the path, a string separated by zero or more " / " symbols, generally used to represent a directory or file address on the host. `Parameters` represents parameters, optional to specify special parameters, which are interpreted by the server-side program. `Query` represents a query, optional in the URL, used to pass parameters to dynamic web pages (such as those created using CGI, ISAPI, PHP / JSP / ASP / ASP.NET, etc.). Multiple parameters are allowed, separated by "&", with each parameter's name and value separated by "=". Fragments represent fragments of information, typically strings, used to specify segments within online resources. For example, if a webpage contains multiple definitions, a fragment can be used to directly locate a specific definition.A specific URL could be like http: / / www.example.com:80 / path / to / myfile.html?key1=value1&key2=value2#SomewhereInTheDoument, where http is the protocol, www.example.com is the domain name (hostname), 80 is the port number, / path / to / myfile.html is the path to the resource, ?key1=value1&key2=value2 are the query parameters, and #SomewhereInTheDoument is the information fragment.

[0023] In implementation, multiple different structured text data can be obtained in various ways. For example, a resource request interception mechanism can be set at the input port of a specified resource server. The intercepted resource requests, their corresponding URLs or resource request addresses, can then be obtained as multiple different structured text data. Another example is aspects, especially security parallel aspects. Security parallel aspects can be used for security detection and threat protection during application runtime. Thus, on a platform, when applications and mini-programs based on those applications are running, the security parallel aspects can perform real-time perception, analysis, and blocking of the application's or mini-program's call chains, parameters, and request behaviors. The structured text data with multiple different levels or substructures with positional dependencies, corresponding to the call chains, parameters, and request behaviors collected by the security parallel aspects, can be used as the obtained structured text data. In addition, it can also be obtained from one or more different fields such as API security, web attack defense, log intelligent analysis, and content recognition, where structured text data with multiple different levels or substructures with positional dependencies are obtained. Specifically, the corresponding structured text data can be obtained according to the method in the above example, and this specification does not limit this embodiment.

[0024] In step S204, each structured text data is structurally decomposed to obtain the structure node corresponding to each structured text data, and each structure node is segmented to obtain the segmented words contained in each structure node.

[0025] In this context, a structure node can be one of several different levels or substructures with positional dependencies. For example, if the structured text data is a URL, the structure node can be a domain name or hostname in the URL (such as www.example.com in the example above), a path to a resource in the URL (such as / path / to / myfile.html in the example above), a query parameter in the URL (such as ?key1=value 1&key2=value2 in the example above), or a fragment of information in the URL (such as #SomewhereInTheDoument in the example above), etc. The specific settings can be determined according to the actual situation.

[0026] In implementation, to facilitate subsequent processing, each structured text data can be structurally decomposed into a hierarchical or tree-like structure. Each decomposed level or substructure can serve as a structural node. Through these decomposed hierarchical or tree-like structures, semantic learning can be guided to construct the context of each structural node according to the path indicated by the hierarchical or tree-like structure. During the decomposition process, a specified structural node in the structured text data can be used as the root node. Combined with the sequential or dependency relationships between different hierarchical or tree-like structures, the corresponding hierarchical structure or tree-like structure for each structured text data can be constructed. The specified structural node can be a single decomposed level or substructure, or it can be the level or substructure that is first in the sequential relationship among multiple decomposed levels or substructures, or it can be the level or substructure that is most dependent on among multiple decomposed levels or substructures in the dependency relationship, etc., depending on the specific circumstances. For example, based on the above example, structured text data can be a URL, such as http: / / www.example.com:80 / path / to / myfile.html?key1=value1&key2=value2#SomewhereInTheDoument. By structurally decomposing the structured text data of the above URL, the following hierarchical or tree structure can be obtained.

[0027] Root node: www.example.com └─80 └─ / path / to / myfile.html └─?key1=value1&key2=value2 └─ #SomewhereInTheDoument Here, www.example.com, 80, / path / to / myfile.html, and #SomewhereInTheDoument can each be treated as a structure node. Furthermore, a context extractor can be set for each structure node to extract its context.

[0028] In practical applications, each structural node can also include attribute-related data such as node identifier and node data. The node identifier can be determined based on the name or representation of the corresponding hierarchy or substructure. For example, the node identifier of the structural node corresponding to www.example.com can be a domain name or hostname, and the corresponding node data can be www.example.com. As another example, the node identifier of the structural node corresponding to / path / to / myfile.html can be a path (or the path of a resource), and the corresponding node data can be / path / to / myfile.html.

[0029] The data for each structural node can be input into a specified word segmentation tool. This tool can then perform word segmentation on the content within each structural node. For example, the node data " / path / to / myfile.html" corresponding to the path of a resource can be input into the specified word segmentation tool. This tool can then perform word segmentation on the content " / path / to / myfile.html" within that structural node, obtaining the words contained in that structural node. In this way, the words contained in each structural node can be obtained. The word segmentation tool can include various tools, such as NLTK, SpaCy, and StanfordCoreNLP. Alternatively, it can be built using machine learning networks, such as convolutional neural networks or BERT, depending on the specific requirements. Furthermore, a context extractor can be set for each word to extract its context.

[0030] In step S206, for each word segment, the context neighborhood corresponding to each word and the corresponding word is determined. Based on each word and the corresponding word's context neighborhood, the semantic saliency information between each word and the corresponding word's context neighborhood, the local contrast loss information corresponding to each word, and the boundary sparsity factor corresponding to each word are determined respectively. Based on the semantic saliency information between each word and the corresponding word's context neighborhood, the local contrast loss information corresponding to each word, and the boundary sparsity factor corresponding to each word, the semantic anomaly saliency information corresponding to each word is determined. The boundary sparsity factor is used to characterize the semantic consistency degree of the context neighborhood corresponding to each word.

[0031] Semantic saliency information can be used to characterize the degree of significant discrepancy or directional shift between the semantic representation of a specified word and its contextual neighborhood, thereby determining whether the word might be a semantic anomaly. Local contrastive loss information can be the loss information corresponding to a loss function constructed by contrastive learning to narrow the distance between the specified word and its semantically matching neighboring words, and amplify the distance (or difference) between the specified word and the anomalous word. Semantic anomaly saliency information can be used to characterize the degree of anomalous saliency of a specified word; the higher the semantic anomaly saliency of a specified word, the higher the probability that the specified word is an anomalous word.

[0032] In implementation, for any given word segment, one or more adjacent or similar words can be obtained from the structural node where the word segment is located and other structural nodes adjacent to that structural node. For example, for the current word i, k words (k can be a positive integer) before word i and k words after word i can be obtained from the structural node where word i is located and other structural nodes adjacent to that structural node, i.e., word ik, ... word i-1, word i+1, ... word i+k. Word ik, ... word i-1, word i+1, ... word i+k can be used as the context neighborhood corresponding to word i. In this way, the context neighborhood corresponding to each word can be determined.

[0033] The encoder can be pre-defined. For example, it can be built using a neural network model, BERT, or the Transformer module. The specific settings can be determined according to the actual situation. Each word segment obtained above can be used as a token. Each token can be input into the encoder to obtain the encoding information corresponding to each token (which can be presented in the form of an embedding vector).

[0034] A semantic saliency algorithm can be pre-defined according to the actual situation. This algorithm calculates the semantic saliency information between each word and its corresponding context neighborhood based on the encoding information corresponding to each word token and the encoding information corresponding to the word tokens contained in the context neighborhood of the corresponding word. Alternatively, a specified model (such as a neural network model or a large language model) can be pre-trained. Each word and its corresponding context neighborhood can be input into the specified model, or the encoding information corresponding to each word token and the encoding information corresponding to the word tokens contained in the context neighborhood of the corresponding word can be input into the specified model. The specified model processes each word and its corresponding context neighborhood to obtain the semantic saliency information between each word and its corresponding context neighborhood. The specific settings can be configured according to the actual situation.

[0035] To enhance the saliency of low-density semantically anomalous word segmentation, a local contrastive loss can be introduced. This reduces the distance between the current word segment token and its semantically matching neighboring word segment tokens, while increasing the distance (or difference) between the current word segment token and the anomalous word segment token. Specifically, a local contrastive loss function can be pre-defined. This function is determined by the distance between the current word segment token and its semantically matching neighboring word segment tokens, and the distance between the current word segment token and the anomalous word segment token. In other words, the local contrastive loss function minimizes the distance between the current word segment token and its semantically matching neighboring word segment tokens, and maximizes the distance between the current word segment token and the anomalous word segment token. The distance between the encoded information corresponding to the current word segment token and the encoded information corresponding to the semantically matching neighboring word segment tokens can be calculated using a pre-defined similarity algorithm (such as cosine similarity or Euclidean distance). The distance between the encoded information corresponding to the current word segment token and the encoded information corresponding to the anomalous word segment token can also be calculated using a pre-defined similarity algorithm. Then, the calculated distances can be substituted into the aforementioned local contrastive loss function to obtain the local contrastive loss information corresponding to the current word segment token. The above method can be used to obtain the local contrast loss information corresponding to each word segmentation.

[0036] To avoid the "dilution" of local abnormal word segments by dense context, a boundary sparsity factor can be introduced to characterize the semantic consistency of the context neighborhood. The higher the consistency, the more reliable the saliency of the current word segment token. The boundary sparsity factor can be set according to the actual situation. Specifically, it can be set directly based on expert experience for different word segments, or a corresponding algorithm can be pre-defined based on each word and its corresponding context neighborhood. This algorithm can calculate the boundary sparsity factor for each word based on each word and its corresponding context neighborhood. Alternatively, a specified model (such as a neural network model, a large language model, etc.) can be pre-built and trained using a certain amount of sample data to obtain the trained model. Each word and its corresponding context neighborhood can be input into the trained model, or the encoding information corresponding to each word token and the encoding information of the word tokens contained in the corresponding context neighborhood can be input into the trained model. The trained model processes each word and its corresponding context neighborhood to obtain the boundary sparsity factor for each word. The specific setting can be determined according to the actual situation.

[0037] The semantic saliency information between each word segment and its corresponding context neighborhood, the local contrast loss information corresponding to each word segment, and the boundary sparsity factor corresponding to each word segment can be fused. The specific fusion method can be set according to the actual situation. For example, the corresponding fusion algorithm can be preset, and the corresponding expression can be as follows:

[0038] in, Indicates the first i Semantic anomaly saliency information corresponding to each word segmentation and Indicates the fusion parameters, Indicates the first i Semantic saliency information between each word and its corresponding context neighborhood. Indicates the first i Each word segmentation corresponds to a boundary sparsity factor. Indicates the first i Local contrast loss information corresponding to each word segmentation.

[0039] The semantic saliency information between each word and its corresponding context neighborhood, the local contrast loss information corresponding to each word, and the boundary sparsification factor corresponding to each word can be substituted into the above expression. The result can be used as the semantic abnormal saliency information corresponding to each word.

[0040] In step S208, based on the semantic anomaly saliency information corresponding to each word segmentation, the corresponding generalizable whitelist information and / or abnormal structured text data are determined from multiple different structured text data.

[0041] Among them, generalizable whitelist information can be related to generalizable whitelists. In practical applications, generalizable whitelist information can include whitelist templates.

[0042] In implementation, since semantic anomaly saliency information is used to characterize the degree of anomaly saliency of a specified word segment, if the semantic anomaly saliency information corresponding to a certain word indicates a high degree of semantic anomaly saliency, then that word segment is more likely to be an anomalous word segment. After obtaining the semantic anomaly saliency information corresponding to each word through the above method, the probability of each word segmenting as an anomalous word segment can be determined based on the semantic anomaly saliency information. Based on the probability of each word segmenting as an anomalous word segment, the probability of the structural node containing each word segmenting as an anomalous structural node can be determined, thereby determining the probability of the corresponding structured text data being anomalous structured text data. The above method allows us to identify anomaly-containing and anomaly-free structured text data within multiple different structured text datasets. We can then statistically summarize the structural nodes contained in each anomaly-free structured text data set, ultimately obtaining one or more different generalizable structured text datasets. For example, if multiple anomaly-free structured text datasets contain the same structural node, we can retain that node and replace different structural nodes with specified characters. The structured text dataset obtained after this processing is considered a generalizable structured text dataset. Based on one or more different generalizable structured text datasets, we can determine generalizable whitelist information.

[0043] This specification provides a data processing method. It involves acquiring multiple different structured text data that can be decomposed into multiple different hierarchical or substructured structures with positional dependencies. Each structured text data is then structurally decomposed to obtain a corresponding structure node. Each structure node is then segmented to obtain the words contained within it. For each word, the contextual neighborhood of each word and its corresponding contextual neighborhood is determined. Based on each word and its corresponding contextual neighborhood, semantic saliency information between each word and its corresponding contextual neighborhood, local contrast loss information for each word, and boundary sparsity factor for each word are determined. Furthermore, based on the semantic saliency information between each word and its corresponding contextual neighborhood, and the local contrast loss information for each word, the method can further determine the semantic saliency information between each word and its corresponding contextual neighborhood, and the local contrast loss information for each word. By comparing the loss information and the boundary sparsity factor corresponding to each word, the semantic anomaly saliency information corresponding to each word is determined. Finally, based on the semantic anomaly saliency information corresponding to each word, corresponding generalizable whitelist information and / or anomalous structured text data can be determined from multiple different structured text data. Thus, by introducing semantic anomaly saliency to verify the semantic consistency of structural node emptiness, semantic recognition capability is improved. Furthermore, the semantic anomaly saliency is evaluated through the semantic saliency information between each word and its corresponding context neighborhood, the local contrastive loss information corresponding to each word, and the boundary sparsity factor corresponding to each word, thereby enhancing the semantic generalization capability of the business system. This makes the content recognition and risk detection mechanism in the business system more robust and interpretable, and improves the accuracy of content recognition and risk detection. In addition, this method has the ability to generalize across different types of structural content, adapting to different structural types of data such as WebAPI, log paths, and file resources without reconstructing the feature extraction logic. Moreover, it can form a highly modular anomaly recognition system, thus significantly improving the multimodal adaptability and evolutionary maintenance efficiency of the business system.

[0044] In practical applications, multiple different structural nodes in structured text data have positional dependencies, and the relative positions of word segments in structural nodes are all crucial. Based on this, positional weights can be introduced. For details, please refer to the following content: Determine the positional weights corresponding to the word segments contained in each structural node.

[0045] In implementation, such as Figure 3As shown, to determine the semantic anomaly saliency information corresponding to each word segment, a context-based semantic saliency recognition network (i.e., Navigational Layer-AwareSemantic Relevance for NLP, NLASR-NLP) model can be set up. Based on the functions of the NLASR-NLP model, it can be divided into a semantic saliency recognition network, a structure-aware encoder, a positional-attentive embedding module, a semantic contrast boundary controller, and a multi-granularity fusion layer. The structure-aware encoder can structurally decompose each structured text data, obtaining the corresponding structure nodes, and guide semantic learning to model the context according to the structural path order of the structured text data (see the aforementioned content for details). The multi-granularity fusion layer can combine the semantic depth of paths in different structured text data to achieve cross-layer semantic unification.

[0046] For each word segment, the contextual neighborhood corresponding to each word and its corresponding contextual neighborhood is determined. Based on each word and its corresponding contextual neighborhood, the semantic saliency information between each word and its corresponding contextual neighborhood, the local contrastive loss information corresponding to each word, and the boundary sparsity factor corresponding to each word are determined. Furthermore, based on the semantic saliency information between each word and its corresponding contextual neighborhood, the local contrastive loss information corresponding to each word, and the boundary sparsity factor corresponding to each word, the semantic anomaly saliency information corresponding to each word is determined. For structural nodes (i.e., a level or substructure in structured text data), positional weights can be introduced into the word tokens in the structural node through a position-aware embedding module, thereby improving the positional awareness capability of the NLASR-NLP model. The positional weights corresponding to each word can be determined according to a specified algorithm or based on expert experience; the specific setting can be determined according to the actual situation.

[0047] Based on the aforementioned position weights, the specific processing methods for determining the semantic saliency information between each word and its corresponding context neighborhood, the local contrast loss information for each word, and the boundary sparsity factor for each word in step S206 can vary. Here is another optional processing method, which may include the following: Based on each word and its corresponding context neighborhood, and the position weights of the words contained in each structural node, determine the semantic saliency information between each word and its corresponding context neighborhood, the local contrast loss information for each word, and the boundary sparsity factor for each word.

[0048] In implementation, the specific processing steps for determining the semantic saliency information between each word and its corresponding context neighborhood, the local contrast loss information for each word, and the boundary sparsity factor for each word can be found in the aforementioned related content. In addition, during the above processing, the position weights corresponding to each word contained in each structural node can be combined simultaneously. The position weights corresponding to each word can be superimposed on the encoding information corresponding to each word to obtain new encoding information for each word. Based on the new encoding information corresponding to each word and its corresponding context neighborhood, the semantic saliency information between each word and its corresponding context neighborhood, the local contrast loss information for each word, and the boundary sparsity factor for each word can be determined. The specific processing steps can be found in the aforementioned related content and will not be repeated here.

[0049] In practical applications, the specific processing methods for determining the semantic saliency information between each word and its corresponding context neighborhood based on each word segment and the position weight of each word segment contained in each structural node can be varied. The following provides another optional processing method, which may include the processing of steps A2 and A4.

[0050] In step A2, based on each word segment and its corresponding context neighborhood, as well as the position weights of the words contained in each structural node, the average similarity between each word segment and its corresponding context neighborhood is determined.

[0051] In implementation, semantic saliency information between each word segment and its corresponding context neighborhood can be determined using semantic density. The semantic density of a word segment can be characterized by the average similarity between the word segment and its corresponding context neighborhood. Specifically, for example... Figure 3 As shown, for a certain token (denoted as...) Based on the semantic saliency recognition network, the word segmentation token can be encoded using the encoder described above, combined with the positional weights corresponding to the word segmentation token, to obtain the corresponding encoded information. The encoding information of the word segmentation in the context neighborhood corresponding to this word segmentation token is: ,in k It is a positive integer. The encoded information corresponding to this token can be calculated. Encoding information of the word segments in the context neighborhood corresponding to the word segmentation token The similarity between the tokens (which can be calculated using similarity algorithms such as cosine similarity and Euclidean distance) can be averaged to obtain the average similarity between the token and its corresponding context neighborhood. In this way, the average similarity between each token and its corresponding context neighborhood can be obtained.

[0052] In step A4, based on the average similarity between each word and its corresponding context neighborhood, the semantic saliency information between each word and its corresponding context neighborhood is determined.

[0053] In implementation, the semantic saliency information between each word segment and its corresponding context neighborhood can be specifically represented by the neighborhood density difference. Specifically, the similarity between different neighboring words within the context neighborhood of any given word can be calculated using a semantic saliency recognition network. The average similarity between different neighboring words within the context neighborhood of any given word can be used as the neighborhood semantic density of that word. The neighborhood semantic density of each word can be obtained in this way. The average similarity between each word and its corresponding context neighborhood, along with the neighborhood semantic density of each word, can be substituted into the following formula.

[0054] Ultimately, we can obtain the semantic saliency information between each word segment and its corresponding context neighborhood. Among these, Indicates the first i Neighborhood semantic density corresponding to each word segmentation Indicates the first i The average similarity between each word segment and its corresponding context neighborhood (or the first word segment) i (Semantic density corresponding to each word segmentation).

[0055] In practical applications, the specific processing method of step A2 above can be varied. The following provides another optional processing method, which may include the processing of steps A22 and A24.

[0056] In step A22, based on each word segment and its corresponding context neighborhood, as well as the position weights of the words contained in each structural node, the attention information corresponding to each word is determined through an attention algorithm.

[0057] The attention algorithm can include various types. The attention algorithm in this embodiment can be an additive attention algorithm. In addition, in practical applications, other types of attention algorithms can also be used, such as multi-head attention algorithms, etc. The specific algorithm can be set according to the actual situation.

[0058] In step A24, based on each word segment and its corresponding context neighborhood, and combining the positional weights of each word segment contained in each structural node with the attention information of each word segment, the average similarity between each word segment and its corresponding context neighborhood is determined.

[0059] Based on the location-aware embedding module, the processing of steps A22 and A24 above can be realized, and attention mechanisms (such as additive attention mechanisms) can be used to focus on local semantic mutation regions.

[0060] In practical applications, the specific processing methods for determining the local contrast loss information corresponding to each word based on each word segment and its corresponding context neighborhood, as well as the position weights of the words contained in each structural node, can be varied. The following provides another optional processing method, which may include the processing steps B2 and B4.

[0061] In step B2, based on each word and its corresponding context neighborhood, as well as the position weight of each word contained in each structural node, the similarity between each word and the semantically matched first neighboring word in the context neighborhood of its corresponding word, and the similarity between each word and the negative example neighboring word in the context neighborhood of its corresponding word are determined.

[0062] In step B4, based on the similarity between each word and the first neighboring word in the context neighborhood corresponding to the corresponding word, and the similarity between each word and the negative example neighboring word in the context neighborhood corresponding to the corresponding word, the local contrast loss information corresponding to each word is determined.

[0063] In implementation, such as Figure 3 As shown, based on the saliency boundary controller, the similarity between each word and the semantically matching first neighboring word in the context neighborhood of the corresponding word, and the similarity between each word and the negative example neighboring word in the context neighborhood of the corresponding word, can be substituted into the following formula.

[0064] The local contrastive loss information corresponding to each word segment is obtained. This indicates local contrast loss information. Indicates the first i The similarity between a word segment and the first neighboring word in the context that semantically matches that word segment. This represents the negative neighboring words in the context neighborhood of the i-th word. Similarity between them This represents the temperature coefficient. The above processing enhances the NLASR-NLP model's ability to identify the semantic boundaries between templated and anomalous structural nodes.

[0065] In practical applications, the specific processing methods for determining the boundary sparsity factor corresponding to each word based on each word segment and its corresponding context neighborhood, as well as the position weight of each word segment contained in each structural node, can be varied. The following provides another optional processing method, which may include the processing steps C2 and C4.

[0066] In step C2, based on each word segment and its corresponding context neighborhood, as well as the position weight of each word segment contained in each structural node, the similarity between the negative neighboring word in the context neighborhood of each word segment and the second neighboring word adjacent to the negative neighboring word is determined.

[0067] The specific processing procedure for step C2 above can be found in the aforementioned related content, and will not be repeated here.

[0068] In step C4, the boundary sparsification factor corresponding to each word is determined based on the similarity between the negative neighboring word in the context neighborhood corresponding to each word and the second neighboring word adjacent to the negative neighboring word.

[0069] In implementation, the similarity between the negative example neighboring word in the context neighborhood corresponding to each word and the second neighboring word adjacent to the negative example neighboring word can be substituted into the following formula.

[0070] The boundary sparsity factor corresponding to each word segment is determined. .in, Indicates the first i Negative neighboring word segments in the context of each word segmentation The second neighboring word adjacent to the negative example neighboring word The similarity between them.

[0071] In practical applications, density saliency information can also be set for each structural node. There are various processing methods. The following is another optional processing method, which can include the processing of steps D2 to D6.

[0072] In step D2, the target string is extracted from each structural node. The target string is a meaningless string composed of multiple preset types of characters, including one or more of the following: number, letter, and symbol.

[0073] The target string can be composed of numbers, letters, or symbols, or a combination of numbers, letters, and symbols. Specific target strings can include timestamps, identity IDs, codes, hash values, etc.

[0074] In implementation, to extract highly repetitive but detail-perturbative string-type anomalies (specifically, "numerical anomalies") from structured text data, a DED-A (Dynamic Entropy-aware Density-based Anomaly Estimator) module can be set up, such as... Figure 4As shown, the DED-A module can include a string density anomaly detector, an exponential density function modeler (Exp-Density Encoder), a local mode calibrator (Local Mode Calibrator), and an adaptive thresholding unit (Adaptive Thresholding Unit). The exponential density function modeler can extract the target string from each structural node to construct an exponentially decaying distribution model (avoiding misjudgment of skewed samples by the normal distribution). The local mode calibrator can identify local clustering centers through a sliding window clustering strategy and mine the density core of pseudo-structural characters (such as variable-length encoded IDs). Specifically, it can divide structural nodes or structured text data according to a fixed sliding window length, focus on the distribution density of tokens within the window, and capture the local situation within or around the structural nodes or structured text data. It can calculate the proportion of characters in the target string within the window and then determine the local clustering centers through the sliding window clustering strategy. By combining the sliding window clustering results with the exponentially decaying distribution model, the corresponding density deviation can be corrected. Subsequently, the density significance information corresponding to each structural node can be determined based on the corrected density deviation. For details, please refer to the following related content. The dynamic threshold adapter can automatically adjust the density determination boundary based on the context span, ensuring robust judgment for different structural nodes. Specifically, in the process of extracting the target string from each structural node, multiple consecutive characters can be identified from the structural node. When the number of consecutive characters exceeds a preset threshold, the string consisting of these consecutive characters can be used as the target string.

[0075] In step D4, based on the target string extracted from each structural node, the proportion of characters in the target string in each structural node is determined.

[0076] In implementation, after extracting the target string, string density index modeling can be performed on the target string. Specifically, for any structural node, the proportion of characters in the target string extracted from the structural node can be calculated. The proportion of characters in the target string in the structural node can be calculated using the following formula.

[0077]

[0078] in, Represents structural nodes The proportion of characters in the target string. Represents structural nodes Length of structural nodes (the number of characters contained) Represents the target string The number of characters in the text.

[0079] Using the above method, the proportion of characters in the target string in each structural node can be calculated.

[0080] In step D6, the density significance information corresponding to each structural node is determined based on the proportion of characters in the target string in each structural node and the expected proportion corresponding to each structural node.

[0081] In implementation, the expected proportions corresponding to structural nodes can be pre-set. Specifically, this can be done by pre-setting the expected proportion for each structural node based on expert experience, or by determining the expected proportion for each structural node through statistical analysis of historical data. The specific settings can be tailored to the actual situation. A density significance algorithm can also be pre-set. This algorithm calculates the density significance information for each structural node based on the proportion of characters in the target string within each structural node and the expected proportion for that structural node. Specifically, a string density anomaly detector can be used to take the difference between the proportion of characters in the target string within any structural node and the expected proportion for that structural node as the density significance information for that structural node. Alternatively, the standard deviation for each structural node can be calculated based on the proportion of characters in the target string within each structural node and the expected proportion for that structural node. The density significance information for each structural node can then be determined using the standard deviation. The specific settings can be tailored to the actual situation.

[0082] In practical applications, the absolute value of the difference between the proportion of characters in the target string in each structural node and the expected proportion corresponding to each structural node can be calculated to obtain the density deviation of each structural node. Then, based on the density deviation of each structural node and the preset dynamic threshold, the density significance information of each structural node can be determined. For specific processing methods, please refer to the following related content.

[0083] Based on the processing of steps D2 to D6 above, the specific processing method of step S208 above can be: based on the semantic anomaly saliency information corresponding to each word segment and the density saliency information corresponding to each structural node, determine the corresponding generalizable whitelist information and / or abnormal structural text data from multiple different structural text data.

[0084] In implementation, a weighted summation can be performed based on the semantic anomaly saliency information corresponding to each word segment and the density saliency information corresponding to each structural node, using preset weights. The weighted summation result can be used to determine the degree of anomaly of different structural nodes, thereby identifying the corresponding generalizable whitelist information and / or anomalous structural text data from multiple different structured text data sets. For details, please refer to the aforementioned related content, which will not be repeated here.

[0085] In practical applications, the above-mentioned preset types include digital types, so the specific processing method of step D6 can be varied. The following provides another optional processing method, which can specifically include the processing of steps D62 and D64.

[0086] In step D62, the absolute value of the difference between the proportion of characters in the target string in each structural node and the expected proportion corresponding to each structural node is calculated to obtain the density deviation degree corresponding to each structural node.

[0087] In implementation, the density of structural nodes can be fitted using an exponentially decaying distribution model. Specifically, for any structural node, the length of the structural node is... l The expected proportion corresponding to this structural node is as follows:

[0088] in, Indicates length is The expected proportion corresponding to the structural nodes, a , b and c It is obtained by fitting a pre-set training set.

[0089] For structural nodes This structural node The corresponding expected ratio is

[0090] The above-mentioned structural node can be Corresponding expected proportions and structural nodes Substitute the proportion of characters in the target string into the following formula

[0091] Obtain structural nodes The corresponding density deviation. Represents structural nodes The corresponding density deviation.

[0092] In step D64, the density saliency information corresponding to each structural node is determined based on the density deviation degree corresponding to each structural node and the preset dynamic threshold.

[0093] The preset dynamic threshold can be set according to the actual situation. For example, the dynamic threshold can be set based on expert experience, or it can be determined by summarizing and analyzing historical data.

[0094] In implementation, a string density anomaly detector can be used to substitute the density deviation of each structural node and a preset dynamic threshold into the following formula.

[0095] The density saliency information corresponding to each structural node is obtained. Represents structural nodes The corresponding density significance information, This indicates a dynamic threshold. The DED-A module described above can adapt to changes in the length of structural nodes without relying on hard-coded rules. It also achieves a high rejection rate for random numbers, high-frequency encoding, and dynamic IDs, and can combine exponential fitting to reduce the false positive rate of shorter structural nodes.

[0096] In practical applications, the dynamic threshold is determined by the mean and standard deviation of the structural nodes in multiple different target structured text data, as well as a preset sensitivity factor.

[0097] In implementation, the dynamic threshold can be constructed in the following way, as shown in the formula below.

[0098] in, and They represent lengths of The mean and standard deviation of the structural nodes, This represents the sensitivity factor.

[0099] In practical applications, structural saliency information can also be set for each structural node. There are various processing methods. The following is another optional processing method, which can include the following: Based on the path mode frequency and structural split gain information of each structural node, determine the structural saliency information corresponding to each structural node.

[0100] Structural saliency information can be used to characterize the degree of saliency of structural anomalies in structured text data. The path pattern frequency corresponding to a structural node can be the frequency of occurrence of the structural node.

[0101] In implementation, the path mode frequency and structural splitting gain information corresponding to each structural node can be substituted into the following formula.

[0102] The structural saliency information corresponding to each structural node is obtained, where, Represents structural nodes The corresponding structural saliency information, Represents structural nodes The corresponding path pattern frequency, This represents the maximum frequency of occurrence of structural nodes. This indicates the structural split gain information.

[0103] Based on the above processing, the specific processing method of step S208 can be as follows: based on the semantic anomaly saliency information corresponding to each word segment and the structural saliency information corresponding to each structural node, determine the corresponding generalizable whitelist information and / or abnormal structural text data from multiple different structural text data.

[0104] In implementation, such as Figure 5 As shown, to integrate multi-perspective saliency information, a saliency-driven positive example voting fusion mechanism, SIVT (Saliency-Integrated Voting and Thresholding), can be set up to fuse multi-source saliency information from structural path analysis, semantic model judgment, and density assessment, effectively improving the accuracy, compression rate, and generalization adaptability of the final pattern. Based on this, a weighted summation can be calculated using preset weights based on the semantic anomaly saliency information corresponding to each word segment and the structural saliency information corresponding to each structural node. The weighted summation result can be used to determine the degree of anomaly of different structural nodes, thereby identifying the corresponding generalizable whitelist information and / or anomalous structural text data from multiple different structured text data. For details, please refer to the aforementioned related content, which will not be repeated here.

[0105] In practical applications, such as Figure 5As shown, based on the semantic anomaly saliency information corresponding to each word segment, the density saliency information corresponding to each structural node, and the structural saliency information corresponding to each structural node, corresponding generalizable whitelist information and / or anomalous structural text data can be determined from multiple different structural text data. Specifically, based on the semantic anomaly saliency information corresponding to each word segment, the density saliency information corresponding to each structural node, and the structural saliency information corresponding to each structural node, a weighted summation calculation can be performed using preset weights to obtain the weighted summation result. The degree of anomalousness of different structural nodes can be judged through the weighted summation result, thereby determining the corresponding generalizable whitelist information and / or anomalous structural text data from multiple different structural text data. For details, please refer to the aforementioned related content, which will not be repeated here.

[0106] In practical applications, the specific processing methods for determining the corresponding generalizable whitelist information and / or abnormal structured text data from multiple different structured text data based on the semantic anomaly saliency information corresponding to each word segment and the structural saliency information corresponding to each structural node can be varied. The following provides another optional processing method, which may include the processing steps E2 to E6.

[0107] In step E2, based on the semantic anomaly saliency information corresponding to each word and the structural saliency information corresponding to each structural node, the fusion saliency score corresponding to each word is determined by a preset weight.

[0108] In implementation, such as Figure 5 As shown, a weighted summation can be calculated based on the semantic anomaly saliency information corresponding to each word and the structural saliency information corresponding to each structural node, using preset weights. The fusion saliency score for each word can then be determined using this weighted summation result.

[0109] In step E4, the target word segment whose difference between the fusion saliency score and the preset threshold is within a preset range is obtained, and a corresponding penalty item is set for the fusion saliency score corresponding to the target word segment. Based on the fusion saliency score corresponding to the target word segment and the corresponding penalty item, the saliency score corresponding to the target word segment is determined.

[0110] This allows for the setting of punitive exclusion rules, with the penalty items set according to the actual situation. For example, penalty items can be set based on expert experience, or they can be determined by summarizing and analyzing historical data.

[0111] In step E6, based on the saliency score corresponding to the target word segment, the fusion saliency score corresponding to other words besides the target word segment, and a preset threshold, the corresponding generalizable whitelist information and / or abnormal structured text data are determined from multiple different structured text data.

[0112] The preset threshold can be set according to the actual situation. For example, the preset threshold can be set by expert experience, or it can be determined by summarizing and analyzing historical data.

[0113] In practical applications, the above penalty term is constructed based on the anomalous prior probability corresponding to the target word segmentation and the preset penalty coefficient.

[0114] In practical applications, a threshold drift strategy can be set. Based on this, the above threshold is constructed based on a preset base threshold, the standard deviation of the fusion saliency score corresponding to each word segment, and a preset sensitivity coefficient. If the structural node is related to the preset business, the sensitivity coefficient is less than the first sensitivity threshold; if the structural node is related to long-tail exploration, the sensitivity coefficient is greater than the second sensitivity threshold.

[0115] The threshold can be determined by the following formula.

[0116] in, T dyn Indicates the threshold. T 0 represents the base threshold. This represents the standard deviation of the fusion significance score for each word segment. This represents the sensitivity coefficient.

[0117] In practical applications, structured text data includes one or more of the following: Uniform Resource Locator (URL), Application Programming Interface (API) path, business log sequence, and rich text fragment.

[0118] Based on the above processing, the saliency-driven positive example voting fusion mechanism SIVT integrates multi-source saliency information from structural path analysis, semantic model judgment, and density assessment, and constructs a threshold drift scheduling strategy. Combined with "penalized outcast rules", it filters out false positive examples to effectively improve the accuracy, compression rate, and generalization adaptability of the final pattern, thereby enhancing robustness.

[0119] In another embodiment, such as Figure 5As shown, to achieve the analysis and processing of multiple different structured text data, an NLASR-NLP model can be set up. Then, the processing steps S202 to S208 above can be as follows: Obtain multiple different structured text data, where the structured text data is text data that can be decomposed into multiple different hierarchical or substructured text data with positional dependencies; input the multiple different structured text data into the NLASR-NLP model respectively, and process the NLASR-NLP... The structural navigation encoder in the model performs structural decomposition processing on each structured text data to obtain the corresponding structure nodes. It then performs word segmentation on each structure node to obtain the words contained within it. Combining the location-aware embedding module and the saliency boundary controller, for each word, it determines the contextual neighborhood of each word and its corresponding contextual neighborhood. Based on each word and its corresponding contextual neighborhood, it determines the semantic saliency information between each word and its corresponding contextual neighborhood, the local contrast loss information for each word, and the boundary sparsity factor for each word. Through a multi-scale path aggregator, based on the semantic saliency information between each word and its corresponding contextual neighborhood, the local contrast loss information for each word, and the boundary sparsity factor for each word, it determines the semantic anomalous saliency information for each word. The boundary sparsity factor is used to characterize the semantic consistency degree of the contextual neighborhood corresponding to each word. Finally, based on the semantic anomalous saliency information for each word, it determines the corresponding generalizable whitelist information and / or anomalous structured text data from the multiple different structured text data. For details on the specific processing procedure, please refer to the aforementioned content, which will not be repeated here.

[0120] In addition, such as Figure 5As shown, in addition to the NLASR-NLP model, it can also include the DED-A module and the saliency-driven positive example voting fusion module. The saliency-driven positive example voting fusion module can be configured with a saliency-driven positive example voting fusion mechanism. The DED-A module can include an exponential density function modeler, a local pattern fitter, and a dynamic threshold adapter. The specific functions and processing procedures of each module or unit can be found in the aforementioned related content, and will not be repeated here. The saliency-driven positive example voting fusion module can include a candidate result collector, a multi-factor fusion scorer, and an outlier suppression engine. The candidate result collector can be used to aggregate multiple data sources (such as the decomposition of structured text data, outlier reverse voting, etc.) to form an initial whitelist information. The multi-factor fusion scorer can be used to fuse the semantic outlier saliency information corresponding to each word segment with the density saliency information corresponding to each structural node and / or the structural saliency information corresponding to each structural node. The outlier suppression engine can be used to determine the corresponding generalizable whitelist information and / or outlier structured text data from multiple different structured text data based on the fusion results and the specified elimination strategy.

[0121] This specification provides a data processing method. It involves acquiring multiple different structured text data that can be decomposed into multiple different hierarchical or substructured structures with positional dependencies. Each structured text data is then structurally decomposed to obtain a corresponding structure node. Each structure node is then segmented to obtain the words contained within it. For each word, the contextual neighborhood of each word and its corresponding contextual neighborhood is determined. Based on each word and its corresponding contextual neighborhood, semantic saliency information between each word and its corresponding contextual neighborhood, local contrast loss information for each word, and boundary sparsity factor for each word are determined. Furthermore, based on the semantic saliency information between each word and its corresponding contextual neighborhood, and the local contrast loss information for each word, the method can further determine the semantic saliency information between each word and its corresponding contextual neighborhood, and the local contrast loss information for each word. By comparing the loss information and the boundary sparsity factor corresponding to each word, the semantic anomaly saliency information corresponding to each word is determined. Finally, based on the semantic anomaly saliency information corresponding to each word, corresponding generalizable whitelist information and / or anomalous structured text data can be determined from multiple different structured text data. Thus, by introducing semantic anomaly saliency to verify the semantic consistency of structural node emptiness, semantic recognition capability is improved. Furthermore, the semantic anomaly saliency is evaluated through the semantic saliency information between each word and its corresponding context neighborhood, the local contrastive loss information corresponding to each word, and the boundary sparsity factor corresponding to each word, thereby enhancing the semantic generalization capability of the business system. This makes the content recognition and risk detection mechanism in the business system more robust and interpretable, and improves the accuracy of content recognition and risk detection. In addition, this method has the ability to generalize across different types of structural content, adapting to different structural types of data such as WebAPI, log paths, and file resources without reconstructing the feature extraction logic. Moreover, it can form a highly modular anomaly recognition system, thus significantly improving the multimodal adaptability and evolutionary maintenance efficiency of the business system.

[0122] In addition, the confidence constraint and outlier disambiguation of the parameter domain are performed by using target string density detection and saliency-driven positive example voting fusion mechanism, thereby reducing the outlier false positive rate in content recognition and risk detection. Moreover, the voting of multiple saliency information can offset occasional biases. Furthermore, the plug-in architecture with a modular strategy can enhance the interpretation and debugging capabilities. In addition, the models and modules have incremental update capabilities and low-resource rapid deployment characteristics.

[0123] The following describes in detail a data processing method provided by the embodiments of this specification, with specific application scenarios, wherein the structured text data is a URL.

[0124] like Figure 6As shown in the embodiments of this specification, a data processing method is provided. The execution subject of this method can be a terminal device or a server, etc. The terminal device can be a mobile terminal device such as a mobile phone or tablet computer, a computer device such as a laptop or desktop computer, or an IoT device (specifically, a smartwatch, in-vehicle device, etc.). The server can be a single server or a server cluster composed of multiple servers. The server can be a backend server for financial business or online shopping business, or a backend server for an application, etc. This embodiment uses a server as the execution subject for detailed description. For the case where the execution subject is a terminal device, please refer to the following server case processing, which will not be repeated here. The method may specifically include the following steps: In step S602, multiple different URLs are obtained.

[0125] In step S604, each URL is structurally decomposed to obtain the structural nodes corresponding to each URL, and each structural node is segmented to obtain the segmented words contained in each structural node.

[0126] In step S606, the position weights corresponding to the word segments contained in each structural node are determined.

[0127] In step S608, based on each word segment and its corresponding context neighborhood, as well as the position weights of the words contained in each structural node, attention information corresponding to each word is determined through an attention algorithm.

[0128] In step S610, based on each word segment and its corresponding context neighborhood, and combining the position weights of the words contained in each structural node and the attention information of each word segment, the average similarity between each word segment and its corresponding context neighborhood is determined.

[0129] In step S612, based on the average similarity between each word and its corresponding context neighborhood, the semantic saliency information between each word and its corresponding context neighborhood is determined.

[0130] In step S614, based on each word and its corresponding context neighborhood, as well as the position weight of each word contained in each structural node, the similarity between each word and the semantically matched first neighboring word in the context neighborhood of its corresponding word, and the similarity between each word and the negative example neighboring word in the context neighborhood of its corresponding word are determined.

[0131] In step S616, based on the similarity between each word and the first neighboring word in the context neighborhood corresponding to the corresponding word, and the similarity between each word and the negative example neighboring word in the context neighborhood corresponding to the corresponding word, the local contrast loss information corresponding to each word is determined.

[0132] In step S618, based on each word segment and its corresponding context neighborhood, as well as the position weight of the word segment contained in each structural node, the similarity between the negative neighbor word in the context neighborhood corresponding to each word segment and the second neighbor word adjacent to the negative neighbor word is determined.

[0133] In step S620, the boundary sparsification factor corresponding to each word is determined based on the similarity between the negative neighboring word in the context neighborhood corresponding to each word and the second neighboring word adjacent to the negative neighboring word.

[0134] In step S622, based on the semantic saliency information between each word and its corresponding context neighborhood, the local contrast loss information corresponding to each word, and the boundary sparsification factor corresponding to each word, the semantic abnormal saliency information corresponding to each word is determined.

[0135] In step S624, the target string is extracted from each structure node. The target string consists of numbers.

[0136] In step S626, the absolute value of the difference between the proportion of characters in the target string in each structural node and the expected proportion corresponding to each structural node is calculated to obtain the density deviation degree corresponding to each structural node.

[0137] In step S628, the density saliency information corresponding to each structural node is determined based on the density deviation degree corresponding to each structural node and the preset dynamic threshold.

[0138] The dynamic threshold is determined by the mean and standard deviation of the structural nodes in multiple different target URLs, as well as a preset sensitivity factor.

[0139] In step S630, the density significance information corresponding to each structural node is determined based on the proportion of characters in the target string in each structural node and the expected proportion corresponding to each structural node.

[0140] In step S632, the structural saliency information corresponding to each structural node is determined based on the path mode frequency and structural split gain information corresponding to each structural node.

[0141] In step S634, based on the semantic anomaly saliency information corresponding to each word, the density saliency information corresponding to each structural node, and the structural saliency information corresponding to each structural node, the fusion saliency score corresponding to each word is determined by a preset weight.

[0142] In step S636, the target word segment whose difference between the fusion saliency score and the preset threshold is within a preset range is obtained, and a corresponding penalty item is set for the fusion saliency score corresponding to the target word segment. Based on the fusion saliency score corresponding to the target word segment and the corresponding penalty item, the saliency score corresponding to the target word segment is determined.

[0143] The penalty term is constructed based on the anomalous prior probability corresponding to the target word segmentation and a preset penalty coefficient.

[0144] In step S638, based on the saliency score corresponding to the target word segment, the fusion saliency score corresponding to other words besides the target word segment, and a preset threshold, the corresponding generalizable whitelist information and / or abnormal URLs are determined from multiple different URLs.

[0145] The threshold is constructed based on a preset base threshold, the annotation difference corresponding to the fusion saliency score of each word segment, and a preset sensitivity coefficient. If the structural node is related to a preset business, the sensitivity coefficient is less than the first sensitivity threshold; if the structural node is related to long-tail exploration, the sensitivity coefficient is greater than the second sensitivity threshold.

[0146] This specification provides a data processing method. It involves acquiring multiple different structured text data that can be decomposed into multiple different hierarchical or substructured structures with positional dependencies. Each structured text data is then structurally decomposed to obtain a corresponding structure node. Each structure node is then segmented to obtain the words contained within it. For each word, the contextual neighborhood of each word and its corresponding contextual neighborhood is determined. Based on each word and its corresponding contextual neighborhood, semantic saliency information between each word and its corresponding contextual neighborhood, local contrast loss information for each word, and boundary sparsity factor for each word are determined. Furthermore, based on the semantic saliency information between each word and its corresponding contextual neighborhood, and the local contrast loss information for each word, the method can further determine the semantic saliency information between each word and its corresponding contextual neighborhood, and the local contrast loss information for each word. By comparing the loss information and the boundary sparsity factor corresponding to each word, the semantic anomaly saliency information corresponding to each word is determined. Finally, based on the semantic anomaly saliency information corresponding to each word, corresponding generalizable whitelist information and / or anomalous structured text data can be determined from multiple different structured text data. Thus, by introducing semantic anomaly saliency to verify the semantic consistency of structural node emptiness, semantic recognition capability is improved. Furthermore, the semantic anomaly saliency is evaluated through the semantic saliency information between each word and its corresponding context neighborhood, the local contrastive loss information corresponding to each word, and the boundary sparsity factor corresponding to each word, thereby enhancing the semantic generalization capability of the business system. This makes the content recognition and risk detection mechanism in the business system more robust and interpretable, and improves the accuracy of content recognition and risk detection. In addition, this method has the ability to generalize across different types of structural content, adapting to different structural types of data such as WebAPI, log paths, and file resources without reconstructing the feature extraction logic. Moreover, it can form a highly modular anomaly recognition system, thus significantly improving the multimodal adaptability and evolutionary maintenance efficiency of the business system.

[0147] The above describes the data processing method provided in the embodiments of this specification. Based on the same idea, the embodiments of this specification also provide a data processing apparatus, such as... Figure 7 As shown.

[0148] The data processing device includes: a data acquisition module 701, a structural navigation coding module 702, a semantic saliency determination module 703, and a recognition module 704, wherein: The data acquisition module 701 acquires multiple different structured text data, wherein the structured text data is text data that can be decomposed into multiple different levels or substructures with positional dependencies. The structure navigation encoding module 702 performs structured decomposition processing on each structured text data to obtain the structure nodes corresponding to each structured text data, and performs word segmentation processing on each structure node to obtain the words contained in each structure node. The semantic saliency determination module 703, for each word segment, determines the contextual neighborhood corresponding to each word segment and the corresponding contextual neighborhood, and based on each word segment and the corresponding contextual neighborhood, determines the semantic saliency information between each word segment and the corresponding contextual neighborhood, the local contrast loss information corresponding to each word segment, and the boundary sparsity factor corresponding to each word segment; based on the semantic saliency information between each word segment and the corresponding contextual neighborhood, the local contrast loss information corresponding to each word segment, and the boundary sparsity factor corresponding to each word segment, determines the semantic abnormality saliency information corresponding to each word segment, wherein the boundary sparsity factor is used to characterize the semantic consistency degree of the contextual neighborhood corresponding to each word segment; The identification module 704 determines the corresponding generalizable whitelist information and / or abnormal structured text data from the multiple different structured text data based on the semantic anomaly saliency information corresponding to each word segmentation.

[0149] In the embodiments described in this specification, the device further includes: The position-aware module determines the position weight of the word segments contained in each structural node; The semantic saliency determination module 703 determines the semantic saliency information between each word and its corresponding context neighborhood, the local contrast loss information of each word, and the boundary sparsification factor of each word based on each word segment and its corresponding context neighborhood, as well as the position weight of each word segment contained in each structural node.

[0150] In the embodiments of this specification, the semantic saliency determination module 703 includes: The semantic density determination unit determines the average similarity between each word and its corresponding context neighborhood based on each word and the position weight of each word contained in each structural node. The semantic saliency determination unit determines the semantic saliency information between each word and its corresponding context neighborhood based on the average similarity between each word and the corresponding context neighborhood.

[0151] In the embodiments of this specification, the semantic saliency determination unit determines the attention information corresponding to each word by using an attention algorithm, based on each word and its corresponding context neighborhood, as well as the position weights of the words contained in each structural node; and determines the average similarity between each word and its corresponding context neighborhood by combining the position weights of the words contained in each structural node and the attention information of each word.

[0152] In the embodiments of this specification, the semantic saliency determination module 703 includes: The similarity determination unit determines the similarity between each word and the first neighboring word in the context of the corresponding word, and the similarity between each word and the negative neighboring word in the context of the corresponding word, based on each word and the corresponding context of each word and the position weight of each word contained in each structural node. The local contrast loss determination unit determines the local contrast loss information for each word based on the similarity between each word and the first neighboring word in the context neighborhood corresponding to the corresponding word, and the similarity between each word and the negative example neighboring word in the context neighborhood corresponding to the corresponding word.

[0153] In the embodiments of this specification, the semantic saliency determination module 703 includes: The neighborhood similarity determination unit determines the similarity between the negative example neighboring words in the context neighborhood corresponding to each word and the corresponding context neighborhood of each word, as well as the position weight of the words contained in each structural node, and the similarity between the negative example neighboring words and the second neighboring words adjacent to the negative example neighboring words. The boundary sparsification unit determines the boundary sparsification factor for each word based on the similarity between the negative neighboring word in the context neighborhood corresponding to each word and the second neighboring word adjacent to the negative neighboring word.

[0154] In the embodiments described in this specification, the device further includes: The string extraction module extracts the target string from each structural node. The target string is a meaningless string composed of multiple preset types of characters, including one or more of the following: number, letter, and symbol. The character ratio determination module determines the ratio of characters in the target string extracted from each structural node. The density significance determination module determines the density significance information of each structural node based on the proportion of characters in the target string in each structural node and the expected proportion corresponding to each structural node. The identification module 704 determines the corresponding generalizable whitelist information and / or abnormal structured text data from the multiple different structured text data based on the semantic anomaly saliency information corresponding to each word segment and the density saliency information corresponding to each structural node.

[0155] In this embodiment of the specification, the preset type includes a number class, and the density salience determination module includes: The density deviation determination unit calculates the absolute value of the difference between the proportion of characters in the target string in each structural node and the expected proportion corresponding to each structural node, and obtains the density deviation corresponding to each structural node. The density saliency determination unit determines the density saliency information of each structural node based on the density deviation degree corresponding to each structural node and a preset dynamic threshold.

[0156] In the embodiments of this specification, the dynamic threshold is determined by the mean and standard deviation of the structural nodes in multiple different target structured text data, as well as a preset sensitivity factor.

[0157] In the embodiments described in this specification, the device further includes: The structural saliency determination module determines the structural saliency information corresponding to each structural node based on the path mode frequency and structural splitting gain information corresponding to each structural node. The identification module 704 determines the corresponding generalizable whitelist information and / or abnormal structured text data from the multiple different structured text data based on the semantic anomaly saliency information corresponding to each word segment and the structural saliency information corresponding to each structural node.

[0158] In this embodiment of the specification, the identification module 704 includes: The fusion scoring unit determines the fusion saliency score for each word based on the semantic anomaly saliency information corresponding to each word and the structural saliency information corresponding to each structural node, through preset weights. The penalty unit obtains the target word whose difference between the fusion saliency score and the preset threshold is within a preset range, sets a corresponding penalty item for the fusion saliency score corresponding to the target word, and determines the saliency score corresponding to the target word based on the fusion saliency score corresponding to the target word and the corresponding penalty item. The identification unit determines, based on the saliency score corresponding to the target word, the fusion saliency score corresponding to other words besides the target word, and the preset threshold, the corresponding generalizable whitelist information and / or abnormal structured text data from the multiple different structured text data.

[0159] In the embodiments described in this specification, the penalty term is constructed based on the anomalous prior probability corresponding to the target word segmentation and a preset penalty coefficient.

[0160] In this embodiment of the specification, the threshold is constructed based on a preset basic threshold, the annotation difference corresponding to the fusion saliency score of each word, and a preset sensitivity coefficient. If the structural node is related to a preset business, the sensitivity coefficient is less than the first sensitivity threshold; if the structural node is related to long-tail exploration, the sensitivity coefficient is greater than the second sensitivity threshold.

[0161] In the embodiments described in this specification, the structured text data includes one or more of the following: Uniform Resource Locator (URL), Application Programming Interface (API) path, business log sequence, and rich text fragment.

[0162] This specification provides a data processing apparatus that acquires multiple different structured text data that can be decomposed into multiple different hierarchical or substructured structures with positional dependencies. Each structured text data is structurally decomposed to obtain a corresponding structure node. Each structure node is then segmented to obtain the words contained within it. For each word, the contextual neighborhood of each word and its corresponding contextual neighborhood is determined. Based on each word and its corresponding contextual neighborhood, semantic saliency information between each word and its corresponding contextual neighborhood, local contrast loss information for each word, and boundary sparsity factor for each word are determined. Furthermore, based on the semantic saliency information between each word and its corresponding contextual neighborhood, and the local contrast loss information for each word, the apparatus can determine the semantic saliency information between each word and its corresponding contextual neighborhood, as well as the local contrast loss information for each word. By comparing the loss information and the boundary sparsity factor corresponding to each word, the semantic anomaly saliency information corresponding to each word is determined. Finally, based on the semantic anomaly saliency information corresponding to each word, corresponding generalizable whitelist information and / or anomalous structured text data can be determined from multiple different structured text data. Thus, by introducing semantic anomaly saliency to verify the semantic consistency of structural node emptiness, semantic recognition capability is improved. Furthermore, the semantic anomaly saliency is evaluated through the semantic saliency information between each word and its corresponding context neighborhood, the local contrastive loss information corresponding to each word, and the boundary sparsity factor corresponding to each word, thereby enhancing the semantic generalization capability of the business system. This makes the content recognition and risk detection mechanism in the business system more robust and interpretable, and improves the accuracy of content recognition and risk detection. In addition, this method has the ability to generalize across different types of structural content, adapting to different structural types of data such as WebAPI, log paths, and file resources without reconstructing the feature extraction logic. Moreover, it can form a highly modular anomaly recognition system, thus significantly improving the multimodal adaptability and evolutionary maintenance efficiency of the business system.

[0163] In addition, the confidence constraint and outlier disambiguation of the parameter domain are performed by using target string density detection and saliency-driven positive example voting fusion mechanism, thereby reducing the outlier false positive rate in content recognition and risk detection. Moreover, the voting of multiple saliency information can offset occasional biases. Furthermore, the plug-in architecture with a modular strategy can enhance the interpretation and debugging capabilities. In addition, the models and modules have incremental update capabilities and low-resource rapid deployment characteristics.

[0164] The above describes the data processing apparatus provided in the embodiments of this specification. Based on the same idea, the embodiments of this specification also provide a data processing device, such as... Figure 8 As shown.

[0165] The data processing device can be a terminal device or a server, as described in the above embodiments.

[0166] Data processing devices can vary significantly in configuration and performance, and may include a communication interface 802, a user interface 804, a processor 806, and a data storage 808. These components are interconnected and communicate with each other via a system bus, network, or other connection mechanism 810. The communication interface 802 enables the data processing device 800 to communicate with other devices, access networks, and transmission networks via analog or digital modulation. For example, the communication interface 802 may include a chipset and antenna for wireless communication with a radio access network or access point. Furthermore, the communication interface 802 can be a wired interface such as Ethernet, Token Ring, or a USB port, or a wireless interface such as Wi-Fi, Bluetooth, Global Positioning System (GPS), or a wide-area wireless interface (e.g., WiMAX or LTE). Of course, the communication interface 802 may also support other forms of physical layer interfaces and standard or proprietary communication protocols. The communication interface 802 may also include multiple physical communication interfaces, such as Wi-Fi, Bluetooth, and wide-area wireless interfaces.

[0167] User interface 804 includes receiving user input and providing output to the user. Therefore, user interface 804 may include input components such as a keypad, keyboard, touch-sensitive or presence-sensitive panel, computer mouse, trackball, joystick, microphone, still camera, and video camera, and output components such as a display screen (which may be combined with a touch-sensitive panel), CRT, LCD, LED, display using DLP technology, printer, and other similar devices known or developed in the future. User interface 804 may also generate auditory output via speakers, speaker jacks, audio output ports, audio output devices, headphones, and other similar devices known or developed in the future. In some embodiments, user interface 804 may include software, circuitry, or other forms of logic capable of transmitting and receiving data from external user input / output devices. Additionally or alternatively, data processing device 800 may support remote access from other devices via communication interface 802 or another physical interface (not shown). User interface 804 may be configured to receive user input, the position and movement of which may be indicated by indicators or cursors described herein. User interface 804 may also be configured as a display device for rendering or displaying text fragments.

[0168] The processor 806 may contain one or more general-purpose processors and / or special-purpose processors.

[0169] Data storage 808 may include one or more volatile and / or non-volatile storage components and may be integrated wholly or partially with processor 806. Data storage 808 may include removable and non-removable components.

[0170] Processor 806 is capable of executing program instructions 818 (e.g., compiled or uncompiled program logic and / or machine code) stored in data storage 808 to perform the various functions described herein. Data storage 808 may contain a non-transitory computer-readable medium on which program instructions are stored, which, when executed by data processing device 800, enable data processing device 800 to perform any methods, processes, or functions disclosed in this specification and / or the accompanying drawings. Processor 806 executing program instructions 818 may result in processor 806 using data 812.

[0171] For example, program instructions 818 may include an operating system 822 (e.g., an operating system kernel, device drivers, and / or other modules) and one or more application programs 820 (e.g., a browser, social media application, or game application) installed on the data processing device 800. Similarly, data 812 may include operating system data 816 and application data 814. Operating system data 816 is primarily accessible to the operating system 822, while application data 814 is primarily accessible to one or more application programs 820. Application data 814 may reside in a file system that is visible or hidden from the user on the data processing device 800.

[0172] Application 820 can communicate with operating system 812 through one or more application programming interfaces (APIs). These APIs help application 820 read and / or write application data 814, transmit or receive information via communication interface 802, receive or display information on user interface 804, etc.

[0173] In some terminology, application 820 may be simply referred to as "app". Furthermore, application 820 can be downloaded to data processing device 800 through one or more online app stores or app markets. However, applications can also be installed on data processing device 800 in other ways, such as through a web browser or a physical interface on data processing device 800 (e.g., a USB port).

[0174] Specifically, in this embodiment, the data processing device 800 includes a data storage 808 and one or more program instructions 818, wherein one or more program instructions 818 are stored in the data storage 808, and one or more program instructions 818 are configured to be executed by one or more processors. The one or more program instructions include computer-executable instructions for performing the following: Acquire multiple different structured text data, wherein the structured text data is text data that can be decomposed into multiple different hierarchical or substructures with positional dependencies; Each structured text data is structurally decomposed to obtain the corresponding structure node, and each structure node is segmented to obtain the words contained in each structure node. For each word segment, the context neighborhood corresponding to each word and its corresponding context neighborhood is determined. Based on each word segment and its corresponding context neighborhood, the semantic saliency information between each word segment and its corresponding context neighborhood, the local contrast loss information corresponding to each word, and the boundary sparsity factor corresponding to each word are determined. Based on the semantic saliency information between each word segment and its corresponding context neighborhood, the local contrast loss information corresponding to each word, and the boundary sparsity factor corresponding to each word, the semantic anomaly saliency information corresponding to each word is determined. The boundary sparsity factor is used to characterize the semantic consistency degree of the context neighborhood corresponding to each word. Based on the semantic anomaly saliency information corresponding to each word segmentation, the corresponding generalizable whitelist information and / or abnormal structured text data are determined from the multiple different structured text data.

[0175] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the data processing device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0176] This specification provides a data processing device that acquires multiple different structured text data that can be decomposed into multiple different hierarchical or substructured structures with positional dependencies. Each structured text data is structurally decomposed to obtain a corresponding structure node. Each structure node is then segmented to obtain the words contained within it. For each word, the device determines the contextual neighborhood of each word and its corresponding contextual neighborhood. Based on each word and its corresponding contextual neighborhood, the device determines the semantic saliency information between each word and its corresponding contextual neighborhood, the local contrast loss information for each word, and the boundary sparsity factor for each word. Furthermore, based on the semantic saliency information between each word and its corresponding contextual neighborhood, and the local contrast loss information for each word, the device can determine the semantic saliency information between each word and its corresponding contextual neighborhood, the local contrast loss information for each word, and the boundary sparsity factor for each word. By comparing the loss information and the boundary sparsity factor corresponding to each word, the semantic anomaly saliency information corresponding to each word is determined. Finally, based on the semantic anomaly saliency information corresponding to each word, corresponding generalizable whitelist information and / or anomalous structured text data can be determined from multiple different structured text data. Thus, by introducing semantic anomaly saliency to verify the semantic consistency of structural node emptiness, semantic recognition capability is improved. Furthermore, the semantic anomaly saliency is evaluated through the semantic saliency information between each word and its corresponding context neighborhood, the local contrastive loss information corresponding to each word, and the boundary sparsity factor corresponding to each word, thereby enhancing the semantic generalization capability of the business system. This makes the content recognition and risk detection mechanism in the business system more robust and interpretable, and improves the accuracy of content recognition and risk detection. In addition, this method has the ability to generalize across different types of structural content, adapting to different structural types of data such as WebAPI, log paths, and file resources without reconstructing the feature extraction logic. Moreover, it can form a highly modular anomaly recognition system, thus significantly improving the multimodal adaptability and evolutionary maintenance efficiency of the business system.

[0177] Furthermore, based on the above Figures 1 to 6 This specification also provides a storage medium for storing computer-executable instruction information in one or more embodiments. In one specific embodiment, the storage medium may be a USB flash drive, optical disc, hard disk, etc. When the computer-executable instruction information stored in the storage medium is executed by a processor, it can realize the following process: Acquire multiple different structured text data, wherein the structured text data is text data that can be decomposed into multiple different hierarchical or substructures with positional dependencies; Each structured text data is structurally decomposed to obtain the corresponding structure node, and each structure node is segmented to obtain the words contained in each structure node. For each word segment, the context neighborhood corresponding to each word and its corresponding context neighborhood is determined. Based on each word segment and its corresponding context neighborhood, the semantic saliency information between each word segment and its corresponding context neighborhood, the local contrast loss information corresponding to each word, and the boundary sparsity factor corresponding to each word are determined. Based on the semantic saliency information between each word segment and its corresponding context neighborhood, the local contrast loss information corresponding to each word, and the boundary sparsity factor corresponding to each word, the semantic anomaly saliency information corresponding to each word is determined. The boundary sparsity factor is used to characterize the semantic consistency degree of the context neighborhood corresponding to each word. Based on the semantic anomaly saliency information corresponding to each word segmentation, the corresponding generalizable whitelist information and / or abnormal structured text data are determined from the multiple different structured text data.

[0178] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the above-described storage medium embodiment is basically similar to the method embodiment, so the description is relatively simple; relevant parts can be referred to the description of the method embodiment.

[0179] This specification provides a storage medium that acquires multiple different structured text data that can be decomposed into multiple different hierarchical or substructures with positional dependencies. Each structured text data is structurally decomposed to obtain a corresponding structure node. Each structure node is then segmented to obtain the words contained within it. For each word, the contextual neighborhood of each word and its corresponding contextual neighborhood is determined. Based on each word and its corresponding contextual neighborhood, semantic saliency information between each word and its corresponding contextual neighborhood, local contrastive loss information for each word, and boundary sparsity factor for each word are determined. Furthermore, based on the semantic saliency information between each word and its corresponding contextual neighborhood, and the local contrastive loss factor for each word, the storage medium can further determine the semantic saliency information between each word and its corresponding contextual neighborhood, and the local contrastive loss factor for each word. By using loss information and the boundary sparsity factor corresponding to each word, the semantic anomaly saliency information corresponding to each word is determined. Finally, based on the semantic anomaly saliency information corresponding to each word, corresponding generalizable whitelist information and / or anomalous structured text data can be determined from multiple different structured text data. Thus, by introducing semantic anomaly saliency to verify the semantic consistency of structural node emptiness, semantic recognition capability is improved. Furthermore, the semantic anomaly saliency is evaluated through the semantic saliency information between each word and its corresponding context neighborhood, the local contrast loss information corresponding to each word, and the boundary sparsity factor corresponding to each word, thereby enhancing the semantic generalization capability of the business system. This makes the content recognition and risk detection mechanism in the business system more robust and interpretable, and improves the accuracy of content recognition and risk detection. In addition, this method has the ability to generalize across different types of structural content, adapting to different structural types of data such as Web APIs, log paths, and file resources without reconstructing the feature extraction logic. Moreover, it can form a highly modular anomaly recognition system, thus significantly improving the multimodal adaptability and evolutionary maintenance efficiency of the business system.

[0180] Furthermore, based on the above Figures 1 to 6 This specification also provides one or more embodiments of a computer program product, including a computer program, which, when executed by a processor, can perform the following processes: Acquire multiple different structured text data, wherein the structured text data is text data that can be decomposed into multiple different hierarchical or substructures with positional dependencies; Each structured text data is structurally decomposed to obtain the corresponding structure node, and each structure node is segmented to obtain the words contained in each structure node. For each word segment, the context neighborhood corresponding to each word and its corresponding context neighborhood is determined. Based on each word segment and its corresponding context neighborhood, the semantic saliency information between each word segment and its corresponding context neighborhood, the local contrast loss information corresponding to each word, and the boundary sparsity factor corresponding to each word are determined. Based on the semantic saliency information between each word segment and its corresponding context neighborhood, the local contrast loss information corresponding to each word, and the boundary sparsity factor corresponding to each word, the semantic anomaly saliency information corresponding to each word is determined. The boundary sparsity factor is used to characterize the semantic consistency degree of the context neighborhood corresponding to each word. Based on the semantic anomaly saliency information corresponding to each word segmentation, the corresponding generalizable whitelist information and / or abnormal structured text data are determined from the multiple different structured text data.

[0181] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the above-described embodiment of a computer program product is relatively simple in description because it is fundamentally similar to the method embodiment; relevant parts can be referred to the description of the method embodiment.

[0182] This specification provides a computer program product that acquires multiple different structured text data that can be decomposed into multiple different hierarchical or substructures with positional dependencies. Each structured text data is structurally decomposed to obtain a corresponding structure node. Each structure node is then segmented to obtain the words contained within it. For each word, the contextual neighborhood of each word and its corresponding contextual neighborhood is determined. Based on each word and its corresponding contextual neighborhood, semantic saliency information between each word and its corresponding contextual neighborhood, local contrastive loss information for each word, and boundary sparsity factor for each word are determined. Furthermore, based on the semantic saliency information between each word and its corresponding contextual neighborhood, and the local contrastive loss information for each word, ... semantic saliency information between each word and its corresponding contextual neighborhood, the semantic saliency information between each word, and the local contrastive loss information for each word, the semantic saliency information between each word, and the local contrastive loss information for each word, the semantic saliency information between each word, and the local contrastive loss information for each word, the semantic saliency information between each word, and the local contrastive loss information for each word, the semantic saliency information between each word, and the semantic saliency information between each word, the semantic saliency information between each word, and the local contrastive loss information for each word, By comparing the loss information and the boundary sparsity factor corresponding to each word, the semantic anomaly saliency information corresponding to each word is determined. Finally, based on the semantic anomaly saliency information corresponding to each word, corresponding generalizable whitelist information and / or anomalous structured text data can be determined from multiple different structured text data. Thus, by introducing semantic anomaly saliency to verify the semantic consistency of structural node emptiness, semantic recognition capability is improved. Furthermore, the semantic anomaly saliency is evaluated through the semantic saliency information between each word and its corresponding context neighborhood, the local contrastive loss information corresponding to each word, and the boundary sparsity factor corresponding to each word, thereby enhancing the semantic generalization capability of the business system. This makes the content recognition and risk detection mechanism in the business system more robust and interpretable, and improves the accuracy of content recognition and risk detection. In addition, this method has the ability to generalize across different types of structural content, adapting to different structural types of data such as WebAPI, log paths, and file resources without reconstructing the feature extraction logic. Moreover, it can form a highly modular anomaly recognition system, thus significantly improving the multimodal adaptability and evolutionary maintenance efficiency of the business system.

[0183] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in a different order than those shown in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are possible or may be advantageous. Moreover, although one or more embodiments of this specification provide method steps as described in the embodiments or flowcharts, it is understood that the order of steps listed in the embodiments or flowcharts is merely one possible execution order among many steps and does not represent the only execution order. Therefore, when method steps are involved in the claims, adjustments to the order of those steps, or parallelism between steps, are also within the scope of protection of the claims.

[0184] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0185] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0186] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0187] For ease of description, the above apparatus is described by dividing it into various functional units. Of course, when implementing one or more embodiments of this specification, the functions of each unit can be implemented in one or more software and / or hardware.

[0188] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, one or more embodiments of this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0189] Embodiments in this specification are described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable parallel device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable parallel device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0190] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable fraud device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0191] These computer program instructions can also be loaded onto a computer or other programmable device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0192] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0193] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0194] Computer-readable media include both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0195] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical or equivalent elements in the process, method, article, or apparatus that includes said element. Furthermore, "a," "an," and "the" are not specifically singular and may include plural forms. Ordinal numbers such as "first," "second," etc., do not necessarily indicate order; they are often used to distinguish objects. For example, "first server" and "second server" usually refer to two servers, described as "first server" and "second server" to differentiate them; however, sometimes these two servers may be the same server. Moreover, in this specification, unless explicitly stated otherwise, "receiving and sending data" does not necessarily mean direct receiving and sending; it can be indirect receiving and sending (i.e., receiving and sending indirectly through one or more entities). Similarly, in this specification, unless otherwise stated, the relationships between structures can be direct or indirect.

[0196] Furthermore, the specific terms used in this specification to describe embodiments, such as "an embodiment," "one embodiment," or "some embodiments," refer to a particular feature, structure, or characteristic related to at least one embodiment of this specification. Therefore, it should be emphasized and noted that "an embodiment," "one embodiment," or "an alternative embodiment" mentioned twice or more in different locations in this specification do not necessarily refer to the same embodiment. Moreover, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples, without contradiction.

[0197] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, one or more embodiments of this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0198] One or more embodiments of this specification can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. One or more embodiments of this specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0199] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0200] The above description is merely an embodiment of this specification and is not intended to limit this document. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims in this document.

Claims

1. A data processing method, the method comprising: Acquire multiple different structured text data, wherein the structured text data is text data that can be decomposed into multiple different hierarchical or substructures with positional dependencies; Each structured text data is structurally decomposed to obtain the corresponding structure node, and each structure node is segmented to obtain the words contained in each structure node. For each word segment, determine the context neighborhood of each word and the corresponding word, and based on each word and the corresponding word's context neighborhood, determine the semantic saliency information between each word and the corresponding word's context neighborhood, the local contrast loss information of each word, and the boundary sparsity factor of each word. Based on the semantic saliency information between each word and its corresponding context neighborhood, the local contrast loss information corresponding to each word, and the boundary sparsification factor corresponding to each word, the semantic abnormality saliency information corresponding to each word is determined. The boundary sparsification factor is used to characterize the semantic consistency degree of the context neighborhood corresponding to each word. Based on the semantic anomaly saliency information corresponding to each word segmentation, the corresponding generalizable whitelist information and / or abnormal structured text data are determined from the multiple different structured text data.

2. The method according to claim 1, further comprising: Determine the positional weights of the words contained in each structural node; The step of determining semantic saliency information between each word segment and its corresponding context neighborhood, local contrastive loss information for each word segment, and boundary sparsification factor for each word segment based on each word segment and its corresponding context neighborhood includes: Based on each word segment and its corresponding context neighborhood, as well as the position weights of the words contained in each structural node, the semantic saliency information between each word segment and its corresponding context neighborhood, the local contrast loss information of each word segment, and the boundary sparsity factor of each word segment are determined respectively.

3. The method according to claim 2, based on each word segment and its corresponding context neighborhood, and the positional weights of the words contained in each structural node, determines the semantic saliency information between each word segment and its corresponding context neighborhood, including: Based on each word segment and its corresponding context neighborhood, as well as the position weights of the words contained in each structural node, the average similarity between each word segment and its corresponding context neighborhood is determined. Based on the average similarity between each word and its corresponding context neighborhood, the semantic saliency information between each word and its corresponding context neighborhood is determined.

4. The method according to claim 3, wherein determining the average similarity between each word segment and its corresponding context neighborhood based on each word segment and the position weight of the word segment contained in each structural node includes: Based on each word segment and its corresponding context neighborhood, as well as the position weights of the words contained in each structural node, the attention information corresponding to each word is determined through an attention algorithm. Based on each word segment and its corresponding context neighborhood, and combining the positional weights of each word segment contained in each structural node with the attention information of each word segment, the average similarity between each word segment and its corresponding context neighborhood is determined.

5. The method according to claim 2, based on each word segment and its corresponding context neighborhood, and the positional weights of the words contained in each structural node, determines the local contrastive loss information corresponding to each word segment, including: Based on each word segment and its corresponding context neighborhood, as well as the position weight of each word segment contained in each structural node, the similarity between each word segment and the first neighboring word segment in the context neighborhood corresponding to the corresponding word segment, and the similarity between each word segment and the negative neighboring word segment in the context neighborhood corresponding to the corresponding word segment are determined respectively. Based on the similarity between each word and the first neighboring word in the context of the corresponding word, and the similarity between each word and the negative neighboring word in the context of the corresponding word, the local contrast loss information corresponding to each word is determined.

6. The method according to claim 2, wherein based on each word segment and its corresponding context neighborhood, and the position weights of the words contained in each structural node, a boundary sparsity factor is determined for each word segment, including: Based on each word segment and its corresponding context neighborhood, as well as the position weight of each word segment contained in each structural node, the similarity between the negative example neighboring word in the context neighborhood corresponding to each word and the second neighboring word adjacent to the negative example neighboring word is determined. Based on the similarity between the negative neighboring words in the context neighborhood corresponding to each word and the second neighboring words adjacent to the negative neighboring words, the boundary sparsity factor corresponding to each word is determined.

7. The method according to any one of claims 1-6, further comprising: Extract the target string from each structural node. The target string is a meaningless string composed of multiple preset types of characters, including one or more of the following preset types: number, letter, and symbol. Based on the target string extracted from each structural node, determine the proportion of characters in the target string in each structural node; Based on the proportion of characters in the target string in each structural node and the expected proportion corresponding to each structural node, the density significance information corresponding to each structural node is determined. The step of determining corresponding generalizable whitelist information and / or anomalous structured text data from the multiple different structured text data based on the semantic anomaly saliency information corresponding to each word segmentation includes: Based on the semantic anomaly saliency information corresponding to each word segment and the density saliency information corresponding to each structural node, the corresponding generalizable whitelist information and / or abnormal structural text data are determined from the multiple different structural text data.

8. The method according to claim 7, wherein the preset type includes a numeric class, and determining the density saliency information corresponding to each structural node based on the proportion of characters in the target string in each structural node and the expected proportion corresponding to each structural node includes: Calculate the absolute value of the difference between the proportion of characters in the target string in each structural node and the expected proportion corresponding to each structural node to obtain the density deviation degree corresponding to each structural node; Based on the density deviation of each structural node and the preset dynamic threshold, the density saliency information of each structural node is determined.

9. The method according to claim 8, wherein the dynamic threshold is determined by the mean and standard deviation of the structural nodes in multiple different target structured text data, and a preset sensitivity factor.

10. The method according to claim 7, further comprising: Based on the path mode frequency and structural splitting gain information corresponding to each structural node, the structural saliency information corresponding to each structural node is determined. The step of determining corresponding generalizable whitelist information and / or anomalous structured text data from the multiple different structured text data based on the semantic anomaly saliency information corresponding to each word segmentation includes: Based on the semantic anomaly saliency information corresponding to each word segment and the structural saliency information corresponding to each structural node, the corresponding generalizable whitelist information and / or abnormal structural text data are determined from the multiple different structural text data.

11. The method according to claim 10, wherein determining the corresponding generalizable whitelist information and / or anomalous structured text data from the plurality of different structured text data based on the semantic anomaly saliency information corresponding to each word segment and the structural saliency information corresponding to each structural node includes: Based on the semantic anomaly saliency information corresponding to each word segment and the structural saliency information corresponding to each structural node, the fusion saliency score corresponding to each word segment is determined by a preset weight. Obtain target word segments whose difference between the fusion saliency score and a preset threshold is within a preset range, and set corresponding penalty items for the fusion saliency scores corresponding to the target word segments. Based on the fusion saliency scores corresponding to the target word segments and the corresponding penalty items, determine the saliency score corresponding to the target word segments. Based on the saliency score corresponding to the target word segment, the fusion saliency score corresponding to other word segments besides the target word segment, and the preset threshold, corresponding generalizable whitelist information and / or abnormal structured text data are determined from the multiple different structured text data.

12. The method according to claim 11, wherein the penalty term is constructed based on the anomalous prior probability corresponding to the target word segmentation and a preset penalty coefficient.

13. The method according to claim 11, wherein the threshold is constructed based on a preset base threshold, the annotation difference corresponding to the fusion saliency score for each word segmentation, and a preset sensitivity coefficient, wherein, If the structure node is related to a preset service, the sensitivity coefficient is less than a first sensitivity threshold; if the structure node is related to long-tail exploration, the sensitivity coefficient is greater than a second sensitivity threshold.

14. The method according to claim 11, wherein the structured text data includes one or more of Uniform Resource Locator (URL), Application Programming Interface (API) path, business log sequence, and rich text fragment.

15. A data processing apparatus, the apparatus comprising: The data acquisition module acquires multiple different structured text data, which are text data that can be decomposed into multiple different levels or substructures with positional dependencies. The structure navigation encoding module performs structured decomposition on each structured text data to obtain the structure nodes corresponding to each structured text data, and performs word segmentation on each structure node to obtain the words contained in each structure node. The semantic saliency determination module determines the context neighborhood of each word segment and the corresponding context neighborhood of each word segment, and determines the semantic saliency information between each word segment and the corresponding context neighborhood of each word segment, the local contrast loss information of each word segment, and the boundary sparsity factor of each word segment based on each word segment and the corresponding context neighborhood of each word segment. Based on the semantic saliency information between each word and its corresponding context neighborhood, the local contrast loss information corresponding to each word, and the boundary sparsification factor corresponding to each word, the semantic abnormality saliency information corresponding to each word is determined. The boundary sparsification factor is used to characterize the semantic consistency degree of the context neighborhood corresponding to each word. The identification module determines the corresponding generalizable whitelist information and / or abnormal structured text data from the multiple different structured text data based on the semantic anomaly saliency information corresponding to each word segmentation.

16. A data processing apparatus, the data processing apparatus comprising: processor; as well as A memory configured to store computer-executable instructions, which, when executed, cause the processor to: Acquire multiple different structured text data, wherein the structured text data is text data that can be decomposed into multiple different hierarchical or substructures with positional dependencies; Each structured text data is structurally decomposed to obtain the corresponding structure node, and each structure node is segmented to obtain the words contained in each structure node. For each word segment, determine the context neighborhood of each word and the corresponding word, and based on each word and the corresponding word's context neighborhood, determine the semantic saliency information between each word and the corresponding word's context neighborhood, the local contrast loss information of each word, and the boundary sparsity factor of each word. Based on the semantic saliency information between each word and its corresponding context neighborhood, the local contrast loss information corresponding to each word, and the boundary sparsification factor corresponding to each word, the semantic abnormality saliency information corresponding to each word is determined. The boundary sparsification factor is used to characterize the semantic consistency degree of the context neighborhood corresponding to each word. Based on the semantic anomaly saliency information corresponding to each word segmentation, the corresponding generalizable whitelist information and / or abnormal structured text data are determined from the multiple different structured text data.