Processing method, device and equipment for data standardization, medium and product

Abstract

The invention discloses a processing method and device for data standardization, equipment, a medium and a product in the technical field of computers, and the method comprises the steps: generating identification information of first data after the first data is received, so that the identification information can uniquely identify a semantic structure of the first data, when it is detected that the first strategy corresponding to the identification information does not exist, the first model is utilized to analyze the first data to obtain a second strategy, so that the second strategy can indicate a standardized processing flow of the first data; and constructing a corresponding relationship between the second strategy and the identification information, and processing the first data according to the second strategy to obtain the second data, so that a new semantic structure can be automatically found, and a standardized processing strategy can be automatically constructed for the new semantic structure. The method can better adapt to semantic structure standardization processing of most safety data with definite semantics, so that the access duration of safety equipment is greatly shortened, and better self-adaptability and high efficiency are presented.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a processing method, apparatus, device, medium, or product for data standardization. Background Technology

[0002] Cyber ​​threat attacks refer to attacks launched by attackers against target devices (terminals, servers, routers), network applications, or data resources via a network, exploiting network protocol vulnerabilities, system configuration defects, or human error. Network applications refer to software programs (or services) that rely on computer networks (LANs, the Internet, etc.) to run and enable cross-device data interaction, resource sharing, or collaborative operation.

[0003] Cyber ​​threats can occur at any stage of network applications. Therefore, to ensure the normal operation of network applications and protect them from attacks, or to detect such attacks promptly, relevant personnel (such as security personnel) can deploy corresponding security devices at each stage of the network application to detect security threats. However, because these security devices have a relatively narrow perspective, analyzing only their own specific aspects, the security analysis provided by these devices may be less effective.

[0004] Furthermore, to overcome the shortcomings mentioned above, some solutions involve using Security Information and Event Management (SIEM) to collect data generated by security devices deployed at various stages. This data is then used by relevant personnel to conduct unified data operation and analysis on SIEM, allowing for a comprehensive examination of the overall issue from multiple perspectives. However, the semantic structure of data generated by different security devices varies, making unified data operation and analysis challenging. Summary of the Invention

[0005] To address the aforementioned technical problems, this application provides a processing method, apparatus, device, medium, or product for data standardization, which maps data generated by different security devices into a unified semantic structure, thereby overcoming the defects caused by the different semantic structures of data generated by different security devices.

[0006] To achieve the above objectives, the technical solution provided in this application is as follows:

[0007] This application provides a data standardization processing method, the method comprising: in response to receiving first data, obtaining first identification information of the first data; in response to the absence of a first strategy corresponding to the first identification information, obtaining a second strategy corresponding to the first data using a first model, the second strategy indicating a data standardization processing flow corresponding to the first data; processing the first data according to the second strategy to obtain second data; wherein the second strategy and the first identification information are correspondingly stored.

[0008] In one possible implementation, obtaining the first identification information of the first data includes: extracting data type features and data structure features from the first data; and determining the identification information of the first data based on the data type features and the data structure features.

[0009] In one possible implementation, the method further includes: in response to the existence of the first strategy, processing the first data according to the first strategy to obtain a data standardization processing result of the first data.

[0010] In one possible implementation, obtaining the second strategy corresponding to the first data using the first model includes: obtaining a first semantic parsing strategy by analyzing the first data using the first model; converting the first semantic parsing strategy into the second strategy, wherein the language corresponding to the second strategy is different from the language corresponding to the first semantic parsing strategy; and storing the first semantic parsing strategy and the first identification information accordingly.

[0011] In one possible implementation, the method further includes: in response to the existence of the first strategy, analyzing the first data using a second model based on a second semantic parsing strategy corresponding to the first identification information to obtain a third semantic parsing strategy; converting the third semantic parsing strategy into a third strategy, wherein the first strategy is obtained by converting the second semantic parsing strategy; generating third data based on the standardization processing result of the first data under the first strategy and the standardization processing result of the first data under the third strategy, wherein the third data indicates the performance difference between the first strategy and the third strategy; and updating the first strategy based on the third data.

[0012] In one possible implementation, the third data indicates the performance difference between the first strategy and the third strategy under multiple fields, the multiple fields including the first field; updating the first strategy based on the third data includes: in response to the third data indicating that the performance of the third strategy under the first field is better than the performance of the first strategy under the first field, updating the first strategy based on the content in the third strategy corresponding to the first field.

[0013] In one possible implementation, the step of using a second model to analyze the first data based on a second semantic parsing strategy corresponding to the first identifier information to obtain a third semantic parsing strategy includes: in response to an indicator corresponding to the first strategy being lower than a preset threshold, using a second model to analyze the first data based on the second semantic parsing strategy corresponding to the first identifier information to obtain a third semantic parsing strategy, wherein the indicator represents the performance of the first strategy; after updating the first strategy based on the third data, the method further includes: updating the indicator corresponding to the first strategy.

[0014] In one possible implementation, generating third data based on the standardization result of the first data under the first strategy and the standardization result of the first data under the third strategy includes: using a third model to perform field-level optimization processing on the standardization result of the first data under the first strategy and the standardization result of the first data under the third strategy to obtain the third data.

[0015] In one possible implementation, the method further includes: in response to detecting an edit operation triggered for the first semantic parsing strategy, updating the first semantic parsing strategy and the second strategy, wherein the updated second strategy is determined based on the updated first semantic parsing strategy.

[0016] In one possible implementation, a preset storage space is configured to store at least one candidate identifier and a strategy corresponding to each candidate identifier; and the second strategy and the first identifier information are correspondingly stored in the preset storage space.

[0017] In one possible implementation, the first data includes security data collected from the first device, which is generated during the operation of a security application running on the first device.

[0018] In one possible implementation, the method satisfies at least one of the following constraints: the first data is semi-structured data; the second data is structured data; the semantic information indicated by the second data includes part or all of the semantic information indicated by the first data; the first identification information indicates at least one characteristic of the first data, the at least one characteristic including at least part of the data type and data structure of the first data; the first identification information includes a hash value, the hash value being determined based on the data type characteristics and data structure characteristics of the first data; and the first model serves a large model.

[0019] This application provides a processing apparatus for data standardization, comprising: a generation unit, configured to acquire first identification information of the first data in response to receiving first data; an analysis unit, configured to acquire a second strategy corresponding to the first data using a first model in response to the absence of a first strategy corresponding to the first identification information, wherein the second strategy indicates a data standardization processing flow corresponding to the first data; and a processing unit, configured to process the first data according to the second strategy to obtain second data; wherein the second strategy and the first identification information are correspondingly stored.

[0020] This application provides an electronic device, the device comprising: a processor and a memory; the memory for storing instructions or computer programs; the processor for executing the instructions or computer programs in the memory to cause the electronic device to perform a processing method for data standardization.

[0021] This application provides a computer-readable medium storing instructions or a computer program that, when executed on a device, causes the device to perform a processing method for data standardization.

[0022] This application provides a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing a processing method for data standardization.

[0023] Compared with related technologies, this application has at least the following advantages:

[0024] In the technical solution provided in this application, after receiving first data (such as security data like logs generated by a security device), the first identification information of the first data is obtained so that the first identification information can uniquely identify the semantic structure of the first data. This allows the determination that the semantic structure presented by the first data is a new semantic structure when a first strategy corresponding to the first identification information is detected. Therefore, a second strategy corresponding to the first data is first obtained using a first model so that the second strategy can indicate the data standardization processing flow corresponding to the first data. This allows the second strategy to indicate how to map security data represented according to this new semantic structure to a unified semantic structure (such as the semantic structure of standardized data). Then, the second strategy and the first identification information are correspondingly saved, and the first data is processed according to the second strategy to obtain second data, so that the second data can represent the standardization processing result of the first data. This enables automatic discovery of new semantic structures and automatic construction of standardization processing strategies for new semantic structures, better adapting to the semantic structure standardization processing of most semantically clear security data, significantly shortening the access time of security devices, and exhibiting better adaptability and high efficiency. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 A flowchart illustrating a data standardization processing method provided in an embodiment of this application;

[0027] Figure 2 A schematic diagram of a standardized processing flow provided for an embodiment of this application;

[0028] Figure 3 A schematic diagram illustrating the working principle of a structure extractor provided in an embodiment of this application;

[0029] Figure 4 A schematic diagram of a strategy generation process provided in an embodiment of this application;

[0030] Figure 5 A schematic diagram illustrating the working principle of a rule-based continuous optimizer provided in an embodiment of this application;

[0031] Figure 6 A schematic diagram of a data standardization processing apparatus provided in an embodiment of this application;

[0032] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0033] Research has revealed that, in certain scenarios, to better improve the effectiveness of security analysis, the aforementioned SIEM has the following requirements: After the SIEM collects data generated by security devices deployed at different stages of network applications, this data should first be standardized to map the data generated by different security devices to a unified semantic structure; then, unified data operation analysis should be performed on this data to obtain the security analysis results of network applications.

[0034] The research also found that, in order to meet the requirements shown above, relevant personnel can first construct a unified semantic structure as the semantic structure for the standardized data presentation; then, relevant personnel can understand the semantic structure of the security data generated in each scenario of each security device; then, relevant personnel can use rule engine technology (such as code writing technology) to manually write rules one by one to cover each scenario in each security device, so that the security data generated in each scenario of each security device can be mapped to a unified semantic structure through the rules, thereby achieving data standardization processing.

[0035] The study also found that the manually written rule scheme described above has the following drawbacks: ① This scheme places high demands on rule writers, requiring them to possess extensive security knowledge, strong R&D capabilities, and excellent coding skills; ② In scenarios with a large number of security devices, easy upgrades to security devices, or frequent changes in security device scenarios, it suffers from low integration efficiency and an inability to proactively adapt to changes. Therefore, this scheme not only places high demands on rule writers but also lacks the efficiency for rapid integration with security devices and the adaptability to timely changes in security data structures.

[0036] Based on the above research, in order to overcome the aforementioned shortcomings, this application provides a processing method for data standardization, which includes: after receiving first data (such as security data such as logs generated by a security device), obtaining first identification information of the first data, so that the first identification information can uniquely identify the semantic structure of the first data, so that when it is detected that there is no first strategy corresponding to the first identification information, it can be determined that the semantic structure presented by the first data is a new semantic structure. Therefore, a second strategy corresponding to the first data is first obtained using a first model, so that the second strategy can indicate the data standardization processing flow corresponding to the first data, thereby making the first... The second strategy indicates how to map security data represented according to this new semantic structure to a unified semantic structure (such as the semantic structure of standardized data). Then, the second strategy and the first identification information are correspondingly saved, and the first data is processed according to the second strategy to obtain second data, so that the second data can represent the standardized processing result of the first data. This enables automatic discovery of new semantic structures and automatic construction of standardized processing strategies for new semantic structures, better adapting to the semantic structure standardization processing of most semantically clear security data, significantly shortening the access time of security devices, and exhibiting better adaptability and high efficiency. Therefore, the data standardization processing method provided in this application possesses high efficiency for rapid access to security devices and high adaptability to timely responses to changes in the data structure of security data, overcoming the shortcomings caused by manually written rules.

[0037] Furthermore, this application does not limit the entity executing the data standardization processing method. For example, the method can be applied to a terminal device or a server. Alternatively, the method can be implemented through data interaction between a terminal device and a server. The terminal device can be a smartphone, computer, personal digital assistant (PDA), tablet computer, etc. The server can be a standalone server, a cluster server, or a cloud server.

[0038] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0039] To better understand the technical solution provided in this application, the data standardization processing method provided in this application will be explained below with reference to some accompanying drawings. For example... Figure 1As shown, the data standardization processing method provided in this application includes S1-S3 below.

[0040] S1: In response to receiving the first data, obtain the first identification information of the first data.

[0041] The first data refers to security data generated by any security device (such as...). Figure 2 The security data shown enables the first data to describe whether the object protected by the security device (such as a link in a network application) is subject to threat attacks, anomalies, or other problems.

[0042] It should be noted that security equipment refers to security software (or hardware) capable of protecting digital assets such as servers, software applications, and network infrastructure; moreover, this application does not limit the implementation method of the security equipment. For example, it can be implemented using any device with security protection functions (such as firewalls). Additionally, security data refers to logs, alarms, and other content generated during the operation of the security equipment, enabling the security data to indicate the data generated when using the security equipment for security protection, such as logs and alarms, thereby giving the security data some security semantics. Furthermore, security semantics refers to the content recorded in the security data that can express issues such as being threatened, attacked, or receiving alarms.

[0043] Therefore, in one possible implementation, when the data standardization processing method provided in this application is applied to a second device (such as a SIEM), the aforementioned first data may include security data collected by the second device from a first device (such as a security device). This security data is generated during the operation of a security protection application running on the first device. The first device is configured to provide security protection for a third device (such as a link) in a network application, so that the first data can represent the data generated when the first device provides security protection for the third device. Based on this, the first data may refer to data collected by the second device from a security device deployed in any link of the network application, so that the first data can represent any data (such as streaming data) collected by the second device, thereby enabling the standardization processing implemented based on the first data to describe the data standardization processing flow implemented by the second device.

[0044] Furthermore, in some scenarios, the aforementioned first data (such as log data) can be semi-structured data. This allows the standardization process for this first data to not only adjust its semantic structure but also perform structured processing, resulting in a standardized processing result (such as...). Figure 2The standardized data shown is structured data, which allows the standardization process to better describe the security semantics carried by the first data.

[0045] Furthermore, in some scenarios, the aforementioned first data may refer to data existing within the streaming data collected by the SIEM from the security device, corresponding to the current moment. Here, the current moment refers to the moment when the SIEM receives the first data.

[0046] The first identification information of the first data can uniquely identify the semantic structure presented by the first data; and this application does not limit the implementation of the first identification information. For ease of understanding, some examples are given below.

[0047] As an example, in some scenarios, the data structure of the security data generated by any security device tends to remain unchanged in a specific scenario. Therefore, based on this characteristic, the device identifier and the scenario identifier can be used to uniquely identify the data structure of the security data. Thus, in one possible implementation, the first identifier information of the first data can be determined based on the device identifier of the security device that generated the first data and the scenario identifier of the scenario in which the first data was generated.

[0048] Research has revealed that, in order to implement the solution described above, SIEM needs to know some additional information, such as the device identifier of the security device that generates the first data and the scene identifier of the scene in which the first data is generated. This not only increases the resource overhead of SIEM, but also leads to incorrect selection of standardization processing strategies when this information is missing or erroneous, thus affecting the standardization processing effect.

[0049] Based on the above research, in order to overcome the shortcomings shown in the previous paragraph, the process of generating the first identifier information of the first data may include: extracting data type features (such as...) from the first data. Figure 3 The data feature encoding shown) and data structure features (such as...) Figure 3 (The structural feature model shown); Based on the data type features and the data structure features, the first identification information of the first data is determined so that the first identification information can uniquely identify the semantic structure presented by the first data. In this way, the first identification information can be automatically generated based on the first data itself without introducing additional information, thus overcoming the defects caused by introducing additional information.

[0050] Regarding the aforementioned data type feature, this feature indicates the data type (e.g., JSON, XML, YAML, etc.) of the first data, enabling it to represent the semantic representation of the first data. Furthermore, this application does not limit the extraction method of this data type feature; for example, it can employ any method capable of extracting data type features from data, such as a pre-built machine learning model with this extraction function (e.g.,...). Figure 3 The data type extractor (as shown) or script can be used for implementation. For example, to improve accuracy, the process of determining the data type characteristics may include: first, the data type extractor runs different data type determination logics to probe the data type of the first data to obtain its data type; then, the data type of the first data is encoded to obtain its data type characteristics (such as...). Figure 3 The data feature encoding shown enables the data type feature to indicate how the first data is semantically represented.

[0051] Regarding the aforementioned data structure feature, this data structure feature can indicate the data structure presented by the first data, so that the data structure feature can represent the semantic presentation of the first data according to what structure. Furthermore, this application does not limit the extraction method of the data structure feature; for example, it can employ any method capable of extracting data structure features from data, such as a pre-built machine learning model with this extraction function (e.g., ...). Figure 3 The process can be implemented using a tree-structure recursive calculator or script. For example, to improve accuracy, the determination of the data structure features can include: first, the tree-structure recursive calculator uses a tree-structure-based recursive algorithm to recursively calculate the feature data of each key at each level, such as data key name features, data key value type features (e.g., object, array, basic type), data hierarchy structure features (e.g., nesting depth), data child node key value composition features, whether data key values ​​are null, and data hierarchy path features (e.g., the ABC path feature); then, the tree-structure recursive calculator summarizes the feature data of all keys and uses the data hierarchy structure features as sorting conditions to construct a structural feature model represented according to an array structure, which serves as the data structure feature of the first data, so that the data structure feature can describe the semantics carried by the first data in an array structure manner.

[0052] Furthermore, this application does not limit the implementation of the above-mentioned step of "determining the first identification information of the first data based on the data type characteristics and the data structure characteristics". For example, it can specifically be: concatenating the data type characteristics and the data structure characteristics to obtain the first identification information of the first data.

[0053] Furthermore, to improve query efficiency, the process for determining the first identifier information of the first data can be as follows: First, a hash value is calculated based on the data type characteristics and data structure characteristics of the first data, so that the calculated hash value can uniquely identify the combination composed of the data type characteristics and the data structure characteristics; then, based on the hash value, the first identifier information of the first data is determined, so that the first identifier information includes at least the hash value, and so that the first identifier information can describe the semantic structure of the first data with as few characters as possible, thereby making the rule retrieval process based on the first identifier information more efficient. This application does not limit the first identifier information. For example, in some scenarios, the first identifier information may only include the hash value, so that it can be directly used as a retrieval index later. Alternatively, in some scenarios, the first identifier information may include the hash value, the data type characteristics, and the data structure characteristics, so that the first identifier information can more comprehensively and accurately represent the semantic structure of the first data, so that the hash value recorded in the first identifier information can be used as a retrieval index later.

[0054] As can be seen, in one possible implementation, the first identification information of the first data may include a hash value, which is determined based on the data type characteristics and data structure characteristics of the first data, so that the first identification information can describe the semantic structure of the first data with as few characters as possible, thereby making the rule retrieval process based on the first identification information more efficient.

[0055] Based on the aforementioned first identification information, in one possible implementation, the first identification information of the first data can indicate at least one characteristic of the first data (such as semantic structure), and the at least one characteristic includes at least a part of the data type and data structure of the first data, so that the first identification information can describe as comprehensively and accurately as possible how the first data is semantically presented, thereby enabling the first identification information to uniquely identify the semantic structure of the first data.

[0056] Based on the above content of S1, it can be seen that for the SIEM, after receiving security data (such as the first data mentioned above) provided by a network device deployed in a certain link of the network application, it first extracts data type characteristics and data structure characteristics from the security data; then, based on these two characteristics, it generates identification information for the security data (such as the first identification information mentioned above or...). Figure 3The data analysis model shown enables the identification information to describe how the security data is semantically presented, thereby allowing the identification information to uniquely identify the semantic structure of the security data, so that subsequent decisions can be made based on the identification information to determine a strategy suitable for standardizing the security data.

[0057] S2: In response to the absence of a first strategy corresponding to the first identification information, a second strategy corresponding to the first data is obtained using the first model, wherein the second strategy indicates the data standardization processing flow corresponding to the first data.

[0058] The first strategy refers to a strategy generated and stored within a historical time period for the semantic structure indicated by the aforementioned first identifier information, which can represent how to standardize security data with that semantic structure. Therefore, the first strategy refers to the standardization processing strategy historically generated and stored for a semantic structure up to the current round.

[0059] The second strategy refers to a strategy, generated for the first time in the current round based on the semantic structure indicated by the aforementioned first identifier information, that can indicate how to standardize security data with that semantic structure. Therefore, the first strategy refers to a standardization processing strategy generated for the first time in the current round based on a semantic structure.

[0060] It should be noted that this application does not limit the implementation of the above-described standardization processing strategy. For example, the standardization processing strategy may include at least one calculation rule (such as by...). Figure 2 (The example shows some operational rules output by the rule converter). These operational rules describe the steps required for standardizing security data according to the requirements of the rule engine deployed in the SIEM. Furthermore, this application does not limit the representation of these operational rules; for example, they can be implemented using a key-value pair format: <security semantic vocabulary [English], code for the acquisition process of the security semantic vocabulary>.

[0061] The first model refers to the machine learning model used when generating a standardized processing strategy for a new semantic structure for the first time (e.g., ...). Figure 2 Or the large model service shown in 4); and this application does not limit the implementation of the first model. For example, it can be implemented using any method that can automatically generate standardized processing strategies based on data, such as a pre-built machine learning model with the generation function (such as a large model service).

[0062] It is evident that, in one possible implementation, the first model can serve a larger model (e.g., Figure 2Alternatively, as shown in Figure 4, the large model service can be used to enable the first model to have at least the ability to semantically understand unknown content (such as security data that presents new semantic structures) and the ability to write fixed rules. This will enable the standardized processing strategy for automatically generating any security data based on the first model, so as to better solve the problems of low efficiency and difficulty in adaptability of manually written rules.

[0063] Based on the above content related to S2, it can be seen that for a SIEM, the first identifier information (such as...) is obtained when the SIEM acquires the first data. Figure 3 Following the data analysis model shown, if the SIEM detects the absence of a first strategy corresponding to the first identifier information, it indicates that the SIEM has not stored the first strategy. This confirms that the SIEM has not yet processed security data with a semantic structure indicated by the first identifier information, and further confirms the discovery of a new semantic structure. Therefore, based on the first data and pre-defined prompts, the SIEM invokes a large model service to automatically generate a standardized processing strategy for the first data under the guidance of the prompts. This results in a second strategy that indicates the standardized processing flow of the first data, thus demonstrating how to map some or all of the semantic information (such as security semantics) recorded in the first data to a unified semantic structure. The prompts refer to pre-defined content that describes the constraints required to automatically generate the standardized processing strategy, and these prompts can be determined based on the standardized processing strategy generation requirements of the actual application scenario.

[0064] S3: The first data is processed according to the second strategy to obtain the second data, and the second strategy and the first identification information are saved accordingly.

[0065] The second data refers to the result obtained by standardizing the first data (e.g., Figure 2 The standardized data shown enables the second data to represent some semantic information (such as security semantics) carried by the first data according to the target structure (such as the "unified semantic structure" mentioned above).

[0066] Therefore, in one possible implementation, the second data can be structured data, so that the second data can describe some semantic information (such as security semantics) carried by the first data according to a pre-set unified semantic structure, so that unified operation analysis of security data from various sources can be carried out based on the second data.

[0067] Based on the above two paragraphs, it can be seen that, in one possible implementation, the semantic information indicated by the second data may include part or all of the semantic information indicated by the first data (such as security semantics), so that the semantic information indicated by the second data can represent the semantic information (such as security semantics) carried by the first data that has security analysis value. This is beneficial to overcome the defects caused by standardizing the semantic information carried by the first data that has no security analysis value (such as significantly increasing resource consumption), so as to better reduce resource (such as computing resources and time resources) overhead.

[0068] Furthermore, this application does not limit the relationship between the execution time of the step "the second strategy and the first identification information are saved accordingly" in S3 and the execution time of the step "processing the first data according to the second strategy to obtain the second data" in S3. For example, the former may precede the latter. Or, the two may be the same. Or, the latter may precede the former.

[0069] As can be seen, in one possible implementation, for the SIEM, after the SIEM automatically generates the second strategy based on the first data, the SIEM constructs and stores the correspondence between the second strategy and the first identification information of the first data, so that the second strategy can be quickly retrieved based on the first identification information; at the same time, the SIEM performs standardization processing on the first data according to the second strategy to obtain the second data.

[0070] Furthermore, this application does not limit the implementation of the step "processing the first data according to the second strategy to obtain the second data" in S3 above. For example, it can be implemented using any method that can map the first data to a unified semantic architecture according to the strategy, such as a machine learning model or script with the mapping function.

[0071] Furthermore, to further reduce the workload of SIEM development and adjustments, the existing rule engine within the SIEM, capable of mapping according to manually written rules, can continue to be used for standardized processing. Based on this, the aforementioned second strategy meets the execution requirements of the rule engine (such as the requirement to use executable code for semantic description); and S3 may include: inputting the second strategy and the first data into the rule engine (e.g., ... Figure 2 The rule engine shown is used to obtain the second data output by the rule engine. The rule engine is configured to standardize the first data according to the second strategy, so as to combine the advantages of the large model (such as semantic understanding) with the advantages of the rule engine to better improve the data standardization effect.

[0072] Therefore, in one possible implementation, the second strategy mentioned above may refer to a rule expressed in machine language (such as the language used by the code that the rule engine can execute), so that the second strategy can be better executed by the rule engine with machine language recognition function, thereby overcoming the defects caused by the difference between the language used by the second strategy and the language that the rule engine can recognize.

[0073] Furthermore, this application does not limit the implementation of the rule engine described above. For example, the working principle of the rule engine may include: after the rule engine receives a set of operation rules represented in key-value pair format (such as all operation rules recorded in the second strategy) and raw data (such as the first data), it iterates through each operation rule, uses the value recorded in each operation rule (such as the value of "code of the security semantic vocabulary acquisition process") as an operator, processes the raw data (such as regular expression operation, JSON extraction operation, etc.) to obtain the processing result, and constructs a new key-value pair corresponding to the operation rule based on the processing result and the key recorded in the operation rule (such as the key-value pair <security semantic vocabulary [English], processing result>), so that after all operation rules are executed, the new key-value pairs corresponding to all operation rules are integrated according to a unified semantic structure to obtain standardized data (such as the second data mentioned above).

[0074] Based on the above-mentioned content of S1 to S3, the data standardization scheme provided by this application includes: after receiving first data (such as security data such as logs generated by security devices), obtaining the first identification information of the first data so that the first identification information can uniquely identify the semantic structure of the first data, so that when it is detected that there is no first strategy corresponding to the first identification information, it can be determined that the semantic structure presented by the first data is a new semantic structure. Therefore, the first model is first used to obtain the second strategy corresponding to the first data so that the second strategy can indicate the data standardization processing flow corresponding to the first data, thereby enabling the second strategy to indicate how to map the security data represented according to this new semantic structure to a unified semantic structure (such as the semantic structure of standardized data); then, the second strategy and the first identification information are saved accordingly, and the first data is processed according to the second strategy to obtain the second data so that the second data can represent the standardization processing result of the first data. In this way, it is possible to automatically discover new semantic structures and automatically construct standardization processing strategies for new semantic structures, so as to better adapt to the semantic structure standardization processing of most semantically clear security data, so as to significantly shorten the access time of security devices and present better adaptability and high efficiency. It is evident that the data standardization scheme provided in this application possesses high efficiency in quickly accessing security devices and high adaptability in responding to changes in the data structure of security data in a timely manner, thereby overcoming the defects caused by manually written rules.

[0075] Research has found that in some scenarios (such as when the rule engine can execute standardized processing strategies expressed in machine language), users may need to view and adjust the rules. This makes it difficult for users to quickly understand and adjust the rules when they view rules expressed in machine language (such as operation rules that can be executed by the rule engine), thus affecting the user experience.

[0076] The study also found that some machine learning models (such as large model services) exhibit slightly lower generative performance in machine language than in other languages ​​(such as languages ​​that are easy for users to understand), which makes it easy for errors to occur when using the model directly to generate standardized processing strategies represented in machine language.

[0077] Based on the above research, in order to overcome the problems shown in the above two paragraphs, the process of determining the second strategy may include: using the first model to analyze the first data to obtain the first semantic parsing strategy (such as...). Figure 4 The first semantic parsing strategy is designed to indicate part or all of the data standardization process corresponding to the first data, such as the extraction process of at least one content (e.g., security semantics) in the first data. This allows the first semantic parsing rule to describe how to standardize the first data in a language that is easier for users or large model services to understand, thereby reducing the difficulty of strategy generation. The first semantic parsing strategy is then converted into a second strategy. The language corresponding to the second strategy (e.g., the machine language used to write code) is different from the language corresponding to the first semantic parsing strategy (e.g., the language that large model services can understand and generate). This overcomes the shortcomings caused by the first model's inability to directly generate executable code, thereby improving accuracy.

[0078] For the first model shown above, it is configured to follow the prompts (such as semantic structure definition, data acquisition rule writing definition, etc.) and related reference knowledge (such as...) Figure 4 The common security semantic fields shown in the knowledge base, etc., are used for security data (such as the first data mentioned above or...). Figure 4The original data shown is parsed to obtain a semantic parsing strategy expressed in a language more easily understood by users or large model services. This semantic parsing strategy includes semantic parsing rules, thereby making the semantic parsing rules recorded in the strategy more readable than the operational rules executable by a rule engine. The semantic parsing rules are configured to describe at least a method for obtaining some security semantics recorded in the security data. Furthermore, this application does not limit the implementation of the prompt word; for example, the prompt word may include at least some of the following: role definition, extraction rules, extraction process, and output format expectations. The role definition is configured to specify that the first model acts as a professional with advanced security and programming knowledge. The extraction process is configured to describe the process constraints that need to be satisfied when extracting semantic parsing rules, such as from a known set of security semantic terms (e.g., ...). Figure 4 The process iterates through the common security semantic fields knowledge base (as shown) to find values ​​that match the semantics (such as security semantics) indicated by the security semantic term, and generates extraction rules and other constraints for these values. This allows the extraction process to indicate how to extract semantic parsing rules from the security data. The extraction rules are configured to describe the constraints that the final extracted semantic parsing rules satisfy, such as using regular expressions or JSON paths to represent values ​​that match the semantics. The output format is expected to be configured to describe the format satisfied by the final extracted semantic parsing rules, such as describing the security semantic term [Chinese] and the extraction rules for that term in a specific format. It should be noted that the "related reference knowledge" indicates the semantic content (such as security semantics) that needs to be considered in the security data when using the first model for semantic parsing rule extraction. This allows for the filtering of semantic content in the security data using the "related security knowledge," avoiding interference from content other than security semantics.

[0079] Therefore, in one possible implementation, the first semantic parsing rule automatically generated based on the aforementioned first data can be implemented using a key-value pair format: <security semantic vocabulary [Chinese], extraction rule describing the acquisition process of the security semantic vocabulary>. It should be noted that this application does not limit the implementation method of the extraction rule. For example, in some scenarios, the extraction rule can use a user-understandable language to describe the acquisition process of the security semantic vocabulary. Alternatively, the extraction rule can directly use code executable by the rule engine to describe the acquisition process of the security semantic vocabulary.

[0080] Furthermore, this application does not limit the implementation method of the step of "converting the first semantic parsing strategy into a second strategy". For example, it can be implemented using any method capable of converting the first semantic parsing strategy into a second strategy that meets the execution requirements of the rule engine, such as a pre-built machine learning model or script with such conversion function. Alternatively, this step can be implemented using... Figure 2 The rule converter (such as a machine learning model, script, or hardware) shown is implemented. This rule converter is used to convert the first semantic parsing rule into an operation rule that can be executed by the rule engine. Furthermore, this application does not limit the implementation method of the rule converter. For example, the working principle of the rule converter may include: after receiving an array containing multiple semantic parsing rules (such as the semantic parsing strategy described above), iterating through each semantic parsing rule in the array, mapping the security semantic term [Chinese] recorded in the semantic parsing rule to a built-in storage key [English] table to obtain the storage key corresponding to the security semantic term [Chinese], such as the storage key "security semantic term [English]"; determining the value corresponding to the storage key based on the extraction rule recorded in the semantic parsing rule to obtain a set of operation rules that meet the execution requirements of the rule engine, such as the key-value pair <security semantic term [English], code of the security semantic term acquisition process>.

[0081] Therefore, in one possible implementation, the second strategy described above can at least satisfy the following constraints: the language corresponding to the second strategy is different from the language corresponding to the first semantic parsing strategy; the semantic information indicated by the second strategy is consistent with the semantic information indicated by the first semantic parsing strategy. Here, the language corresponding to the second strategy refers to the language used when the second strategy expresses its semantics (e.g., the language used by the code executable by the rule engine). The language corresponding to the first semantic parsing strategy refers to the language used when the first semantic parsing strategy expresses its semantics (e.g., a language that is easier for large model services to understand and generate). The language corresponding to the first semantic parsing strategy is easier for large models (or users) to understand than the language corresponding to the second strategy. The language corresponding to the second strategy is easier for the rule engine to understand than the language corresponding to the first semantic parsing strategy.

[0082] Based on the aforementioned second strategy, it is clear that for a SIEM, upon receiving the first data (such as security data or...) Figure 4 After the original data shown, the SIEM first calls the large model service based on pre-set prompts (such as the prompts corresponding to the first model) and relevant reference knowledge, carrying the first data. This allows the large model service to parse the first data under the guidance of the prompts and relevant reference knowledge to obtain the model output data (such as...). Figure 4The generated data shown is used to enable the model output data to describe how to obtain the security semantics in the first data. Then, the SIEM checks whether the output data conforms to the format requirements indicated by the prompt word (such as the requirements indicated by "output format expectation"). If it does not conform to the format requirements, it indicates that the large model service has made a mistake, and the output data can be discarded. However, if it conforms to the format requirements, the output data is used as a semantic parsing rule (such as the semantic parsing strategy mentioned above). The SIEM then converts the semantic parsing rule that has passed the check into an operation rule that conforms to the execution requirements of the rule engine to obtain the second strategy, so as to realize the automatic generation of standardized processing strategies for new semantic structures.

[0083] In addition, to further improve the accuracy of the rules, the above-mentioned processing method for data standardization may also include the following steps: in response to detecting an editing operation triggered by the first semantic parsing strategy, updating the first semantic parsing strategy and the second strategy, wherein the updated second strategy is determined based on the updated first semantic parsing strategy. This allows for fine-tuning of automatically generated strategies (such as semantic parsing strategies, standardization processing strategies, etc.) through human-computer interaction, so as to ensure that the final strategy is more accurate.

[0084] As can be seen, for SIEM, after automatically generating a first semantic parsing strategy based on the first data, the first semantic parsing strategy can be displayed to users (such as security personnel). When an editing operation (such as rule modification, rule deletion, rule addition, etc.) is triggered by the user on the first semantic parsing strategy, the first semantic parsing strategy is updated according to the editing operation. This ensures that the updated first semantic parsing strategy meets the rule modification requirements indicated by the editing operation, making the updated first semantic parsing strategy more in line with the parsing requirements than the original first semantic parsing strategy. At this point, the updated first semantic parsing strategy can be converted into operation rules that meet the execution requirements of the rule engine to obtain the updated second strategy. The updated second strategy includes these operation rules, enabling more refined fine-tuning and updating of both the existing first semantic parsing strategy and the second strategy simultaneously with the editing operation, ensuring that the semantics indicated by the updated second strategy are consistent with the semantics indicated by the updated first semantic parsing strategy.

[0085] Furthermore, to better improve the amount of knowledge available for subsequent strategy optimization, the above-mentioned data standardization processing method may include at least some or all of the following steps: constructing and storing the correspondence between the second strategy and the first identification information, constructing and storing the correspondence between the first semantic parsing strategy and the first identification information, and constructing and storing the correspondence between the first data and the first identification information. This is so that the SIEM not only records the operation rule that can be directly executed by the rule engine corresponding to a semantic structure, but also records other information corresponding to the semantic structure, such as the first semantic parsing strategy that is easy for users to understand, the security data on which the first semantic parsing strategy is based, etc., so that the information recorded in the SIEM can record some characteristics of the operation rule as comprehensively as possible, so that subsequent iterative optimization of the operation rule can be achieved based on these characteristics.

[0086] Based on the above two paragraphs, it can be seen that, in one possible implementation, in order to better improve the information preservation effect, the above-mentioned data standardization processing method may include at least some or all of the following steps: in a preset storage space (such as...) Figure 2 The system constructs and stores the correspondence between the second strategy and the first identification information in the information saver shown; constructs and stores the correspondence between the first semantic parsing strategy and the first identification information in the preset storage space; and constructs and stores the correspondence between the first data and the first identification information in the preset storage space. This enables the persistent storage of current data content (such as the first data), the first identification information, the first semantic parsing strategy, and the second strategy in the preset storage space. This allows these contents to be used as rules and reference knowledge bases for standardizing security data with the same semantic structure, thereby improving information comprehensiveness. In one possible implementation, the second strategy and the first identification information are correspondingly stored in the preset storage space.

[0087] It should be noted that this application does not limit the implementation method of the aforementioned preset storage space. For example, it can be implemented using any device with storage function (such as...). Figure 2The application implements the information storage device shown. This information storage device is configured to persistently store some information. Furthermore, this application does not limit the storage method used by the information storage device. For example, when the first identification information includes a hash value, the information storage device is configured to store some information using the hash value as an index, such as first data, the data type characteristics of the first data, the data structure characteristics of the first data, a first semantic parsing rule generated based on the first data (such as the aforementioned semantic parsing strategy), and an operation rule (such as a second strategy) derived from the first semantic parsing rule. This ensures that the information storage device records the mapping relationship between this information and the hash value, so that this information can be quickly retrieved from the information storage device based on the hash value.

[0088] In another possible implementation, the above-described processing method for data standardization may further include the following steps: in response to the existence of a first strategy corresponding to the first identification information of the first data, it can be determined that the first strategy refers to a standardization processing strategy that has been generated for the semantic structure indicated by the first identification information within a historical time period. Thus, it can be determined that the first strategy can map the first data to a unified semantic structure. Therefore, the first data can be processed according to the first strategy to obtain the data standardization processing result of the first data. This enables automatic identification of the semantic structure of security data and automatic retrieval of a standardization processing strategy that matches the semantic structure, thereby improving efficiency.

[0089] As can be seen, for a SIEM, after the SIEM obtains the first identification information of the first data, if it detects that there is a first policy in the SIEM corresponding to the first identification information, it can be determined that the SIEM has not encountered the semantic structure indicated by the first identification information for the first time. Therefore, the rule engine deployed in the SIEM can directly use the first policy to standardize the first data to improve efficiency.

[0090] In addition, to further improve efficiency, when the aforementioned preset storage space is configured to include at least one candidate identifier (such as a hash value as an index) and a strategy (such as semantic parsing rules, operation rules, etc.) corresponding to each candidate identifier, the aforementioned processing method for data standardization may further include the following steps: comparing the first identifier information of the first data with each candidate identifier to obtain a comparison result, wherein the comparison result indicates whether there is a first strategy corresponding to the first identifier information in the preset storage space, so that the acquisition method of the standardization processing strategy of the first data can be determined based on the comparison result, so as to better realize the standardization processing strategy of automatically discovering new semantic structures and reusing existing semantic structures.

[0091] As can be seen, for a SIEM, after obtaining the first identifier information of the first data, the SIEM uses the hash value recorded in the first identifier information as a search term to search within a preset storage space to determine whether there is an index in the preset storage space that is equal to the hash value. If it exists, it can be determined that the SIEM has not encountered the semantic structure indicated by the first identifier information for the first time. Therefore, the SIEM can directly use the strategy pointed to by the "index that is equal to the hash value" (such as running rules that can be executed by the rule engine) to standardize the first data, thereby improving efficiency.

[0092] Research has shown that in some scenarios, standardized processing strategies may not be generated completely in one go. Therefore, to solve this problem, the generated strategies can be optimized through multiple iterations to achieve the effect of optimizing the strategies while using them.

[0093] Based on the above two paragraphs, it can be seen that, in one possible implementation, the above-mentioned processing method for data standardization may further include the following steps: responding to the existence of a first strategy (such as based on) the existence of a first identification information corresponding to the first data. Figure 5 The operational rules obtained by transforming the old semantic parsing rules shown), utilize the second model based on the second semantic parsing strategy corresponding to the first identifier information (such as...). Figure 5 The old semantic parsing rules shown are used to analyze the first data to obtain the third semantic parsing rule (such as based on the old semantic parsing rules). Figure 5 The new semantic parsing rules shown convert the third semantic parsing strategy into a third strategy (such as based on...). Figure 5 The operational rules derived from the new semantic parsing rules shown are as follows: the first strategy is derived from the second semantic parsing strategy; based on the standardized processing results of the first data under the first strategy (such as...). Figure 5 The standard data 2 shown and the standardization results of the first data under the third strategy (e.g.) Figure 5 The standard data shown in Figure 1) generates third data (such as...). Figure 5 The standard data shown in Figure 3 indicates the performance difference between the first strategy and the third strategy. The first strategy is updated based on the third data so that the performance of the updated first strategy is better than that of the original first strategy. In this way, the performance of the strategy can be continuously improved by continuously optimizing the generated strategy, so as to achieve the self-evolution of the strategy.

[0094] For the aforementioned second model, this second model refers to the machine learning model used when generating a standardized processing strategy for a semantic structure that is not being implemented for the first time. Furthermore, this application does not limit the implementation method of the second model; for example, the second model can be implemented using a large model service. Additionally, the second model is configured to: parse the current data content (such as the first data) according to the prompt words corresponding to the second model (such as semantic structure definitions, data acquisition rule writing definitions, semantic parsing rules historically generated for the first identifier information, etc.) and the relevant reference knowledge corresponding to the second model (such as a conventional security semantic field knowledge base, and historically stored content for the first identifier information) to obtain a new semantic parsing strategy, so that the new semantic parsing strategy can be subsequently converted into a new standardized processing strategy (such as the aforementioned third strategy). The prompt words corresponding to the second model include additional historical information compared to the prompt words corresponding to the first model, such as information about semantic parsing rules (such as the aforementioned second semantic parsing strategy) already generated for the semantic structure indicated by the first identifier information. Compared to the reference knowledge corresponding to the first model, the second model has an additional historical knowledge base. This historical knowledge base records information that has been obtained for the semantic structure within a historical time period, such as the second semantic parsing rules, operation rules (such as the first strategy), and original data (such as the security data used when generating the second semantic parsing rules).

[0095] Therefore, in one possible implementation, the process of determining the second semantic parsing strategy can be as follows: using the second model to analyze the first data based on at least one item of content corresponding to the first identifier information (such as the second semantic parsing strategy mentioned above) to obtain the third semantic parsing rule.

[0096] For the aforementioned "at least one item of content corresponding to the first identification information", the at least one item of content refers to the content that the second model needs to rely on when generating semantic parsing rules and is bound to the first identification information; moreover, this application does not limit the implementation of the at least one item of content. For example, the at least one item of content may include: part or all of the content that corresponds to the first identification information and is recorded in a preset storage space, such as semantic parsing rules, operation rules, raw data, etc., so that these contents can better describe the preconditions for the generation of the new strategy.

[0097] The third data mentioned above is obtained by comparing the standardized processing result of the first data under the first strategy with the standardized processing result of the first data under the third strategy, so that the third data can indicate the performance difference between the first strategy and the third strategy; and this application does not limit the implementation of the third data. For example, it can be implemented using any data that can describe the performance difference between different strategies.

[0098] Research has revealed that security data may contain multiple security semantics, causing it to be mapped to multiple fields after standardization. Therefore, in one possible implementation, the aforementioned third data can indicate the performance differences between the first and third strategies across multiple fields. This allows the third data to highlight the field-level differences between different strategies, enabling subsequent field-level optimization based on this third data to better improve strategy optimization results.

[0099] Furthermore, this application does not limit the implementation method of the aforementioned third data. For example, when the standardized processing result corresponding to the first data under the first strategy is {the standardized processing result corresponding to field 1}, old The standardized processing result corresponding to field 2 old ..., the standardized processing result corresponding to field M old Furthermore, the standardized processing result of the first data under the third strategy is {the standardized processing result corresponding to field 1}. new The standardized processing result corresponding to field 2 new ..., the standardized processing result corresponding to field M new When the third data is determined, the process is as follows: if the standardized processing result corresponding to field i is detected... new Better than the standardized result corresponding to field i old Then, based on the standardized processing result corresponding to field i... new Determine a third set of data, such that the third set of data includes the standardized processing result corresponding to field i. new If the standardized result corresponding to field i is detected... new Not better than the standardized result corresponding to field i old Then, based on the standardized processing result corresponding to field i... old Determine a third set of data, such that the third set of data includes the standardized processing result corresponding to field i. old Where i is a positive integer, i≤M, and M is a positive integer, so that the third data records the optimal results of different strategies in each field, thereby enabling the third data to describe the field-level performance differences of different strategies.

[0100] Therefore, in one possible implementation, the aforementioned third data may include the standardized processing results of the first data under multiple fields. The standardized processing result for each field recorded in the third data may originate from the standardized processing result of the first data under the first strategy, or it may originate from the standardized processing result of the first data under the third strategy, so that the standardized processing result for each field recorded in the third data is not inferior to the standardized processing result for the corresponding field recorded in the "standardized processing result of the first data under the first strategy," nor is it inferior to the standardized processing result for the corresponding field recorded in the "standardized processing result of the first data under the third strategy."

[0101] Furthermore, this application does not limit the method of obtaining the aforementioned third data. For example, it can specifically be: using a third model to perform field-level optimization processing on the standardized processing results of the first data under the first strategy and the standardized processing results of the first data under the third strategy to obtain the third data, so that the third data can describe the field-level performance differences presented by different strategies, thereby enabling the third data to indicate which field-level rules recorded in different strategies are better, and thus making the strategy obtained based on the third data better.

[0102] It is evident that the aforementioned third data can be obtained using a third model (such as...). Figure 5 The large model service shown) generates the third model, and the third model is configured to be based on the prompt words corresponding to the third model, the relevant reference knowledge corresponding to the third model, and the first data (such as...). Figure 5 The original data shown is used to perform field-level optimization processing on the standardized results of the first data under the first strategy and the standardized results of the first data under the third strategy to obtain better standardized data (such as the third data or...). Figure 5 The standard data shown is 3). The prompts for this third model include a role definition, selection rules, and output format expectations. The role definition is configured to specify that the third model acts as a professional with extensive security and programming knowledge. The selection rules are configured to describe how field-level selection is achieved, such as comparing each field of two standardized processing results one by one to obtain the optimal result for that field. The output format expectations are configured to describe the format of the third model's output data, such as requiring the reassembly of the optimal results for each field into a new standardized data output. Furthermore, this application does not limit the implementation method of the relevant reference knowledge corresponding to this third model; for example, it can be determined based on the standardization processing requirements of the actual application scenario.

[0103] Furthermore, this application does not limit the implementation of the above-mentioned "updating the first strategy based on the third data" step. For example, when the third data indicates the performance difference between the first strategy and the third strategy under multiple fields, and the multiple fields include the first field, this step may specifically include: in response to the third data indicating that the performance of the third strategy under the first field is better than that of the first strategy under the first field, updating the first strategy based on the content (such as operation rules) in the third strategy corresponding to the first field, so as to realize that the content in the third strategy corresponding to the first field can be directly replaced by the content in the first strategy corresponding to the first field, so as to ensure that the updated first strategy can present the optimal operation rules corresponding to these fields, so as to better improve the strategy optimization effect.

[0104] Research has found that the semantic structure of security data generated by security devices in specific scenarios is usually unchanging, so that the standardized strategy corresponding to the semantic structure can be generated through a limited number of iterations. Therefore, in order to better save resources, a pre-set indicator (such as maturity) can be used to decide whether the strategy needs to be further optimized, so as to avoid resource waste caused by too many optimizations of the strategy.

[0105] Based on the above research, in one possible implementation, the data standardization processing method may further include the following steps: initializing an indicator corresponding to the second strategy, whereby the indicator represents the performance of the second strategy, so that the indicator can represent the maturity of the second strategy, thereby enabling the indicator to indicate to some extent whether the second strategy is complete, and further enabling the indicator to indicate to some extent whether the second strategy needs to be further optimized and updated. This allows for the generation of some strategies for a new semantic structure, followed by marking the indicators corresponding to these strategies with initial values ​​(e.g., 1), so that subsequent decisions on whether to iteratively optimize these strategies can be made based on the indicator. This application does not limit the initial value of the indicator; for example, the initial value of the indicator can be 1.

[0106] Therefore, in one possible implementation, the above-described data standardization processing method may further include the following steps: in response to the existence of a first strategy corresponding to the first identification information of the first data, determining whether the indicator corresponding to the first strategy is lower than a preset threshold (e.g., ...). Figure 2N (where N is a positive integer) represents the performance of the first strategy. In response to the first strategy's corresponding indicator falling below a preset threshold, it can be determined that the first strategy still needs optimization. Therefore, the second model, based on the second semantic parsing strategy corresponding to the first identifier information, analyzes the first data to obtain a third semantic parsing strategy, and converts this third semantic parsing strategy into a third strategy. Based on the standardized processing results of the first data under the first strategy and the standardized processing results of the first data under the third strategy, third data is generated so that it can indicate the field-level performance differences between the first and second strategies. The first strategy is updated based on the third data, and the corresponding indicator is also updated so that the strategy performance represented by the updated indicator is higher than the strategy performance represented by the updated indicator. This achieves automatic updating of corresponding indicators (such as maturity) as the strategy iterates, enabling the indicator to characterize the strategy's performance in real time, so that subsequent decisions can be made more accurately based on the indicator to determine whether to iteratively optimize these strategies.

[0107] It should be noted that this application does not limit the implementation of the above-mentioned "updating the indicator corresponding to the first strategy" step. For example, it can satisfy the following constraint: the updated indicator is increased by 1 compared with the indicator before the update, so that the number of strategy updates represented by the updated indicator is higher than the number of strategy updates represented by the indicator before the update.

[0108] As can be seen, for a SIEM, after generating the first identifier information of the first data, if the SIEM finds the first strategy corresponding to the first identifier information based on the hash value recorded in the first identifier information, it can not only use the first strategy to achieve standardization processing of the first data, but also, when it is determined that the indicator corresponding to the first strategy is lower than a preset threshold, first use the large model service to analyze the first data to obtain a new semantic parsing strategy (such as a third semantic parsing strategy); then convert the new semantic parsing strategy into the third strategy, so that a better semantic parsing strategy and a better standardization processing strategy that can be executed by the rule engine (such as the updated first strategy) can be obtained by selecting the field-level standardization processing results of the first data under the first strategy and the standardization processing results of the first data under the third strategy. At this time, it can be based on Based on a better semantic parsing strategy (such as some semantic parsing rules), a better standardization processing strategy that can be executed by the rule engine (such as some operation rules), and the first data, the content recorded in the preset storage space corresponding to the hash value is updated to ensure that the preset storage space stores the mapping relationship between the hash value and these latest contents (such as a better semantic parsing strategy, a better standardization processing strategy that can be executed by the rule engine, and the first data, etc.). This ensures that the preset storage space at least records the latest version of the standardization processing strategy corresponding to the hash value and its related information, so that these contents can be used as rules and reference knowledge bases for standardizing security data in the future, thereby improving the comprehensiveness of information. At this time, the index corresponding to the standardization processing strategy can also be incremented by 1 to indicate that the standardization processing strategy has undergone another round of optimization.

[0109] Based on the above-mentioned processing methods for data standardization, it can be seen that for a SIEM that implements security data standardization according to these processing methods, the SIEM exhibits the characteristics shown in (I) to (VI) below. Among them, security data standardization refers to the process of converting security logs and event data (such as firewall logs, alarms, endpoint detection data, etc.) from different security devices and with different semantic formats into a unified, standardized format.

[0110] (i) After the SIEM receives security data (such as the first data mentioned above) generated by any security device and input in a streaming manner, the structure extractor deployed in the SIEM (such as...) Figure 2 or Figure 3The structure extractor shown performs semantic structure extraction processing on the security data to obtain the identification information of the security data. This identification information is designed to uniquely and completely identify the semantic structure of the security data, so that it can be used as the sole representative of this semantic structure in subsequent processing. The structure extractor is configured to perform... Figure 3 The semantic structure extraction process is shown below. It can be seen that after the structure extractor receives the secure data, it first performs feature extraction calculations from both the data type and data structure dimensions to obtain data type features and data structure features. Then, the structure extractor integrates these two features (such as hash value calculation and concatenation) to obtain the identification information of the secure data, so that this identification information can uniquely identify the semantic structure presented by the secure data.

[0111] (ii) After the structure extractor deployed in the SIEM generates identification information based on security data, the structure extractor sends the security data along with the identification information to the processing distributor deployed in the SIEM (e.g., Figure 2 The processing distributor (shown) enables the processing distributor to use the hash value recorded in the identification information as a search term to search for a large amount of information recorded in the information store deployed in the SIEM, in order to determine whether there is an index in the information store that is equal to the hash value, so that the processing distributor can flexibly decide on the distribution strategy to be used in the future based on the search results.

[0112] (iii) If the processing distributor deployed in the SIEM determines through retrieval that there is no index equal to the hash value, it can be determined that the information store deployed in the SIEM does not store a policy capable of standardizing the current security data. Therefore, the processing distributor can send the security data to the rule generator deployed in the SIEM (such as...). Figure 2 or Figure 4 The rule generator shown enables it to utilize predetermined prompts (such as the prompts corresponding to the first model mentioned above) and relevant reference knowledge to call the large model service to parse the security data, thereby obtaining some semantic parsing rules (such as...). Figure 4 The generated data shown is sent to the rule inspector deployed in the SIEM (such as...). Figure 2 or Figure 4 The rule checker shown then examines each semantic parsing rule to remove those that do not meet the established requirements (such as the expected output format indicated by the prompt word corresponding to the first model). This allows the rule checker to then send the successfully checked semantic parsing rules to the rule converter deployed in the SIEM (e.g., ...). Figure 2The rule converter shown enables the passed semantic parsing rules to be transformed into operational rules (such as the second strategy) that meet the execution requirements of the rule engine deployed in the SIEM, according to a predetermined transformation logic. Subsequently, the rule converter sends these operational rules, the security data, the security data's identification information, and the passed semantic parsing rules to the information storage device deployed in the SIEM for storage. The corresponding metrics (such as maturity) of these stored contents are set to 1, and the method of executing these operational rules via the rule engine is used to map the security data to a unified semantic structure, thereby obtaining the standardized processing result corresponding to the security data (such as...). Figure 2 (Standardized data shown).

[0113] (iv) If the processing distributor deployed in the SIEM determines through retrieval that an index equal to the hash value exists, it can be determined that the information store deployed in the SIEM stores a strategy capable of standardizing the current security data (such as the first strategy mentioned above). Therefore, the processing distributor can send the security data to the rule extractor deployed in the SIEM (such as...). Figure 2 The rule extractor shown enables it to read relevant content indexed by the hash value from the information store, such as operational rules and metrics. When the rule extractor detects that the metric (e.g., maturity) corresponding to the hash value is not lower than a preset threshold, it can determine that the rule corresponding to the hash value largely meets the relevant requirements. Therefore, the rule engine deployed in the SIEM executes these operational rules to map the security data to a unified semantic structure, thereby obtaining standardized processing results (e.g., ...) for the security data. Figure 2 The standardized data shown); however, when the rule extractor detects that the metric corresponding to the hash value is lower than a preset threshold, it can be determined that the performance of the rule corresponding to the hash value still needs improvement. Therefore, the rule extractor not only sends the security data and the operation rules corresponding to the hash value to the rule engine deployed in the SIEM, so that the security data can be mapped to a unified semantic structure to obtain the standardized processing result corresponding to the security data by executing these operation rules through the rule engine; the rule extractor also sends the security data and the content corresponding to the hash value recorded in the information saver to the rule continuous optimizer deployed in the SIEM (such as Figure 2 or Figure 5The rule-based continuous optimizer (shown) enables it to utilize predetermined prompts (such as the prompts corresponding to the second model mentioned above) and relevant reference knowledge to call the large model service to parse the security data, obtain new semantic parsing rules, and then compare these new semantic parsing rules with the semantic parsing rules corresponding to the hash value recorded in the information saver to obtain better semantic parsing rules (such as...). Figure 5 The improved semantic parsing rules are shown so that the information saver can subsequently store the current data content (such as security data), the improved semantic parsing rules, the operation rules transformed based on the improved semantic parsing rules, and other content, and increase the corresponding indicators (such as maturity) of these stored contents by 1.

[0114] (v) In some scenarios, relevant personnel (such as...) Figure 2 Users can view and fine-tune the rules (such as semantic parsing rules) recorded in the information store deployed in SIEM according to their actual needs, so as to obtain a better standardized processing strategy.

[0115] (vi) For the rule continuous optimizer deployed in SIEM (e.g.) Figure 5 For the rule-based continuous optimizer shown, the working principle of the rule-based continuous optimizer can include: when the rule-based continuous optimizer receives the original data (such as the first data) and the already generated old semantic parsing strategy 1 (such as the previous one) which includes multiple semantic parsing rules, the following can be considered: Figure 5 Following the old semantic parsing rules shown, the rule continuous optimizer first uses predetermined prompts (such as semantic structure definitions, data acquisition rule writing definitions, historically generated semantic parsing rules corresponding to the identifier information, etc.) and relevant reference knowledge to call the large model service to parse the security data and obtain a new semantic parsing strategy 2 (such as...). Figure 5 The new semantic parsing rules shown enable the continuous optimizer to reuse the previous rule generation process; then, the continuous optimizer, with the help of the rule engine deployed in the SIEM, calculates the standardized processing result 1 corresponding to the original data under the operation rules transformed based on the semantic parsing strategy 1 (e.g., Figure 5 The standard data 2 shown and the standardized processing result 2 corresponding to the original data under the operation rules obtained based on the semantic parsing strategy 2 (e.g., Figure 5 The standard data shown in Figure 1 is used to enable the rule continuous optimizer to reuse the previous rule execution process; then, the rule selector deployed in the rule continuous optimizer (such as...) Figure 5 The optimizer shown uses predetermined prompts, relevant reference knowledge, and the original data, carrying the standardized processing result 1 and the standardized processing result 2, to call the large model service to obtain a better standardized processing result (such as...) output by the rule optimizer. Figure 5 The standard data shown is 3), so that the rule-based continuous optimizer can subsequently deduce the semantic parsing rule corresponding to the optimal result of each field based on the better normalization result, the normalization result 1, and the normalization result 2, in order to obtain a better semantic parsing rule (such as...). Figure 5 The better semantic parsing rules shown include the semantic parsing rules corresponding to the optimal results of these fields, so that the operation rules can be transformed based on the better semantic parsing rules to achieve iterative updates to the existing standardized processing strategies.

[0116] Therefore, the data standardization scheme provided in this application has at least the following advantages: ① This scheme is based on the ability of large models to understand data semantics and the tendency of the semantic structure of data generated by security devices to stabilize in specific scenarios. This allows the scheme to combine the advantages of large models and rule engines, enabling the extraction of relatively stable identifier information from dynamically variable raw data received by SIEM using feature engineering. Semantic parsing rules are then derived from the raw data through semantic understanding using large models, and these rules are bound to the identifier information for continuous optimization. ② After the semantic parsing rules mature and stabilize, they and the rule engine are directly used to standardize subsequently received security data, significantly improving standardization efficiency. ③ Because this scheme incorporates the characteristics of large models, it can greatly improve the access efficiency of security devices and possesses good adaptability and robustness. Based on this, it can be seen that this application constructs an efficient security data standardization scheme by leveraging the characteristics of large models, effectively reducing the cost and complexity of security analysis and significantly improving the access efficiency of security devices.

[0117] Based on the data standardization processing method provided in the embodiments of this application, the embodiments of this application also provide a data standardization processing apparatus, which will be described below in conjunction with... Figure 6 Explanation and clarification will be provided. Among them, Figure 6 This is a schematic diagram of a data standardization processing apparatus provided in an embodiment of this application. It should be noted that for technical details of the data standardization processing apparatus provided in this embodiment, please refer to the relevant content of the data standardization processing method described above.

[0118] like Figure 6 As shown in the embodiment of this application, the processing apparatus 600 for data standardization includes:

[0119] The generation unit 601 is configured to, in response to receiving the first data, obtain the first identification information of the first data;

[0120] Analysis unit 602 is configured to, in response to the absence of a first strategy corresponding to the first identification information, use a first model to obtain a second strategy corresponding to the first data, wherein the second strategy indicates a data standardization processing flow corresponding to the first data;

[0121] The processing unit 603 is used to process the first data according to the second strategy to obtain the second data; wherein the second strategy and the first identification information are stored accordingly.

[0122] In one possible implementation, the generation unit 601 is specifically used to: extract data type features and data structure features from the first data; and determine the first identification information based on the data type features and the data structure features.

[0123] In one possible implementation, the processing apparatus 600 for data standardization further includes: an execution unit, configured to process the first data according to the first strategy in response to the existence of the first strategy to obtain a data standardization processing result of the first data.

[0124] In one possible implementation, the analysis unit 602 is specifically used to: obtain a first semantic parsing strategy by analyzing the first data using the first model; convert the first semantic parsing strategy into a second strategy, wherein the language corresponding to the second strategy is different from the language corresponding to the first semantic parsing strategy, and the first semantic parsing strategy and the first identification information are correspondingly saved.

[0125] In one possible implementation, the processing apparatus 600 for data standardization further includes: an optimization unit, configured to, in response to the existence of the first strategy, analyze the first data using a second model based on a second semantic parsing strategy corresponding to the first identification information to obtain a third semantic parsing strategy, convert the third semantic parsing strategy into a third strategy, wherein the first strategy is obtained by converting the second semantic parsing strategy; generate third data based on the standardization processing result of the first data under the first strategy and the standardization processing result of the first data under the third strategy, wherein the third data indicates the performance difference between the first strategy and the third strategy; and update the first strategy based on the third data.

[0126] In one possible implementation, the third data indicates the performance difference between the first strategy and the third strategy under multiple fields, the multiple fields including the first field; the optimization unit is specifically configured to: in response to the third data indicating that the performance of the third strategy under the first field is better than the performance of the first strategy under the first field, update the first strategy according to the content of the third strategy corresponding to the first field.

[0127] In one possible implementation, the optimization unit is specifically used to: in response to the indicator corresponding to the first strategy being lower than a preset threshold, use a second model to analyze the first data based on a second semantic parsing strategy corresponding to the first identifier information to obtain a third semantic parsing strategy, wherein the indicator represents the performance of the first strategy;

[0128] The optimization unit is further configured to: update the metrics corresponding to the first strategy after updating the first strategy based on the third data.

[0129] In one possible implementation, the optimization unit is specifically used to: use a third model to perform field-level optimization processing on the standardized processing result of the first data under the first strategy and the standardized processing result of the first data under the third strategy to obtain the third data.

[0130] In one possible implementation, the processing apparatus 600 for data standardization further includes: an editing unit, configured to update the first semantic parsing strategy and the second strategy in response to detecting an editing operation triggered for the first semantic parsing strategy, wherein the updated second strategy is determined based on the updated first semantic parsing strategy.

[0131] In one possible implementation, a preset storage space is configured to store at least one candidate identifier and a strategy corresponding to each candidate identifier; and the second strategy and the first identifier information are stored in the preset storage space.

[0132] In one possible implementation, the first data includes security data collected from the first device, which is generated during the operation of a security application running on the first device.

[0133] In one possible implementation, the processing apparatus 600 for data standardization satisfies at least one of the following constraints: the first data is semi-structured data; the second data is structured data; the semantic information indicated by the second data includes part or all of the semantic information indicated by the first data; the first identification information indicates at least one characteristic of the first data, the at least one characteristic including at least part of the data type and data structure of the first data; the first identification information includes a hash value, the hash value being determined based on the data type characteristics and data structure characteristics of the first data; and the first model serves a large model.

[0134] Based on the aforementioned content regarding the data standardization processing device 600, its working principle includes: upon receiving first data (such as security data like logs generated by a security device), acquiring first identification information of the first data so that the first identification information can uniquely identify the semantic structure of the first data. This allows the device to determine that the semantic structure presented by the first data is a new semantic structure when a first strategy corresponding to the first identification information is detected. Therefore, a second strategy corresponding to the first data is first obtained using a first model, so that the second strategy can indicate the data standardization processing flow corresponding to the first data, thereby enabling the second... The strategy can indicate how to map security data represented according to this new semantic structure to a unified semantic structure (such as the semantic structure of standardized data); then, the second strategy and the first identification information are stored accordingly, and the first data is processed according to the second strategy to obtain second data, so that the second data can represent the standardized processing result of the first data. In this way, it is possible to automatically discover new semantic structures and automatically build standardized processing strategies for new semantic structures, so as to better adapt to the semantic structure standardization processing of most semantically clear security data, significantly shorten the access time of security devices, and present better adaptability and high efficiency.

[0135] In addition, this application also provides an electronic device, the device including a processor and a memory: the memory is used to store instructions or computer programs; the processor is used to execute the instructions or computer programs in the memory so that the electronic device performs any implementation of the data standardization processing method provided in this application.

[0136] See Figure 7The diagram illustrates a structural schematic of an electronic device 700 suitable for implementing embodiments of the present disclosure. Terminal devices in embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 7 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0137] like Figure 7 As shown, the electronic device 700 may include a processing unit (e.g., a central processing unit, a graphics processor, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. The RAM 703 also stores various programs and data required for the operation of the electronic device 700. The processing unit 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0138] Typically, the following devices can be connected to I / O interface 705: input devices 706 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 707 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 708 including, for example, magnetic tapes, hard disks, etc.; and communication devices 709. Communication device 709 allows electronic device 700 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 7 An electronic device 700 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0139] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 709, or installed from storage device 708, or installed from ROM 702. When the computer program is executed by processing device 701, it performs the functions defined in the methods of embodiments of this disclosure.

[0140] The electronic device provided in this embodiment belongs to the same inventive concept as the method provided in the above embodiments. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0141] This application also provides a computer-readable medium storing instructions or a computer program that, when executed on a device, causes the device to perform any implementation of the data standardization processing method provided in this application.

[0142] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0143] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0144] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0145] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, enable the electronic device to perform the aforementioned methods.

[0146] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0147] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0148] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units / modules do not necessarily limit the specific unit itself.

[0149] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0150] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0151] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems or apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and relevant parts can be referred to the method section.

[0152] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0153] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0154] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0155] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A processing method for data standardization, characterized in that, The method comprises: in response to receiving first data, obtaining first identification information of the first data; in response to the absence of a first policy corresponding to the first identification information, obtaining a second policy corresponding to the first data by using a first model, the second policy indicating a data standardization processing procedure corresponding to the first data; processing the first data according to the second policy to obtain second data; wherein the second policy and the first identification information are correspondingly stored.

2. The method of claim 1, wherein, The method further comprises: in response to the presence of the first policy, processing the first data according to the first policy to obtain a data standardization processing result of the first data. The method further comprises:

3. The method of claim 1, wherein, in response to the presence of the first policy, processing the first data according to the first policy to obtain a data standardization processing result of the first data. The method further comprises:

4. The method of claim 1, wherein, in response to the presence of the first policy, processing the first data according to the first policy to obtain a data standardization processing result of the first data. The method further comprises: in response to the presence of the first policy, processing the first data according to the first policy to obtain a data standardization processing result of the first data.

5. The method of claim 4, wherein, The third data indicates performance differences between the first policy and the third policy in a plurality of fields, and the plurality of fields comprises a first field. The method further comprises: in response to the third data indicating that the performance of the third policy in the first field is better than the performance of the first policy in the first field, updating the first policy according to the content in the third policy corresponding to the first field. The method further comprises:

6. The method of claim 5, wherein, in response to the index corresponding to the first policy being lower than a preset threshold, obtaining a third semantic analysis policy of the first data based on a second semantic analysis policy corresponding to the first identification information by using a second model, the index representing the performance of the first policy. After updating the first policy according to the third data, the method further comprises updating the index corresponding to the first policy.

7. The method of claim 5, wherein, ​ ​ 8. The method of claim 5, wherein, The third data is generated according to the standardized processing result corresponding to the first strategy of the first data and the standardized processing result corresponding to the third strategy of the first data. The third data is obtained by performing field-level optimization processing on the standardized processing result corresponding to the first strategy of the first data and the standardized processing result corresponding to the third strategy of the first data by using a third model.

9. The method of claim 4, wherein, The method further comprises: In response to detecting an editing operation triggered for the first semantic analysis strategy, updating the first semantic analysis strategy and the second strategy, and the updated second strategy is determined according to the updated first semantic analysis strategy.

10. The method of claim 1, wherein, The pre-set storage space is configured to save at least one candidate identifier and the corresponding strategy of each candidate identifier. The second strategy and the first identifier information are saved in the pre-set storage space.

11. The method of claim 1, wherein, The first data includes security data collected from a first device, and the security data is generated by a security protection application running on the first device during running.

12. The method according to any one of claims 1 to 11, characterized in that, The method satisfies at least one of the following constraints: The first data is semi-structured data; The second data is structured data; The semantic information indicated by the second data includes part or all of the semantic information indicated by the first data; The first identifier information indicates at least one feature of the first data, and the at least one feature includes at least part of the data type and the data structure of the first data; The first identifier information includes a hash value, and the hash value is determined based on the data type feature and the data structure feature of the first data; The first model is a large model service.

13. An electronic device, comprising: The device comprises a processor and a memory. The memory is configured to store instructions or computer programs. The processor is configured to execute the instructions or computer programs in the memory, so that the electronic device executes the method of any one of claims 1-12.

14. A computer readable medium characterized by The computer readable medium stores instructions or computer programs, which, when executed on a device, cause the device to execute the method of any one of claims 1-12.

15. A computer program product, characterised in that, It comprises a computer program carried on a non-transitory computer readable medium, and the computer program comprises program code for executing the method of any one of claims 1-12.

Images