Data Enrichment Method, Apparatus, Computing Device, and Storage Medium
By pre-constructing the rule base to automatically form logical code, the problems of high labor costs and security risks in traditional data enrichment methods are solved, and fast and secure data enrichment processing is achieved.
Patent Information
- Application Number
- CN202111675284.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-31
- Publication Date
- 2025-07-18
- Estimated Expiration
- 2041-12-31
AI Technical Summary
Traditional data enrichment methods rely on manual writing of logical code, resulting in high labor costs, long time and increased security risks.
By pre-constructing the rule base, users select logical processing rules and enter business parameters to automatically form executable logical code to realize data enrichment processing.
No manual coding is required, data enrichment is quickly achieved and labor costs and security risks are reduced.
Smart Images

Figure CN114327410B_ABST
Abstract
Description
Technical Field
[0001] Embodiments of the present invention relate to the technical field of data processing, and in particular to a data enrichment method, apparatus, computing device, and storage medium. Background Art
[0002] In the security industry, it is often necessary to enrich security data in combination with a threat intelligence library, and then use the enriched data for comprehensive analysis to discover potential threats.
[0003] Traditional data enrichment methods are to manually write enrichment logic code after determining the enrichment logic, and use the enrichment logic code to enrich data. However, the method of manually writing code not only wastes human resources but also takes a long time. In addition, a long security window period will also increase security risks. Summary of the Invention
[0004] Based on the problems that manually writing enrichment logic code wastes human resources and takes a long time, which will increase security risks, embodiments of the present invention provide a data enrichment method, apparatus, computing device, and storage medium, which can effectively reduce human resources and the time taken to form enrichment logic code, and further reduce security risks.
[0005] In a first aspect, an embodiment of the present invention provides a data enrichment method, including:
[0006] Determine the corresponding logical processing rule selected by the user from a pre-constructed rule library based on the enrichment logic required for data enrichment;
[0007] Obtain the service parameters input by the user for the selected logical processing rule;
[0008] According to the selected logical processing rule and the input service parameters, form an executable logical code for data enrichment;
[0009] Obtain the data stream to be enriched, and execute the logical code for data enrichment to enrich the data stream.
[0010] Preferably, before selecting the corresponding logical processing rule from the pre-constructed rule library, it further includes: forming corresponding processing codes for each logical processing rule included in the rule library;
[0011] The forming of the executable logical code for data enrichment includes: respectively obtaining the corresponding processing codes for the selected logical processing rule; combining the obtained processing codes in the processing order of the enrichment logic, and writing the input service parameters into the variable parameters of the corresponding processing code to obtain the executable logical code for data enrichment.
[0012] Preferably, the service parameters at least include: a target data identifier for beacon collision and a target enrichment information library for providing enriched data;
[0013] The logic code for data enrichment at least includes: a screening logic based on the target data identifier, an access logic to the target enrichment information library, a query logic to the target enrichment information library based on the target data identifier, and a combination logic for combining the queried enriched data with the data stream; wherein, the processing code corresponding to the access logic does not contain variable parameters; the screening logic, the query logic, and the combination logic all contain variable parameters.
[0014] Preferably, performing the logic code for data enrichment to enrich the data stream includes:
[0015] Using the screening logic, screening the data stream to obtain matching elements corresponding to the target data identifier in the data stream;
[0016] Using the access logic, accessing the target enrichment information library;
[0017] Using the query logic, obtaining target enrichment information mapped to the matching elements in the target enrichment information library;
[0018] Using the combination logic, combining the target enrichment information with the data stream to obtain an enriched data stream.
[0019] In a second aspect, an embodiment of the present invention further provides a data enrichment device, including:
[0020] A rule determination unit for determining a corresponding logical processing rule selected by the user from a pre-constructed rule library based on the enrichment logic required for data enrichment;
[0021] A parameter acquisition unit for acquiring service parameters input by the user for the selected logical processing rule;
[0022] A logic code formation unit for forming an executable logic code for data enrichment according to the selected logical processing rule and the input service parameters;
[0023] An enrichment processing unit for acquiring a data stream to be enriched and executing the logic code for data enrichment to enrich the data stream.
[0024] Preferably, it further includes: a storage unit for storing corresponding processing codes respectively formed for each logical processing rule included in the rule library;
[0025] The logic code formation unit is specifically configured to respectively obtain corresponding processing codes for the selected logic processing rules; combine the obtained processing codes according to the processing sequence of the enrichment logic, and write the input service parameters into the variable parameters of the corresponding processing codes to obtain an executable logic code for data enrichment.
[0026] Preferably, the service parameters at least include: a target data identifier for beacon collision and a target enrichment information library for providing enriched data;
[0027] The logic code for data enrichment at least includes: a screening logic based on the target data identifier, an access logic to the target enrichment information library, a query logic for querying the target enrichment information library based on the target data identifier, and a combination logic for combining the queried enriched data with the data stream; wherein, the processing code corresponding to the access logic does not include variable parameters; the screening logic, the query logic, and the combination logic all include variable parameters.
[0028] Preferably, the enrichment processing unit is specifically configured to use the screening logic to screen the data stream to obtain matching elements corresponding to the target data identifier in the data stream; use the access logic to access the target enrichment information library; use the query logic to obtain target enrichment information mapped to the matching elements in the target enrichment information library; use the combination logic to combine the target enrichment information with the data stream to obtain an enriched data stream.
[0029] In a third aspect, an embodiment of the present invention further provides a computing device, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the method described in any embodiment of this specification is implemented.
[0030] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed on a computer, the computer is made to execute the method described in any embodiment of this specification.
[0031] An embodiment of the present invention provides a data enrichment method, apparatus, computing device, and storage medium. By pre-constructing a rule base containing logical processing rules, after a user determines the enrichment logic, the corresponding logical processing rules are selected from the rule base, and service parameters are input. Based on the selected logical processing rules and the input service parameters, logical code for data enrichment can be automatically formed, and then the data stream to be enriched is enriched using the formed logical code. It can be seen that the entire process does not require manual coding. Only by selecting the required logical processing rules and service parameters can data enrichment be quickly achieved, which can effectively reduce labor costs, reduce the time-consuming for forming the enrichment logic code, and further reduce security risks. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following-described drawings are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.
[0033] Figure 1 is a flowchart of a data enrichment method provided by an embodiment of the present invention;
[0034] Figure 2 is a schematic diagram of a data enrichment process provided by an embodiment of the present invention;
[0035] Figure 3 is another schematic diagram of a data enrichment process provided by an embodiment of the present invention;
[0036] Figure 4 is a hardware architecture diagram of a computing device provided by an embodiment of the present invention;
[0037] Figure 5 is a structural diagram of a data enrichment apparatus provided by an embodiment of the present invention;
[0038] Figure 6 is another structural diagram of a data enrichment apparatus provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0039] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments of the present invention belong to the scope of protection of the present invention.
[0040] As described above, after determining the enrichment logic, the enrichment logic code is manually written, and the data is enriched using the enrichment logic code. However, the method of manually writing code not only wastes labor costs but also takes a long time. In addition, the long security window period will also increase security risks. Based on the problems existing in manually writing code, it is possible to consider forming the corresponding new enrichment logic by configuring the changed content of the enrichment logic in the form of rules, and placing the processing logic code corresponding to the configured rules in the original code of the enrichment logic when parsing the new enrichment logic, so that the function of automatically forming the enrichment logic can be realized without modifying the code.
[0041] The specific implementation manners of the above concept are described below.
[0042] Please refer to Figure 1 , an embodiment of the present invention provides a data enrichment method, which includes:
[0043] Step 100, determining the corresponding logical processing rule selected by the user from a pre-constructed rule library based on the enrichment logic required for data enrichment;
[0044] Step 102, obtaining the service parameters input by the user for the selected logical processing rule;
[0045] Step 104, forming an executable logical code for data enrichment according to the selected logical processing rule and the input service parameters;
[0046] Step 106, obtaining the data stream to be enriched, and executing the logical code for data enrichment to enrich the data stream.
[0047] In the embodiment of the present invention, by pre-constructing a rule library containing logical processing rules, after the user determines the enrichment logic, the corresponding logical processing rule is selected from the rule library, and service parameters are input. Based on the selected logical processing rule and the input service parameters, the logical code for data enrichment can be automatically formed, and then the data stream to be enriched is enriched using the formed logical code. It can be seen that the entire process does not require manual coding, and only by selecting the required logical processing rules and service parameters, data enrichment can be quickly realized, which can effectively reduce labor costs, reduce the time-consuming for forming the enrichment logic code, and further reduce security risks.
[0048] The following describes Figure 1 the execution manners of the following steps.
[0049] First, steps 100 "determining the corresponding logical processing rule selected by the user from a pre-constructed rule library based on the enrichment logic required for data enrichment" and 102 "obtaining the service parameters input by the user for the selected logical processing rule" are described.
[0050] In an embodiment of the present invention, a rule library needs to be pre-constructed, and the rule library includes multiple logical processing rules. For example, the user selects a screening logical processing rule, an access logical processing rule for the enrichment information library, a query logical processing rule, and a combination logical processing rule from the rule library based on the required enrichment logic.
[0051] Further, before step 100, it may include: forming corresponding processing codes for each logical processing rule included in the rule library. Among them, the processing code corresponding to each logical processing rule can be stored in the form of a module or a text. So that it can be directly called when forming the logical code.
[0052] In an embodiment of the present invention, in order to ensure the reusability of the processing code, when forming the processing code, it can be determined whether the corresponding logical processing rule has variable parameters. If so, the processing code of the corresponding logical processing rule is encoded according to the variable parameters, and the processing code contains the variable parameters.
[0053] Among them, there can be multiple access logical processing rules for the enrichment information library included in the rule library, corresponding to different types of enrichment information libraries. For example, if the enrichment information library is stored in an ES search engine, then a solid processing code can be formed for the access logical processing rule of the ES enrichment information library, that is, the processing code does not contain variable parameters. After the user determines the type of the enrichment information library required in the enrichment logic, the user can directly select the access logical processing rule of the corresponding type of enrichment information library from the rule library.
[0054] It can be seen that the processing code corresponding to the logical processing rule may contain variable parameters. Therefore, after determining the logical processing rule selected by the user, it can be determined whether the processing code corresponding to the selected logical processing rule contains variable parameters. If it contains, the variable parameters are displayed to the user, and the user inputs business parameters for replacing the values of the variable parameters according to the displayed variable parameters.
[0055] In an embodiment of the present invention, the business parameters input by the user may at least include: a target data identifier for beacon collision and a target enrichment information library for providing enriched data.
[0056] For example, the target data identifier is an IP address, an MD5 value, a file name, etc., and the target enrichment information library is an ES enrichment information library.
[0057] Next, for step 104, an executable logical code for data enrichment is formed according to the selected logical processing rule and the input business parameters.
[0058] In one embodiment of the present invention, based on the processing codes formed in advance for each logical processing rule, when forming the executable logical code for data enrichment, it specifically includes: respectively obtaining the corresponding processing codes for the selected logical processing rules; combining the obtained processing codes in the processing order of the enrichment logic, and writing the input service parameters into the variable parameters of the corresponding processing codes to obtain the executable logical code for data enrichment.
[0059] Among them, based on the service parameters input by the user, the logical code for data enrichment at least includes: the screening logic based on the target data identifier, the access logic to the target enrichment information library, the query logic to the target enrichment information library based on the target data identifier, and the combination logic for combining the queried enriched data with the data stream; wherein, the processing code corresponding to the access logic does not contain variable parameters; the screening logic, the query logic, and the combination logic all contain variable parameters.
[0060] It can be seen that when the data processed by the processing code containing variable parameters changes, the processing code can still be reused, and only different service parameters need to be used to replace the values of the variable parameters it contains. And the processing code that does not contain variable parameters belongs to the fixed code, and corresponding processing codes need to be generated in advance for different situations, and the corresponding processing codes can be directly retrieved during use.
[0061] The executable logical code for data enrichment can be encapsulated into an operator for executing the enrichment logic. It can be seen that the processing logic in the operator can be configured, and the service parameters can also be configured. When the user performs data enrichment, corresponding configurations can be made according to the requirements, and the logical code can be automatically and quickly formed without manual coding, so that data enrichment processing can be quickly realized.
[0062] It should be noted that each logic included in the above-mentioned formed logical code for data enrichment is one kind of enrichment logic. If the enrichment logic is different, other logics can be included in the formed logical code.
[0063] Finally, for step 106, obtain the data stream to be enriched, and execute the logical code for data enrichment to enrich the data stream.
[0064] Among them, the data stream can be any JSON - formatted streaming data, which can be the streaming data output from the data bus or the streaming data converted from the static data obtained from the static database.
[0065] Specifically, when enriching the data stream, it may include: using the screening logic to screen the data stream to obtain the matching elements corresponding to the target data identifier in the data stream; using the access logic to access the target enrichment information library; using the query logic to obtain the target enrichment information mapped to the matching elements in the target enrichment information library; using the combination logic to combine the target enrichment information with the data stream to obtain the enriched data stream.
[0066] For example, this enrichment process is to enrich the geographical location corresponding to the IP address in the data stream. Among them, the target enrichment information library is the ES enrichment information library, and the ES enrichment information library stores the mapping relationship between the IP address and the geographical location. Taking the IP address belonging to Jiangsu Province as the target data identifier as an example, then using the screening logic, screen the received data stream for the IP addresses belonging to Jiangsu Province. After screening the IP addresses belonging to Jiangsu Province (such as screening to IP1), use the access logic to access the ES enrichment information library and query the geographical location mapped to this IP1 in the target enrichment information library (such as querying the geographical location as "XX Street, XX County, XX City, Jiangsu Province, No. XX"), then use the combination logic to combine the geographical location "XX Street, XX County, XX City, Jiangsu Province, No. XX" to the specified position in the data stream.
[0067] Among them, when combining, combining the enrichment information to the specified position in the data stream can be either fixed or specified by the user by inputting service parameters.
[0068] Please refer to Figure 2 , which is a schematic diagram of the entire data enrichment process. The data enrichment operator in the figure is used to form logical code and execute the logical code. Among them, the rule parsing module is used to execute step 100, step 102, and step 104. The streaming data is input into this data enrichment operator. The data screening module is used to execute the screening logic of the data stream. The access query module is used to execute the access logic and query logic for the enrichment information library. The data combination module is used to execute the combination logic. The combined data stream is output from the data enrichment operator. The output data format is still streaming data.
[0069] Among them, the data stream can be introduced through methods such as kafka and syslog.
[0070] Please refer to Figure 3, when the data enrichment operator performs enrichment processing, based on the target data identifier, the value corresponding to the key field in the data stream can be filtered out. The value is used to query the enrichment information library, and the infoKey field in the enrichment information library is queried. According to the mapping of the infoKey field, the value field corresponding to the richKey field of the enrichment information is obtained, and the value field is combined and written into the position of the value corresponding to the targetKey field of the data stream, thereby realizing data enrichment processing.
[0071] In the embodiment of the present invention, when data enrichment of a data stream is required, no manual coding is needed. Only by selecting the required logical processing rules and service parameters can data enrichment be quickly realized. This method has high usability and no requirements for coding technology. Even operation and maintenance personnel who do not understand coding can realize data enrichment processing through rule configuration, and this method has higher readability and lower maintenance threshold compared with coding.
[0072] As Figure 4 , Figure 5 shown, the embodiment of the present invention provides a data enrichment device. The device embodiment can be implemented by software, or by hardware or a combination of software and hardware. From the hardware level, as Figure 4 shown, it is a hardware architecture diagram of a computing device where the data enrichment device provided by the embodiment of the present invention is located. In addition to Figure 4 the processor, memory, network interface, and non-volatile memory shown, the computing device where the device is located in the embodiment usually may also include other hardware, such as a forwarding chip responsible for processing packets, etc. Taking software implementation as an example, as Figure 5 shown, as a logically meaningful device, it is formed by the CPU of its computing device reading the corresponding computer program in the non-volatile memory into the memory and running. A data enrichment device provided by this embodiment includes:
[0073] A rule determination unit 501, configured to determine the corresponding logical processing rule selected by the user from a pre-constructed rule library based on the enrichment logic required for data enrichment;
[0074] A parameter acquisition unit 502, configured to acquire the service parameters input by the user for the selected logical processing rule;
[0075] A logical code formation unit 503, configured to form an executable logical code for data enrichment according to the selected logical processing rule and the input service parameters;
[0076] An enrichment processing unit 504, configured to acquire the data stream to be enriched and execute the logical code for data enrichment to enrich the data stream.
[0077] In one embodiment of the present invention, please refer to Figure 6 , further comprising: a storage unit 505 for storing corresponding processing codes respectively formed for each logical processing rule included in the rule library;
[0078] The logic code forming unit is specifically configured to respectively obtain corresponding processing codes for the selected logical processing rules; combine the obtained processing codes according to the processing sequence of the enrichment logic, and write the input service parameters into the variable parameters of the corresponding processing codes to obtain an executable logic code for data enrichment.
[0079] In one embodiment of the present invention, the service parameters at least include: a target data identifier for beacon collision and a target enrichment information library for providing enriched data;
[0080] The logic code for data enrichment at least includes: a screening logic based on the target data identifier, an access logic to the target enrichment information library, a query logic to the target enrichment information library based on the target data identifier, and a combination logic for combining the queried enriched data with the data stream; wherein, the processing code corresponding to the access logic does not include variable parameters; the screening logic, the query logic, and the combination logic all include variable parameters.
[0081] In one embodiment of the present invention, the enrichment processing unit is specifically configured to use the screening logic to screen the data stream to obtain matching elements corresponding to the target data identifier in the data stream; use the access logic to access the target enrichment information library; use the query logic to obtain target enrichment information mapped to the matching elements in the target enrichment information library; use the combination logic to combine the target enrichment information with the data stream to obtain an enriched data stream.
[0082] It can be understood that the structure schematically shown in the embodiments of the present invention does not constitute a specific limitation on a data enrichment device. In other embodiments of the present invention, a data enrichment device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or have different component arrangements. The components shown in the figure can be implemented in hardware, software, or a combination of software and hardware.
[0083] Regarding the information interaction, execution process, etc. among the various modules in the above device, since they are based on the same concept as the method embodiments of the present invention, the specific content can be referred to the description in the method embodiments of the present invention and will not be elaborated here.
[0084] An embodiment of the present invention further provides a computing device, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, a data enrichment method in any embodiment of the present invention is implemented.
[0085] An embodiment of the present invention further provides a computer-readable storage medium. A computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, the processor is caused to execute a data enrichment method in any embodiment of the present invention.
[0086] Specifically, a system or device equipped with a storage medium can be provided. A software program code for implementing the functions in any one of the above embodiments is stored on the storage medium, and the computer (or CPU or MPU) of the system or device is caused to read and execute the program code stored in the storage medium.
[0087] In this case, the program code read from the storage medium itself can implement the functions in any one of the above embodiments. Therefore, the program code and the storage medium storing the program code constitute a part of the present invention.
[0088] Embodiments of the storage medium for providing the program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Optionally, the program code can be downloaded from a server computer via a communication network.
[0089] In addition, it should be clear that not only can the functions in any one of the above embodiments be implemented by executing the program code read by the computer, but also by causing an operating system or the like operating on the computer based on the instructions of the program code to complete part or all of the actual operations.
[0090] In addition, it can be understood that the program code read from the storage medium is written into a memory provided in an expansion board inserted into the computer or into a memory provided in an expansion module connected to the computer. Subsequently, based on the instructions of the program code, a CPU or the like installed on the expansion board or the expansion module is caused to execute part and all of the actual operations, thereby implementing the functions in any one of the above embodiments.
[0091] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements not only includes those elements but also includes other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the phrase "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising said element.
[0092] Those of ordinary skill in the art can understand that all or part of the steps for implementing the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps including those of the above method embodiments; and the foregoing storage medium includes various media that can store program codes, such as ROM, RAM, magnetic disks or optical discs.
[0093] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments or perform equivalent replacements for some of the technical features. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A data enrichment method, characterized in that, Including: Determine the corresponding logical processing rule selected by the user from a pre - constructed rule library based on the enrichment logic required for data enrichment; Obtain the business parameters input by the user for the selected logical processing rule; Based on the selected logical processing rule and the input business parameters, form an executable logical code for data enrichment; Obtain the data stream to be enriched, and execute the logical code for data enrichment to perform enrichment processing on the data stream; Before the corresponding logical processing rule selected from the pre - constructed rule library, it further includes: for each logical processing rule included in the rule library, respectively form corresponding processing codes; The forming of the executable logical code for data enrichment includes: respectively obtain the corresponding processing codes for the selected logical processing rule; combine the obtained processing codes in the processing order of the enrichment logic, and write the input business parameters into the variable parameters of the corresponding processing code to obtain the executable logical code for data enrichment; The business parameters at least include: the target data identifier for beacon collision and the target enrichment information library for providing enriched data; The logical code for data enrichment at least includes: the screening logic based on the target data identifier, the access logic to the target enrichment information library, the query logic to the target enrichment information library based on the target data identifier, and the combination logic for combining the queried enriched data with the data stream; among them, the processing code corresponding to the access logic does not contain variable parameters; the screening logic, the query logic, and the combination logic all contain variable parameters.
2. The method according to claim 1, characterized in that, The executing the logical code for data enrichment to perform enrichment processing on the data stream includes: Using the screening logic, screen the data stream to obtain the matching elements corresponding to the target data identifier in the data stream; Using the access logic, access the target enrichment information library; Using the query logic, obtain the target enrichment information mapped to the matching elements in the target enrichment information library; Using the combination logic, combine the target enrichment information with the data stream to obtain the enriched data stream.
3. A data enrichment device, characterized in that, Including: A rule determination unit for determining the corresponding logical processing rule selected by the user from a pre - constructed rule library based on the enrichment logic required for data enrichment; A parameter acquisition unit for obtaining the business parameters input by the user for the selected logical processing rule; A logical code formation unit for forming an executable logical code for data enrichment based on the selected logical processing rule and the input business parameters; An enrichment processing unit for obtaining the data stream to be enriched and executing the logical code for data enrichment to perform enrichment processing on the data stream; It further includes: a storage unit for storing the corresponding processing codes respectively formed for each logical processing rule included in the rule library. The logic code formation unit is specifically configured to respectively obtain corresponding processing codes for the selected logic processing rules; combine the obtained processing codes according to the processing sequence of the enrichment logic, and write the input service parameters into the variable parameters of the corresponding processing codes to obtain executable logic codes for data enrichment; The service parameters at least include: a target data identifier for beacon collision and a target enrichment information library for providing enriched data; The logic codes for data enrichment at least include: a screening logic based on the target data identifier, an access logic to the target enrichment information library, a query logic to the target enrichment information library based on the target data identifier, and a combination logic for combining the queried enriched data with the data stream; wherein, the processing code corresponding to the access logic does not contain variable parameters; the screening logic, the query logic, and the combination logic all contain variable parameters.
4. The data enrichment device according to claim 3, wherein The enrichment processing unit is specifically configured to use the screening logic to screen the data stream to obtain matching elements corresponding to the target data identifier in the data stream; use the access logic to access the target enrichment information library; Use the query logic to obtain target enrichment information mapped to the matching elements in the target enrichment information library; use the combination logic to combine the target enrichment information with the data stream to obtain an enriched data stream.
5. A computing device, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the method according to any one of claims 1-2 is implemented.
6. A computer-readable storage medium, on which a computer program is stored, and when the computer program is executed in a computer, the computer is made to execute the method according to any one of claims 1-2.
Citation Information
Patent Citations
Method and system for dynamically converting telecommunications service data
CN101958987A
Data augmentation for text-based ai applications
CN111967604A
Method and device for automatically enriching data stream
CN116910135A