Data packet method, device, computer equipment and computer readable storage medium
By refreshing the target strategy in real time through database monitoring and distributed caching, and binding the target group identifier, the problem of slow grouping speed and low efficiency in existing technologies is solved, achieving fast response and efficient grouping.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- KANG JIAN INFORMATION TECH (SHENZHEN) CO LTD
- Filing Date
- 2022-10-18
- Publication Date
- 2026-04-24
AI Technical Summary
In existing technologies, incorporating the logic generated by business data into the data grouping process increases the complexity of the business and degrades processing performance, resulting in slow grouping speed, low efficiency, and difficulty in achieving rapid response.
By monitoring the specified database, the target events to be grouped are obtained, and the target strategy is read from the distributed cache that is refreshed in real time on the data group management platform. The target group identifier of the target data table is determined according to the target strategy and bound to the target event. The group strategy configuration does not need to intrude into the business layer; it only binds the group identifier to the corresponding data table.
It reduces business complexity, improves grouping speed and efficiency, and enables rapid response to the grouping needs of target events.
Smart Images
Figure CN115563085B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of Internet technology, and in particular to a data packetization method, apparatus, computer device, and computer-readable storage medium. Background Technology
[0002] With the continuous advancement of internet technology, technological progress, and the free flow of information, communication between people has become increasingly close, and life has become more convenient, generating big data. Various businesses generate corresponding business data, which is rapidly expanding and growing in size. Data management has become increasingly important, leading to the need for data grouping and management.
[0003] In related technologies, in order to achieve data group management, most of them adopt offline t+1 processing or directly access the group in the logical process that requires business data to be generated. Adjusting the group requires rescheduling the release and cannot dynamically backtrack to adjust the group.
[0004] In the process of developing this application, the applicant discovered that the relevant technology has at least the following problems:
[0005] Integrating business data into the grouping process increases business complexity and processing performance overhead, making it too invasive. This necessitates rescheduling and re-issuing the grouping, resulting in slow grouping speed, low efficiency, and difficulty in achieving rapid response. Summary of the Invention
[0006] In view of this, this application provides a data grouping method, apparatus, computer device, and computer-readable storage medium. The main purpose is to solve the problem that the current approach of accessing the logic generated by business data into the grouping process increases the complexity of the business and the loss of processing performance, making it too invasive, requiring rescheduling and re-release of the grouping, resulting in slow grouping speed, low efficiency, and difficulty in achieving rapid response.
[0007] According to a first aspect of this application, a data grouping method is provided, which is applicable to a grouping processing platform, including:
[0008] Monitor data in a specified database to obtain target events to be grouped;
[0009] In the distributed cache, the target strategy published by the target data table corresponding to the target event is read, and the data in the distributed cache comes from the real-time refresh of the data group management platform;
[0010] According to the target strategy, the target group identifier corresponding to the target data table is determined, and the target group identifier is bound to the target event. The target group identifier is used to indicate the group corresponding to the target data table.
[0011] According to a second aspect of this application, a data grouping method is provided, which is applicable to a data group management platform, including:
[0012] Policy information is set for each data table that is accessed, resulting in multiple data tables. These multiple data tables are then accessed to the group processing platform for review and publication. The table definition of each data table includes fixed data group fields and attribute elements.
[0013] When a successful publication message is received, the second specified data table corresponding to the successful publication message is determined, and the specified strategy corresponding to the second specified data table is refreshed to the distributed cache.
[0014] According to a third aspect of this application, a data packetization apparatus is provided, which is suitable for a packet processing platform, comprising:
[0015] The acquisition module is used to listen to data in a specified database and acquire target events to be grouped.
[0016] The reading module is used to read the target strategy published in the target data table corresponding to the target event in the distributed cache. The data in the distributed cache comes from the real-time refresh of the data group management platform.
[0017] The determination module is used to determine the target group identifier corresponding to the target data table according to the target strategy, and bind the target group identifier to the target event. The target group identifier is used to indicate the group corresponding to the target data table.
[0018] According to a fourth aspect of this application, a data grouping apparatus is provided, which is suitable for a data group management platform, comprising:
[0019] The access module is used to set policy information for each accessed data table, obtain multiple data tables, and access the multiple data tables to the group processing platform for review and publication. The table definition of the data table includes fixed data group fields and attribute elements.
[0020] The caching module is used to determine the second specified data table corresponding to the successful publication message when a successful publication message is received, and to refresh the specified strategy corresponding to the second specified data table to the distributed cache.
[0021] According to a fifth aspect of this application, a computer device is provided, including a storage device, a processor, and a computer program stored on the storage device and executable on the processor, wherein the processor executes the program to implement the steps of any of the methods described in the first aspect above.
[0022] According to a sixth aspect of this application, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any of the first aspects above.
[0023] By employing the above technical solutions, this application provides a data grouping method, apparatus, computer device, and computer-readable storage medium. First, this application listens to a specified database to obtain target events to be grouped. Then, it reads the target policy published by the target data table corresponding to the target event from the distributed cache that is refreshed in real-time on the data group management platform. Finally, according to the target policy, it determines the target group identifier corresponding to the target data table and binds the target group identifier indicating grouping to the target event. The grouping policy configuration does not need to intrude into the business layer; it only binds the group identifier to the corresponding data table, thereby reducing business complexity. Furthermore, the data in the distributed cache is refreshed in real-time by the data group management platform, enabling grouping immediately upon the occurrence of target events, achieving rapid response, and thus improving the speed and efficiency of grouping.
[0024] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below. Attached Figure Description
[0025] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:
[0026] Figure 1A This illustration shows a schematic flowchart of a data grouping method provided in an embodiment of this application;
[0027] Figure 1B This illustration shows a schematic flowchart of a data grouping method provided in an embodiment of this application;
[0028] Figure 2 This illustration shows a schematic flowchart of a data grouping method provided in an embodiment of this application;
[0029] Figure 3A This illustration shows a structural schematic diagram of a data packetization device provided in an embodiment of this application;
[0030] Figure 3B This illustration shows a structural schematic diagram of a data packetization device provided in an embodiment of this application;
[0031] Figure 3CThis illustration shows a structural schematic diagram of a data packetization device provided in an embodiment of this application;
[0032] Figure 3D This illustration shows a structural schematic diagram of a data packetization device provided in an embodiment of this application;
[0033] Figure 3E This illustration shows a structural schematic diagram of a data packetization device provided in an embodiment of this application;
[0034] Figure 4 A schematic diagram of the device structure of a computer device provided in an embodiment of this application is shown. Detailed Implementation
[0035] Exemplary embodiments of the present application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the scope of the present application to those skilled in the art.
[0036] This application provides a data grouping method, such as... Figure 1A As shown, this method is applicable to packet processing platforms and includes:
[0037] 101. Monitor data in the specified database and obtain the target events to be grouped.
[0038] In this embodiment, the designated database can be a MySQL database. MySQL is a relational database management system. Relational databases store data in different tables, rather than putting all data in one large repository, thus increasing speed and improving flexibility. Furthermore, data monitoring is performed on the designated database to ensure asynchronous, near real-time data processing event grouping.
[0039] 102. In the distributed cache, the target strategy published by the target data table corresponding to the target event is read. The data in the distributed cache comes from the real-time refresh of the data group management platform.
[0040] Each target event corresponds to a target data table, which stores the data generated by the target event. The data group management platform configures corresponding policy information for each data table and refreshes the policy information to the distributed cache in real time after the data table is successfully published. Therefore, the group processing platform reads the target policy from the target data table in the distributed cache.
[0041] 103. According to the target strategy, determine the target group identifier corresponding to the target data table, and bind the target group identifier to the target event. The target group identifier is used to indicate the group corresponding to the target data table.
[0042] Furthermore, according to the target strategy, the target group identifier corresponding to the target data table is determined, that is, the corresponding group identifier. The target group identifier can be a number, a letter, a combination of numbers and letters, etc., and this application does not limit the specific form and content of the identifier. For example, if the calculated target group identifier is 1, then the group is bound to group 1.
[0043] The method provided in this application first involves monitoring a specified database to obtain target events to be grouped. Then, it reads the target policy published by the target data table corresponding to the target event from the distributed cache that is refreshed in real-time on the data group management platform. Finally, according to the target policy, it determines the target group identifier corresponding to the target data table and binds the target group identifier indicating the grouping to the target event. The grouping policy configuration does not need to intrude into the business layer; it only binds the group identifier to the corresponding data table, thus reducing business complexity. Furthermore, the data in the distributed cache is refreshed in real-time by the data group management platform, enabling immediate grouping of target events upon their occurrence, achieving rapid response and improving the speed and efficiency of grouping.
[0044] This application provides a data grouping method, such as... Figure 1B As shown, this method is applicable to data group management platforms, including:
[0045] 104. Set policy information for each data table to obtain multiple data tables. Connect these multiple data tables to the group processing platform for review and publication. The table definition of each data table includes fixed data group fields and attribute elements.
[0046] For the data group management platform, this platform sets corresponding policy information for each data table stored in the database for subsequent grouping operations. Further, after configuring the policy information, these data tables need to be sent to the grouping processing platform for access, where they are reviewed and published. It should be noted that the data table includes a database name, table name, and table definition. The table definition includes a fixed data group field `group_id`, as well as attribute elements used for subsequent grouping calculations.
[0047] 105. When a successful publication message is received, determine the second specified data table corresponding to the successful publication message, and refresh the specified strategy corresponding to the second specified data table to the distributed cache.
[0048] Once the data group management platform receives a successful publication notification, it identifies the notification and retrieves the data table specified in the notification (the second designated data table). Further, it refreshes the specified policy configured for this second designated data table to the distributed cache. Distributed caching can handle large amounts of dynamic data, making it well-suited for scenarios in the Web 2.0 era, such as social networking sites, where user-generated content is required. Expanding from local caching to distributed caching also shifts the focus from the data transfer speed differences between CPU, memory, and cache to the data transfer speed differences between business systems, databases, and the distributed cache.
[0049] The method provided in this application first involves monitoring a specified database to obtain target events to be grouped. Then, it reads the target policy published by the target data table corresponding to the target event from the distributed cache that is refreshed in real-time on the data group management platform. Finally, according to the target policy, it determines the target group identifier corresponding to the target data table and binds the target group identifier indicating the grouping to the target event. The grouping policy configuration does not need to intrude into the business layer; it only binds the group identifier to the corresponding data table, thus reducing business complexity. Furthermore, the data in the distributed cache is refreshed in real-time by the data group management platform, enabling immediate grouping of target events upon their occurrence, achieving rapid response and improving the speed and efficiency of grouping.
[0050] This application provides a data grouping method, such as... Figure 2 As shown, the method includes:
[0051] 201. Set policy information for each data table to obtain multiple data tables. Connect these multiple data tables to the group processing platform for review and publication. The table definition of each data table includes fixed data group fields and attribute elements.
[0052] With the continuous advancement of internet technology, technological progress, and information flow, communication between people has become increasingly close, and life has become more convenient, generating big data. Various businesses generate corresponding business data, which is rapidly expanding and increasing in size, making data management increasingly important and bringing about the need for data grouping management. Currently, to achieve data grouping management, most methods employ offline T+1 processing or directly integrate grouping into the logic processes that generate business data. Adjusting groups requires rescheduling and re-release, making dynamic retrospective adjustments impossible. However, the applicant recognizes that integrating business data generation logic into grouping increases business complexity and processing performance overhead, making it too invasive. Rescheduling and re-releaseing grouping adjustments results in slow grouping speed, low efficiency, and difficulty in achieving rapid response.
[0053] Therefore, this application provides a data grouping method, apparatus, computer device, and computer-readable storage medium. First, this application listens to a specified database to obtain target events to be grouped. Then, it reads the target policy published by the target data table corresponding to the target event from the distributed cache that is refreshed in real time on the data group management platform. Finally, according to the target policy, it determines the target group identifier corresponding to the target data table and binds the target group identifier indicating grouping to the target event. The grouping policy configuration does not need to intrude into the business layer; it only binds the group identifier to the corresponding data table, thereby reducing business complexity. In addition, the data in the distributed cache is refreshed in real time by the data group management platform, enabling immediate grouping upon the occurrence of target events, achieving rapid response, and thus improving the speed and efficiency of grouping.
[0054] This solution utilizes two systems, a data group platform and a grouping processing platform, working together. In this embodiment, the data group management platform sets corresponding policy information for each data table stored in the database for subsequent grouping operations. Further, these configured data tables are sent to the grouping processing platform, which then accesses the data tables and reviews and publishes them. It should be noted that each data table includes a database name, a table name, and a table definition. The table definition includes a fixed data group field, `group_id`, and attribute elements storing data for subsequent grouping calculations.
[0055] 202. Connect multiple data tables from the data group management platform to the group processing platform, review the multiple data tables, generate multiple review results, and issue different release prompts to the data group management platform based on the review results.
[0056] Next, to ensure the accuracy of the policy information configured in each data table, each data table needs to be reviewed. Specifically, multiple data tables from the data group management platform are connected to the group processing platform for review, generating multiple review results. In a specific implementation scenario, relevant personnel can pre-upload review rules, and the group processing platform reviews each data table according to the preset review rules. Alternatively, multiple data tables can be sent to the backend terminal for display, where relevant reviewers can review each data table and upload the review results. Further, for each review result, if the review result indicates failure, a review failure mark is added to the corresponding data table, and a publication failure message is generated and sent to the data group management platform. If the review result indicates approval, the policy is configured and published for the corresponding data table, and a publication success message is generated and sent to the data group management platform.
[0057] 203. When the data group management platform receives a successful publication notification, it determines the second specified data table corresponding to the successful publication notification and refreshes the specified policy corresponding to the second specified data table to the distributed cache.
[0058] When the data group management platform receives a successful publication notification from the group processing platform, it identifies the notification and retrieves the data table indicated by the notification (the second specified data table). Further, it refreshes the specified policy configured for this second specified data table to the distributed cache. Distributed caching can handle large amounts of dynamic data, making it well-suited for scenarios in the Web 2.0 era, such as social networking sites, where user-generated content is required. Expanding from local caching to distributed caching shifts the focus from the data transfer speed differences between CPU, memory, and cache to the data transfer speed differences between business systems, databases, and the distributed cache.
[0059] Additionally, when the data group management platform receives a publication failure notification, it needs to identify the corresponding third-party specified data table. Subsequently, the third-party specified data table is validated to determine if it contains formatting errors, etc. After verification, the third-party specified data table is re-integrated into the group processing platform for review and publication.
[0060] 204. Monitor the specified database and obtain the target events to be grouped.
[0061] In this embodiment, the designated database can be a MySQL database. MySQL is a relational database management system. Relational databases store data in different tables, rather than putting all data in one large repository, thus increasing speed and improving flexibility. Furthermore, data monitoring is performed on the designated database to ensure asynchronous, near real-time data processing event grouping.
[0062] 205. In the distributed cache, the target strategy published by the target data table corresponding to the target event is read. The data in the distributed cache comes from the real-time refresh of the data group management platform.
[0063] Each target event corresponds to a target data table, which stores the data generated by the target event. The data group management platform configures corresponding policy information for each data table and refreshes the policy information to the distributed cache in real time after the data table is successfully published. Therefore, the group processing platform reads the target policy from the target data table in the distributed cache.
[0064] 206. In the table definition of the target data table, read the attribute elements corresponding to the target data table, determine the group identifier of the attribute element according to the policy rules indicated by the target policy, use the group identifier as the target group identifier, and bind the target group identifier to the target event.
[0065] Furthermore, according to the target strategy, the target group identifier corresponding to the target data table is determined, that is, the corresponding group identifier. The target group identifier can be a number, a letter, a combination of numbers and letters, etc., and this application does not limit the specific form and content of the identifier. For example, if the calculated target group identifier is 1, then the group is bound to group 1. The specific process of determining the target group identifier is as follows:
[0066] First, in the target data table's table definition, the corresponding attribute elements are read, and the grouping identifier for the matched attribute elements is determined according to the policy rules indicated by the target policy. In actual operation, different data tables may correspond to different policy rules.
[0067] For example, when the policy rule is the last digit range of a numeric attribute (if the method is the last digit of a numeric attribute, then it is necessary to verify that the policy list range contains 1-9), if the attribute element of the table is 'a', and the last digit of 'a' is in the range of 1-2, then group 1 is hit, and the target group identifier is 1. If the attribute element of the table is 'b', and the last digit of 'b' is in the range of 3-9, then group 2 is hit, and the target group identifier is 2. When the policy rule is an attribute value, if the attribute element of the table is 'c', then c is determined to hit attribute group 3, and the target group identifier is 3.
[0068] Next, the group identifier is used as the target group identifier, and the target group identifier is bound to the group_id field of the target event's data table to complete the grouping of the target event.
[0069] 207. When a backtracking instruction is detected, identify the backtracking instruction and reset the grouping of the first specified data table indicated by the backtracking instruction.
[0070] In this embodiment, to meet the requirements of backtracking and regrouping, and dynamically changing groups, the data group management platform supports selecting data backtracking and the backtracking range. The group processing platform redistributes the new grouping strategy and generates backtracking instructions. In fact, this application also includes a backtracking unit. This unit identifies the backtracking instructions to obtain the first specified data table indicated by the backtracking instruction and the backtracking range. Subsequently, based on the backtracking range, the first specified data table is retrieved using a pagination method based on group identifier intervals, and the new specified strategy corresponding to the first specified data table is determined. Finally, according to the new specified strategy, the specified identifier corresponding to the first specified data table is determined, and the specified identifier is bound to the group_id in the data table.
[0071] The method provided in this application first involves monitoring a specified database to obtain target events to be grouped. Then, it reads the target policy published by the target data table corresponding to the target event from the distributed cache that is refreshed in real-time on the data group management platform. Finally, according to the target policy, it determines the target group identifier corresponding to the target data table and binds the target group identifier indicating the grouping to the target event. The grouping policy configuration does not need to intrude into the business layer; it only binds the group identifier to the corresponding data table, thus reducing business complexity. Furthermore, the data in the distributed cache is refreshed in real-time by the data group management platform, enabling immediate grouping of target events upon their occurrence, achieving rapid response and improving the speed and efficiency of grouping.
[0072] Furthermore, as Figure 1A To specifically implement the method, this application provides a data grouping device, such as... Figure 3A As shown, the device is suitable for a group processing platform and includes: an acquisition module 301, a reading module 302, and a determination module 303.
[0073] The acquisition module 301 is used to listen to data in a specified database and acquire target events to be grouped.
[0074] The reading module 302 is used to read the target strategy published by the target data table corresponding to the target event in the distributed cache. The data in the distributed cache comes from the real-time refresh of the data group management platform.
[0075] The determining module 303 is used to determine the target group identifier corresponding to the target data table according to the target strategy, and bind the target group identifier to the target event. The target group identifier is used to indicate the group corresponding to the target data table.
[0076] In specific application scenarios, such as Figure 3B As shown, the device further includes: an audit module 304, a first generation module 305, and a second generation module 306.
[0077] The audit module 304 is used to connect multiple data tables of the data group management platform to the group processing platform, audit the multiple data tables, and generate multiple audit results;
[0078] The first generation module 305 is used to, for each of the multiple review results, if the review result indicates that it is not approved, add a review failure mark to the data table corresponding to the review result, generate a publication failure prompt, and send the publication failure prompt to the data group management platform;
[0079] The second generation module 306 is used to configure and publish the strategy for the data table corresponding to the audit result if the audit result indicates that the audit is successful, and to generate a successful publication prompt and send the successful publication prompt to the data group management platform.
[0080] In a specific application scenario, the determining module 303 is used to read the attribute elements corresponding to the target data table in the table definition of the target data table; determine the group identifier that the attribute element hits according to the strategy rules indicated by the target strategy; and use the group identifier as the target group identifier.
[0081] In specific application scenarios, such as Figure 3C As shown, the device also includes: a backtracking module 307, a pull module 308, and an identification module 309.
[0082] The backtracking module 307 is used to identify the backtracking instruction when a backtracking instruction is detected, and to obtain the first specified data table indicated by the backtracking instruction and the backtracking range;
[0083] The retrieval module 308 is used to retrieve the first specified data table by pagination using group identifier intervals according to the backtracking range;
[0084] The identification module 309 is used to determine the new designation strategy corresponding to the first designated data table, determine the designated identifier corresponding to the first designated data table according to the new designation strategy, and bind the designated identifier to the first designated data table.
[0085] Furthermore, as Figure 1B To specifically implement the method, this application provides a data grouping device, such as... Figure 3D As shown, the device is suitable for a packet processing platform and includes: an access module 310 and a cache module 311.
[0086] The access module 310 is used to set policy information for each accessed data table to obtain multiple data tables, and to access the multiple data tables to the group processing platform for review and publication. The table definition of the data table includes fixed data group fields and attribute elements.
[0087] The caching module 311 is used to determine the second specified data table corresponding to the successful publication prompt when a successful publication prompt is received, and to refresh the specified strategy corresponding to the second specified data table to the distributed cache.
[0088] In specific application scenarios, such as Figure 3E As shown, the device also includes: a verification module 312 and a sending module 313.
[0089] The verification module 312 is used to determine the third specified data table corresponding to the publishing failure message when a publishing failure message is received, and to verify the third specified data table.
[0090] The sending module 313 is used to reconnect the third designated data table to the group processing platform for review and publication after verification.
[0091] The apparatus provided in this application first listens to a specified database to obtain target events to be grouped. Then, it reads the target policy published by the target data table corresponding to the target event from the distributed cache that is refreshed in real time by the data group management platform. Finally, according to the target policy, it determines the target group identifier corresponding to the target data table and binds the target group identifier indicating the grouping to the target event. The grouping policy configuration does not need to intrude into the business layer; it only binds the group identifier to the corresponding data table, thereby reducing business complexity. Furthermore, the data in the distributed cache is refreshed in real time by the data group management platform, enabling immediate grouping of target events upon their occurrence, achieving rapid response, and thus improving the speed and efficiency of grouping.
[0092] It should be noted that other corresponding descriptions of the functional units involved in the data packetization device provided in this application embodiment can be found by referring to... Figures 1A to 1B and Figure 2 The corresponding descriptions in [the document] will not be repeated here.
[0093] In an exemplary embodiment, see Figure 4 Furthermore, a device is provided, comprising a communication bus, a processor, a memory, and a communication interface. It may also include an input / output interface and a display device, wherein the various functional units can communicate with each other via the bus. The memory stores a computer program, and the processor executes the program stored in the memory, performing the data grouping method described in the above embodiments.
[0094] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the data grouping method.
[0095] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented in hardware or by using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) and includes several instructions to cause a computer device (such as a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0096] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing this application.
[0097] Those skilled in the art will understand that the modules in the apparatus of the implementation scenario can be distributed within the apparatus of the implementation scenario as described, or they can be located in one or more apparatuses different from this implementation scenario, with corresponding changes. The modules of the above-described implementation scenario can be combined into one module, or they can be further divided into multiple sub-modules.
[0098] The serial numbers in this application are for descriptive purposes only and do not represent the superiority or inferiority of the implementation scenario.
[0099] The above disclosures are only a few specific implementation scenarios of this application. However, this application is not limited to these. Any variations that can be conceived by those skilled in the art should fall within the protection scope of this application.
Claims
1. A data grouping method, characterized in that, The method is applicable to packet processing platforms and includes: Monitor data in a specified database to obtain target events to be grouped; In the distributed cache, the target strategy published by the target data table corresponding to the target event is read, and the data in the distributed cache comes from the real-time refresh of the data group management platform; According to the target strategy, the target group identifier corresponding to the target data table is determined, and the target group identifier is bound to the target event. The target group identifier is used to indicate the group corresponding to the target data table. When a backtracking instruction is detected, the first specified data table indicated by the backtracking instruction is retrieved according to the backtracking range indicated by the backtracking instruction. Determine the new specified strategy corresponding to the first specified data table. The new specified strategy is a new grouping strategy that the grouping processing platform re-releases for the published target strategy. According to the new specified strategy, the specified identifier corresponding to the first specified data table is determined, and the specified identifier is bound to the first specified data table.
2. The method according to claim 1, characterized in that, Before monitoring the specified database and obtaining the target events to be grouped, the method further includes: The data group management platform connects multiple data tables to the group processing platform, reviews the multiple data tables, and generates multiple review results. For each of the multiple review results, if the review result indicates that it is not approved, a review failure mark is added to the data table corresponding to the review result, and a publication failure prompt is generated and sent to the data group management platform. If the audit result indicates approval, then the strategy for the data table corresponding to the audit result is configured and published, and a publication success message is generated and sent to the data group management platform.
3. The method according to claim 1, characterized in that, The step of determining the target group identifier corresponding to the target data table according to the target strategy includes: In the table definition of the target data table, read the attribute elements corresponding to the target data table; According to the policy rules indicated by the target policy, determine the group identifier that the attribute element hits, and use the group identifier as the target group identifier.
4. The method according to claim 1, characterized in that, The method further includes: When a backtracking instruction is detected, the backtracking instruction is identified to obtain the first specified data table indicated by the backtracking instruction and the backtracking range; Based on the backtracking range, the first specified data table is retrieved using a pagination method with group identifier intervals. A new designation strategy corresponding to the first designated data table is determined. According to the new designation strategy, a designation identifier corresponding to the first designated data table is determined. The designation identifier is bound to the first designated data table. The new designation strategy is the grouping strategy corresponding to the first designated data table that the grouping processing platform republishes in response to the backtracking instruction.
5. A data grouping method, characterized in that, The method is applicable to data group management platforms, including: Policy information is set for each accessed data table, resulting in multiple data tables. These multiple data tables are then accessed to a group processing platform for review and publication. The table definition of each data table includes fixed data group fields and attribute elements. The group processing platform is used to monitor data in a specified database, obtain target events to be grouped, and read the target policy published by the target data table corresponding to the target event from a distributed cache. The data in the distributed cache originates from real-time updates from the data group management platform. According to the target policy, the target group identifier corresponding to the target data table is determined, and the target group identifier is bound to the target event. The target group identifier is used to indicate the group corresponding to the target data table. When a backtracking instruction is detected, the first specified data table indicated by the backtracking instruction is retrieved according to the backtracking range indicated by the backtracking instruction. A new specified policy corresponding to the first specified data table is determined. The new specified policy is a new grouping policy republished by the group processing platform for the already published target policy. According to the new specified policy, a specified identifier corresponding to the first specified data table is determined, and the specified identifier is bound to the first specified data table. When a successful publication message is received, the second designated data table corresponding to the successful publication message is determined, and the designated policy corresponding to the second designated data table is refreshed to the distributed cache. The designated policy is the grouping policy configured by the data group management platform for the second designated data table.
6. The method according to claim 5, characterized in that, The method further includes: When a publishing failure message is received, determine the third designated data table corresponding to the publishing failure message and verify the third designated data table. After verification, the third designated data table is reconnected to the group processing platform for review and publication.
7. A data packetization device, characterized in that, The device is suitable for a packet processing platform and includes: The acquisition module is used to listen to data in a specified database and acquire target events to be grouped. The reading module is used to read the target strategy published in the target data table corresponding to the target event in the distributed cache. The data in the distributed cache comes from the real-time refresh of the data group management platform. The determination module is used to determine the target group identifier corresponding to the target data table according to the target strategy, and bind the target group identifier to the target event. The target group identifier is used to indicate the group corresponding to the target data table. When a backtracking instruction is detected, the module retrieves the first specified data table indicated by the backtracking instruction according to the backtracking range indicated by the backtracking instruction. The module determines a new specified strategy corresponding to the first specified data table. The new specified strategy is a new grouping strategy republished by the group processing platform for the published target strategy. According to the new specified strategy, the module determines the specified identifier corresponding to the first specified data table and binds the specified identifier to the first specified data table.
8. A data packetization device, characterized in that, The device is suitable for a data group management platform and includes: The access module is used to set policy information for each accessed data table, resulting in multiple data tables. These multiple data tables are then accessed to the group processing platform for review and publication. The table definitions of each data table include fixed data group fields and attribute elements. The group processing platform is used to monitor a specified database, obtain target events to be grouped, and read the target policy published by the target data table corresponding to the target event from a distributed cache. The data in the distributed cache originates from real-time updates by the data group management platform. According to the target policy, a target group identifier corresponding to the target data table is determined, and the target group identifier is bound to the target event. The target group identifier indicates the group corresponding to the target data table. When a backtracking instruction is detected, the first specified data table indicated by the backtracking instruction is retrieved according to the backtracking range indicated by the instruction. A new specified policy corresponding to the first specified data table is determined. This new specified policy is a new grouping policy republished by the group processing platform for the already published target policy. According to the new specified policy, a specified identifier corresponding to the first specified data table is determined, and the specified identifier is bound to the first specified data table. The caching module is used to determine the second specified data table corresponding to the successful publication prompt when a successful publication prompt is received, and to refresh the specified policy corresponding to the second specified data table to the distributed cache. The specified policy is the grouping policy configured by the data group management platform for the second specified data table.
9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.
Citation Information
Patent Citations
Data processing method and device
CN111813805A
Request processing method and device, computing equipment and medium
CN114282979A