A statistical information collection method, device and equipment and computer storage medium
By synchronously collecting statistical information in the TA system and utilizing preset scheduling schemes and triggering conditions, the problem of statistical information lag was solved, ensuring the timeliness and accuracy of statistical information and improving the system's operational stability and efficiency.
Patent Information
- Application Number
- CN202311235064.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-22
- Publication Date
- 2025-11-07
- Estimated Expiration
- 2043-09-22
AI Technical Summary
When performing special business, the existing open-ended fund registration and transfer system suffers from a lag in the collection of statistical information, which leads to a mismatch between the statistical information and the actual data distribution, affecting the creation of query plans and even causing the system to crash.
When the TA system performs business operations, statistical information is collected synchronously. Through the target scheduling scheme saved in the preset configuration file, the system adopts pre-collection scheduling, post-collection scheduling or dynamic collection scheduling, and collects statistical information in a timely manner according to the task identifier and triggering conditions.
This avoids mismatches between statistical information and actual data distribution, improves the accuracy of query plans and the efficiency of system clearing, reduces the risk of system downtime, and enhances the user experience.
Smart Images

Figure CN117216129B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of data processing, in particular to a statistical information collection method, device and equipment and computer storage medium. BACKGROUND
[0002] The total data volume involved in an open-end fund registration and transfer (TA) system is very large. In order to quickly perform data processing when performing business operations, the database of the TA system is usually divided into a transaction database and a value-added database; the transaction database is used to store only the data of uncompleted business in the TA system and is used for daily normal business operation, and the value-added database is used to store the data of completed business. The transaction database archives the data of the day to the value-added database every day, so that the transaction database can only store the data of the day every day and does not need to carry a large amount of historical data for data processing, thereby greatly improving the efficiency of daily data processing of the TA system.
[0003] Statistical information describes the distribution of values of certain columns in a data table or index view and belongs to a database object. According to the statistical information, the query optimizer can evaluate the number of rows to be read and the result set during the query process and can also create a high-quality query plan.
[0004] When the TA system performs a system clearing task, the statistical information of the daily data distribution in the transaction database needs to be collected to create a corresponding query plan and quickly complete the clearing of the data in the database.
[0005] However, the general TA system collects statistical information in a fixed time every day, which leads to the situation that if the TA system performs some special business such as newly created products on the day, the collected statistical information is 0, but the actual amount of newly added data exceeds 10,000. This seriously affects the creation of a query plan, causes the execution time of the system clearing to be longer, and even may cause the system to be down.
[0006] Therefore, how to accurately and timely collect statistical information in the TA system is a technical problem to be solved. SUMMARY
[0007] The present application provides a statistical information collection method, device and equipment and computer storage medium to accurately and timely collect statistical information.
[0008] In a first aspect, an embodiment of the present application provides a statistical information collection method applied to a TA system, and the method comprises the following steps.
[0009] receive a task trigger instruction; the task trigger instruction at least comprises an execution task identifier, the task trigger instruction is used to instruct the TA system to execute a system task corresponding to the execution task identifier; in response to the task trigger instruction, obtain a target scheduling scheme corresponding to the execution task identifier from a preset configuration file; wherein the target scheduling scheme at least comprises a target scheduling mode corresponding to the execution task identifier; collect at least one statistical information corresponding to the execution task identifier in the TA system based on the target scheduling mode contained in the target scheduling scheme.
[0010] In the scheme, a corresponding configuration file for instructing the collection of statistical information is configured for each system task to be executed in the TA system in advance, thus ensuring that when the TA system executes any system task, whether the collection of statistical information is needed during the execution of the system task and how to collect the statistical information can be determined based on the preset configuration file. In this way, when the TA system executes a system task that may cause data change, the corresponding statistical information in the process can be collected in time, avoiding the lag of collecting the statistical information at a fixed time. Meanwhile, the confirmation of the collection of statistical information based on the execution task avoids the missing of the collection of statistical information due to the addition of storage addresses or storage files in some special cases.
[0011] Optionally, the collecting of the at least one statistical information corresponding to the execution task identifier in the TA system based on the target scheduling mode contained in the target scheduling scheme comprises: obtaining the identification information of to-be-counted data corresponding to the execution task identifier from the target scheduling scheme; determining the storage address of the statistical information corresponding to the identification information of the to-be-counted data in the TA system based on the identification information of the to-be-counted data; collecting at least one statistical information generated by the TA system during the execution of the system task from the storage address based on the target scheduling mode contained in the target scheduling scheme.
[0012] In the manner, the temporary storage of the statistical information is provided by obtaining the statistical information from the preset storage address, avoiding the problem of loss of the statistical information due to the fact that the statistical information cannot be saved in time after being collected.
[0013] Optionally, the target scheduling mode comprises one or more of the following: a pre-collection scheduling, a post-collection scheduling, or a dynamic-collection scheduling; wherein the pre-collection scheduling is used to indicate that the at least one statistical information is collected after the task trigger instruction is received and before the system task corresponding to the execution task identifier is executed; the post-collection scheduling is used to indicate that the at least one statistical information is collected after the system task corresponding to the execution task identifier is executed; and the dynamic-collection scheduling is used to indicate that the at least one statistical information is collected in real time while the system task corresponding to the execution task identifier is executed.
[0014] In the present mode, different scheduling modes are provided to meet the different collection modes of statistical information required by various system tasks in the TA system, thereby improving the feasibility and flexibility of the present scheme.
[0015] Optionally, the target scheduling scheme further comprises a target trigger mode corresponding to the execution task identifier; and the collecting the at least one statistical information in the TA system corresponding to the execution task identifier comprises: obtaining a collection trigger condition included in the target trigger mode; and collecting at least one statistical information stored in the TA system corresponding to the to-be-counted data indicated in the target scheduling scheme when the TA system meets the collection trigger condition while executing the system task.
[0016] In the present mode, the statistical information is collected only when the corresponding collection trigger condition is met, which not only ensures the comprehensiveness of the statistical information collection, but also reduces the number of times of collecting the statistical information, thereby avoiding reducing the running speed of the TA system and effectively improving the user experience.
[0017] Optionally, the collection trigger condition included in the target trigger mode is that the number of statistical information stored in the TA system at the current time is a preset value; or the collection trigger condition included in the target trigger mode is that the at least one statistical information is directly collected after the collection trigger condition is obtained; or the collection trigger condition included in the target trigger mode is that the number of statistical information generated by the TA system while executing the system task is an integer multiple of a target threshold.
[0018] In the present mode, several different trigger conditions are provided, which can be used to collect the statistical information for different system tasks in the TA system, thereby improving the feasibility and flexibility of the present scheme.
[0019] Optionally, the target scheduling manner at least includes a dynamic collection scheduling, and the collecting the at least one statistical information corresponding to the execution task identifier in the TA system includes: obtaining a collection trigger condition included in the target trigger manner; based on at least one preset time node included in the collection trigger condition, obtaining a completed record number indicating a quantity of the statistical information collected by the TA system at a current time at each preset time node, and calculating a dynamic threshold value for triggering the collection of the statistical information; based on the dynamic threshold value, collecting at least one statistical information corresponding to the to-be-counted data indicated in the target scheduling scheme and stored in the TA system within a time range corresponding to the dynamic threshold value.
[0020] In the manner, the dynamic collection of the statistical information is provided, and the collection of the statistical information is performed correspondingly while the TA system performs the corresponding system task. In this way, the real-time performance of the collection of the statistical information is improved, and the occupation of the computing resources by the collection task according to the fixed time node is avoided.
[0021] In a second aspect, an embodiment of the present application provides a statistical information collection apparatus applied to a TA system, and the apparatus includes: a receiving module configured to receive a task trigger instruction; the task trigger instruction at least includes an execution task identifier, and the task trigger instruction is configured to instruct the TA system to perform a system task corresponding to the execution task identifier; a responding module configured to respond to the task trigger instruction, and obtain a target scheduling scheme corresponding to the execution task identifier from a preset configuration file; wherein the target scheduling scheme at least includes a target scheduling manner corresponding to the execution task identifier; and a collecting module configured to collect at least one statistical information corresponding to the execution task identifier in the TA system based on the target scheduling manner included in the target scheduling scheme.
[0022] Optionally, when the collecting module is configured to collect at least one statistical information corresponding to the execution task identifier in the TA system based on the target scheduling manner included in the target scheduling scheme, the collecting module is specifically configured to: obtain identification information of to-be-counted data corresponding to the execution task identifier from the target scheduling scheme; determine a storage address of the statistical information corresponding to the identification information of the to-be-counted data in the TA system based on the identification information of the to-be-counted data; and collect at least one statistical information generated by the TA system when performing the system task from the storage address based on the target scheduling manner included in the target scheduling scheme.
[0023] Optionally, the target scheduling manner comprises one or more of the following: a pre-collection scheduling, a post-collection scheduling, or a dynamic-collection scheduling; the pre-collection scheduling is used to indicate that the at least one statistical information is collected after the task trigger instruction is received and before the system task corresponding to the execution task identifier is executed; the post-collection scheduling is used to indicate that the at least one statistical information is collected after the system task corresponding to the execution task identifier is executed; and the dynamic-collection scheduling is used to indicate that the at least one statistical information is collected in real time when the system task corresponding to the execution task identifier is executed.
[0024] Optionally, the target scheduling scheme further comprises a target trigger manner corresponding to the execution task identifier; and when the collection module collects the at least one statistical information in the TA system corresponding to the execution task identifier, the collection module is specifically configured to: acquire a collection trigger condition included in the target trigger manner; and the TA system collects at least one statistical information stored in the TA system and corresponding to the to-be-counted data indicated in the target scheduling scheme when the TA system satisfies the collection trigger condition during execution of the system task.
[0025] Optionally, the collection trigger condition included in the target trigger manner is that a quantity of statistical information stored in the TA system at a current time is a preset value; or the collection trigger condition included in the target trigger manner is that the at least one statistical information is directly collected after the collection trigger condition is acquired; or the collection trigger condition included in the target trigger manner is that a quantity of statistical information generated by the TA system during execution of the system task is an integer multiple of a target threshold value.
[0026] Optionally, the target scheduling manner at least comprises a dynamic-collection scheduling, and when the collection module collects the at least one statistical information in the TA system corresponding to the execution task identifier, the collection module is specifically configured to: acquire a collection trigger condition included in the target trigger manner; acquire a completed record number indicating a quantity of the statistical information collected by the TA system at a current time at each preset time node based on at least one preset time node included in the collection trigger condition, and calculate a dynamic threshold value for triggering collection of the statistical information; and collect at least one statistical information stored in the TA system and corresponding to the to-be-counted data indicated in the target scheduling scheme based on the dynamic threshold value within a time range corresponding to the dynamic threshold value.
[0027] In a third aspect, an electronic device is provided, comprising: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the at least one processor, by executing the instructions stored in the memory, causes the at least one processor to perform the method in the first aspect or any of the optional implementation manners of the first aspect.
[0028] In a fourth aspect, a computer readable storage medium is provided, which stores instructions that, when executed, cause the method in the first aspect or any of the optional implementation manners of the first aspect to be implemented.
[0029] In a fifth aspect, a computer program product is provided, which stores instructions that, when executed on a computer, cause the computer to perform the method in the first aspect or any of the optional implementation manners of the first aspect.
[0030] The technical effects or advantages of one or more technical solutions provided in the second, third, fourth and fifth aspects in the embodiments of the present application can be explained by the corresponding one or more technical solutions provided in the first aspect. BRIEF DESCRIPTION OF DRAWINGS
[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor based on these drawings.
[0032] Figure 1 A possible application scenario provided in the embodiments of the present application is shown in the following schematic diagram.
[0033] Figure 2 A flowchart of a statistical information collection method provided in the embodiments of the present application is shown in the following schematic diagram.
[0034] Figure 3 A flowchart of a method for obtaining statistical information provided in the embodiments of the present application is shown in the following schematic diagram.
[0035] Figure 4 A flowchart of a method for collecting statistical information based on a triggering mode provided in the embodiments of the present application is shown in the following schematic diagram.
[0036] Figure 5 A flowchart of a method for collecting statistical information based on dynamic collection scheduling provided in the embodiments of the present application is shown in the following schematic diagram.
[0037] Figure 6A logic diagram of a statistical information collection method provided by an embodiment of the present application;
[0038] Figure 7 A schematic diagram of an execution template of a TA system performing a system task provided by an embodiment of the present application;
[0039] Figure 8 A structural schematic diagram of a statistical information collection device provided by an embodiment of the present application;
[0040] Figure 9 A structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION
[0041] The technical solutions of the present application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific features in the embodiments and the specific features in the embodiments are detailed descriptions of the technical solutions of the present application, and are not limitations of the technical solutions of the present application. In the case of no conflict, the technical features in the embodiments and the embodiments can be combined with each other.
[0042] It should be understood that in the description of the embodiments of the present application, the terms "first", "second", etc. are used only for the purpose of distinguishing the description and cannot be understood as indicating or implying relative importance. Also, it cannot be understood as indicating or implying order. In the description of the embodiments of the present application, "a plurality of" means two or more.
[0043] The term "and / or" in the embodiments of the present application is only a description of the association relationship between the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the three cases of A alone, A and B together, and B alone. In addition, the character " / " in this paper generally represents that the front and rear associated objects are a "or" relationship.
[0044] In the embodiments of the present application, the collection, transmission, use, etc. of data comply with the requirements of relevant national laws and regulations.
[0045] When collecting statistical information, the TA system usually collects statistical information of the data stored in the transaction database in the day at a fixed time at night. Thus, after determining the data distribution in the transaction database according to the statistical information, if a system clearing task needs to be executed, a suitable query plan can be executed according to the data distribution in the statistical information, so as to improve the efficiency of system clearing.
[0046] However, the above statistical information collection has the problem that if there are some special businesses (for example, a fund is established by a new manager, income transfer, etc.) on the day, because there is no related information of the special businesses in the related configuration when the statistical information is collected, the data of the businesses cannot be collected when the statistical information is collected, and thus the collected statistical information may be 0, but the amount of new data is large. In this way, the statistical information does not match the actual data distribution, which may cause a serious error in the creation of a subsequent query plan, and thus the query cannot be performed according to the query plan, and only full data scanning can be performed, which makes the system clearing task take a long time to execute or even appear to be down.
[0047] Therefore, an embodiment of the present application provides a statistical information collection method, which synchronously collects statistical information when a TA system performs a business operation, to avoid the mismatch between the statistical information and the actual data distribution. Specifically, when the TA system performs a system task corresponding to each type of business, at least one statistical information corresponding thereto is collected in the TA system based on a target scheduling scheme of the statistical information corresponding to each system task saved in a preset configuration file.
[0048] The target scheduling scheme includes at least one of the following scheduling modes, that is, a front-end collection scheduling, a rear-end collection scheduling, or a dynamic collection scheduling. The front-end collection scheduling is used to indicate that at least one statistical information is collected before the system task is performed, the rear-end collection scheduling is used to indicate that at least one statistical information is collected after the system task is performed, and the dynamic collection scheduling is used to indicate that at least one statistical information is collected in real time during the performance of the system task.
[0049] Further, the target scheduling scheme includes a corresponding triggering mode, which is used to indicate that the collection of the statistical information is started only when the statistical information in the TA system meets a preset triggering condition.
[0050] The overall idea of the scheme provided by the embodiment of the present application is introduced above, and the specific implementation of the embodiment of the present application will be introduced in each aspect.
[0051] The application scenarios to which the technical scheme of the embodiment of the present application can be applied will be briefly introduced below. It should be noted that the following application scenarios are only used to illustrate the embodiment of the present application, but not to limit it. In the specific implementation process, the technical scheme provided by the embodiment of the present application can be flexibly applied according to actual needs.
[0052] Referring to Figure 1 A possible application scenario provided by the embodiment of the present application is shown in the following figure, which can include a terminal device 101 and a server 102.
[0053] The terminal device 101 can be a mobile phone, a tablet computer (PAD), a personal computer (PC), a wearable device, a vehicle-mounted terminal, and the like, or can be a camera, a video camera, a vehicle recording device, and the like. The server 102 can be a stand-alone physical server, or can be a server cluster or a distributed system formed by multiple physical servers, or can be a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and basic cloud computing services such as big data and artificial intelligence platforms.
[0054] The terminal device 101 and the server 102 can be directly or indirectly connected through one or more communication networks 103. The communication network 103 can be a wired network or a wireless network. For example, the wireless network can be a mobile cellular network or a wireless fidelity (WIFI) network, and can also be other possible networks. The embodiments of the present application do not limit the communication network 103.
[0055] It should be noted that the method for collecting statistical information in the embodiments of the present application can be executed by a computer device, which can be the terminal device 101 or the server 102. For example, when the terminal device 101 has the TA system described above, the terminal device 101 can be the execution subject to collect the corresponding statistical information through the method for collecting statistical information provided by the embodiments of the present application, whenever the TA system executes a corresponding system task. When the method for collecting statistical information provided by the embodiments of the present application is executed by the server 102 as the execution subject, the server 102 can collect the statistical information in the TA system through the method for collecting statistical information provided by the embodiments of the present application.
[0056] It should be understood that Figure 1 It should be understood that
[0057] The method for collecting statistical information provided by the exemplary embodiments of the present application will be described below in combination with the application scenarios described above and the accompanying drawings. It should be noted that the application scenarios described above are only used to facilitate the understanding of the spirit and principles of the present application, and the number and communication modes of the terminal devices and the server are not limited in the embodiments of the present application.
[0058] Referring to Figure 2A flowchart of a statistical information collection method provided by an embodiment of the present application. As described above, the statistical information collection method proposed by the embodiment of the present application can be performed by a terminal device or a server alone, and in order to make the subsequent description simple and clear, the server alone will be taken as the execution subject of the method in the following description. As shown in Figure 2 The method specifically implements the following steps:
[0059] Step S201: receiving a task triggering instruction, the task triggering instruction at least including an execution task identifier, the task triggering instruction being used to instruct the TA system to execute a system task corresponding to the execution task identifier.
[0060] When the TA system performs business processing every day, the server carrying the TA system will correspondingly execute an execution task corresponding to each business processing. Therefore, the execution task identifier used to instruct the TA system to execute a corresponding system task will be included in the task triggering instruction. Exemplarily, the system task required to be executed by the TA system can be data import, account opening, bonus distribution, subscription, redemption, data summarization, etc.
[0061] Step S202: in response to the task triggering instruction, obtaining a target scheduling scheme corresponding to the execution task identifier in the task triggering instruction from a preset configuration file; wherein the target scheduling scheme at least includes a target scheduling mode corresponding to the execution task identifier.
[0062] In the server, a configuration file related to statistical information collection can be stored in advance, and the configuration file includes a target scheduling scheme corresponding to each system task executed by the TA system. The creation of the configuration file can be automatic creation by the server according to a collection rule stored in the server, or can be created by the TA system related operation and maintenance personnel and then imported, and the present application does not limit this.
[0063] And the contents contained in the specific configuration file can be: different execution task identifiers, scheduling schemes corresponding to different execution task identifiers, file identifiers corresponding to the data to be collected, storage addresses corresponding to each file identifier, etc.
[0064] Among them, different execution task identifiers can be the literal name or digital code of the system task required to be executed by the TA system, for example, data import (001), account opening (002), bonus distribution (003), subscription (004), redemption (005), data summarization (006).
[0065] The scheduling scheme corresponding to different task identifiers also has corresponding differences. Specifically, in any scheduling scheme, at least one or more of the following scheduling modes is included: pre-collection scheduling, post-collection scheduling, or dynamic collection scheduling, which are respectively used to instruct the server to collect statistical information before, after, or during the execution of a system task.
[0066] Specifically, the pre-collection scheduling is used to instruct the collection of at least one statistical information after receiving the task trigger instruction and before executing the system task corresponding to the execution task identifier; the post-collection scheduling is used to instruct the collection of at least one statistical information after executing the system task corresponding to the execution task identifier; and the dynamic collection scheduling is used to instruct the real-time collection of at least one statistical information during the execution of the system task corresponding to the execution task identifier.
[0067] For example, in the target scheduling scheme corresponding to the clearing pre-check system task among various system tasks executed by the TA system, only the pre-collection scheduling is included, because it only needs to detect whether each piece of data is compliant, and the data volume does not change after the check is completed, so it only needs to execute the statistical information collection corresponding to the pre-collection scheduling once.
[0068] The file identifier corresponding to the statistical data and the storage address corresponding to each file identifier are also corresponding to the execution task identifier. Because different system tasks are processed based on different data during execution, and these data can be saved in different forms in the database corresponding to the TA system, such as tables, images, and texts. Therefore, when collecting statistical information for different system tasks, the scheduling scheme will also indicate the server to collect statistical information based on the files in the corresponding address through the two items of the file identifier corresponding to the statistical data (such as table name, image name, or text file name, etc.) and the storage address corresponding to each file identifier.
[0069] Therefore, in step S202, when the server responds to the task trigger instruction, it needs to filter out the target scheduling scheme corresponding to the execution task identifier from the scheduling schemes included in the preset configuration file, and the target scheduling scheme will also include the target scheduling mode to be used when the server collects statistical information.
[0070] Step S203: Collect at least one statistical information corresponding to the execution task identifier in the TA system based on the target scheduling mode included in the target scheduling scheme.
[0071] After the server obtains the target scheduling scheme corresponding to the execution task identifier from the configuration file, the server can collect at least one statistical information corresponding to the execution task identifier in the TA system according to the target scheduling mode contained in the target scheduling scheme. As described above, there are three scheduling modes, i.e., pre-collection scheduling, post-collection scheduling and dynamic collection scheduling, and the target scheduling mode in the target scheduling scheme is a possible combination of the three scheduling modes. For example, when pre-collection scheduling and dynamic collection scheduling need to be performed simultaneously, the target scheduling mode contains the identification information of the pre-collection scheduling and the dynamic collection scheduling, and the configuration parameters corresponding to the two scheduling modes corresponding to the execution task identifier. In this way, the server can collect the corresponding statistical information according to the target scheduling mode.
[0072] In the present scheme, a corresponding configuration file for indicating the collection of statistical information is configured for each system task to be executed in the TA system. In this way, it is ensured that when any system task is executed in the TA system, whether the collection of statistical information is needed during the execution of the system task and, if the collection of statistical information is needed, how to collect the statistical information can be determined based on the preset configuration file. In this way, when the TA system executes a system task that may cause data change, the corresponding statistical information in the process can be collected in time, avoiding the lag of collecting statistical information at a fixed time. Meanwhile, the confirmation of the collection of statistical information based on the execution task avoids the omission of the collection of statistical information due to the addition of storage addresses or storage files in some special cases.
[0073] Optionally, when the step S203 is executed, the server can further execute the following mode to ensure the comprehensiveness and accuracy of the collection of statistical information.
[0074] Referring to Figure 3 A flowchart of a method for obtaining statistical information provided by the embodiment of the present application is shown in Figure 3 The specific implementation steps of the method are as follows:
[0075] Step S301: Obtain the identification information of the to-be-collected data corresponding to the execution task identifier from the target scheduling scheme.
[0076] As described above, the corresponding to-be-processed data of the TA system is different when the TA system is executing different system tasks, therefore, when collecting the statistical information, the server needs to obtain the identification information of the to-be-processed data corresponding to the execution task identification from the target scheduling scheme, and then determine the file that needs to collect the statistical information from the database of the TA system. It should be noted that in the process of executing a system task, multiple files can be involved, therefore, in the step S301, the obtained identification information of the to-be-processed data can be more than one, and the server needs to collect at least one statistical information corresponding to the at least one identification information.
[0077] Step S302: determining the storage address of the statistical information corresponding to the identification information of the to-be-processed data in the TA system based on the identification information of the to-be-processed data.
[0078] The server needs to obtain the storage address of the statistical information corresponding to the identification information of the to-be-processed data at the same time of obtaining the identification information of the to-be-processed data. The server statistical information can be pre-stored in the corresponding storage address in the process of running the TA system, and when the server needs to collect the statistical information, the following operations can be performed according to the storage address:
[0079] Step S303: collecting at least one statistical information generated by the TA system when executing the system task from the storage address based on the target scheduling mode contained in the target scheduling scheme.
[0080] After obtaining the storage address of the statistical information, the server can obtain at least one statistical information generated by the TA system when executing the system task from the storage address.
[0081] In this way, the statistical information is obtained from the preset storage address, which provides space for the temporary storage of the statistical information, and avoids the problem of loss of the statistical information due to the fact that the statistical information cannot be saved in time after being collected.
[0082] Optionally, the target scheduling scheme can further include a target triggering mode corresponding to the execution task identification. The target triggering mode is used to indicate the execution condition of the server for collecting the statistical information.
[0083] Referring to Figure 4 A flowchart of a method for collecting statistical information based on a triggering mode provided by an embodiment of the present application is shown in Figure 4 The embodiment of the present application provides a method for collecting statistical information based on a triggering mode, and the method comprises the following steps:
[0084] Step S401: obtaining a collection triggering condition contained in a target triggering mode.
[0085] The collection trigger condition can include one of the following trigger conditions:
[0086] Firstly, the number of the statistical information stored in the TA system at the current time is a preset value, wherein the current time is the time when the server acquires the trigger condition, and the preset value is a threshold, and the specific value of the threshold can be 0. When the preset value is 0, the trigger condition can be referred to as a "zero trigger".
[0087] Secondly, after acquiring the collection trigger condition, the server directly starts to collect the corresponding statistical information regardless of the number of the statistical information stored in the TA system at the current time.
[0088] In the trigger condition, the number of the statistical information stored in the TA system at the current time is a preset value, wherein the current time is the time when the server acquires the trigger condition, and the preset value is a threshold, and the specific value of the threshold can be 0. When the preset value is 0, the trigger condition can be referred to as a "zero trigger".
[0089] Thirdly, the number of the statistical information generated by the TA system when performing the system task is an integer multiple of the target threshold, for example, when the number of the statistical information is 10, 20 and 30, the server needs to start to collect the statistical information correspondingly. The trigger condition can be referred to as a "progressive trigger".
[0090] Step S402: When the TA system satisfies the collection trigger condition when performing the system task, at least one statistical information corresponding to the to-be-counted data indicated in the target scheduling scheme is collected.
[0091] After acquiring the corresponding target scheduling scheme from the preset configuration file, the server can collect the statistical information according to the target scheduling mode in the target scheduling scheme and the corresponding target trigger mode. The scheduling mode in the target scheduling mode and the trigger condition in the target trigger mode can be combined in any manner, for example, the front-end collection scheduling mode can be combined with the "zero trigger" trigger condition, the back-end collection scheduling mode can be combined with the "forced trigger" trigger condition, or both can be combined with the "progressive trigger" trigger condition. It should be noted that the above combinations are only used for illustration and not limitation, and other combinations can also exist in actual applications.
[0092] Among the above combinations, one combination needs to be described in more detail, that is, the dynamic collection scheduling mode combined with the progressive trigger.
[0093] Optionally, when the target scheduling mode at least includes the dynamic collection scheduling, the collection of the statistical information needs to perform the following operations:
[0094] Referring to Figure 5A flowchart of a method for collecting statistical information based on dynamic collection scheduling is provided for the embodiments of the present application. As shown in Figure 5 the method specifically implements the following steps:
[0095] Step S501: Obtain the collection trigger condition contained in the target trigger mode.
[0096] Step S502: Based on at least one preset time node contained in the collection trigger condition, obtain the completed record number indicating the number of statistical information collected by the TA system at the current time at each preset time node, and calculate the dynamic threshold for triggering the collection of statistical information.
[0097] Specifically, the server needs to obtain the number of statistical information corresponding to the execution task identifier recorded in the TA system at the current time at the preset time node, i.e., the completed record number. Then, based on the completed record number and the estimated total record number contained in the collection trigger condition, the dynamic threshold for triggering the collection of statistical information is calculated.
[0098] The preset time node can be several time points with fixed time intervals, such as the time points of 5 milliseconds, 10 milliseconds, and 15 milliseconds when the system task is started to be executed, or the time points when the server executes certain events, such as the time corresponding to the completion of a fixed number of segments when recording the corresponding statistical information of the data table.
[0099] Step S503: Based on the above dynamic threshold, collect at least one statistical information corresponding to the data to be counted indicated in the target scheduling scheme stored in the TA system within the time range corresponding to the dynamic threshold.
[0100] For example, when the server executes the collection of statistical information corresponding to the dynamic collection scheduling mode, the server can execute as follows:
[0101] First, obtain the collection trigger condition contained in the target trigger mode, and determine the trigger mode and at least one time node contained in the collection trigger condition.
[0102] Then, obtain the completed record number of the TA system at the current time at each preset time node, and calculate the dynamic threshold according to the completed record number.
[0103] For example, let the time nodes be T0, T1, T2…T n , and the completed record numbers recorded at each preset time node be X0, X1, X2…X n; wherein, X0 represents the number of completed tasks recorded in the TA system when starting to execute the system task. Thus, when collecting the statistical data, the server first needs to determine whether X0 is greater than the preset trigger base number (in actual application, the general trigger technique can be 2000), if not, the actually used base number is the preset trigger base number, if greater than the trigger base number, the value of X0 is used as the actually used base number. After determining the base number, the calculation of the dynamic threshold is to calculate the base number and the corresponding progressive multiplier at each preset time node.
[0104] As described above, the time node when the first time needs to determine whether to trigger the collection of statistical information is T1, at this time, it is needed to determine whether the number of newly added statistical information (X1-X0) is greater than X0, if not, it is determined that the collection of statistical information is not needed at the time node;
[0105] If it is determined that (X1-X0) is greater than X0, it is further needed to determine whether (X1-X0) is greater than or equal to the dynamic threshold D1 (base number*progressive multiplier) corresponding to the current time node T1;
[0106] If not, it is determined that the collection of statistical information is not needed at the time node; if greater than or equal to, the newly added statistical information in the time range (T1-T0) corresponding to the dynamic threshold needs to be collected at the current time node.
[0107] Similarly, when the second time needs to determine whether to trigger the collection of statistical information, the time node T2, it is also needed to determine whether the total number of newly added statistical information at the current time (X2-X0) is greater than X0, if not, it is also determined that the collection of statistical information is not needed at the time node;
[0108] Otherwise, it is also needed to determine whether the current collection of statistical information is the first or the second collection of statistical information, if the second collection, it is determined that the dynamic threshold D2 corresponding to the current time is (base number*2*progressive multiplier), and it is further needed to determine whether (X2-X0) is greater than or equal to the dynamic threshold D2 corresponding to the current time;
[0109] If not, it is determined that the collection of statistical information is not needed at the time node; if greater than or equal to, the newly added statistical information in the time range (T2-T1) corresponding to the dynamic threshold needs to be collected at the current time node.
[0110] In this way, at the next time nodes, the server can complete the collection of statistical information according to the above-mentioned manner.
[0111] The above dynamic collection and scheduling scheme can also be combined into a zero-triggering triggering method. The specific implementation of this combination scheme can refer to the implementation method of the above scheme. The difference is that it only needs to collect statistical information once when it is determined that (X1-X0) is greater than or equal to D0.
[0112] Optionally, when collecting statistical information, the server also needs to determine the data source type in the system task corresponding to each execution task identifier. If the type is all data sources, the server needs to query all sub-databases in the transaction database of the TA system and collect statistical information for each sub-database separately; if the type is only the current data source, statistical information is collected only for the current database. Correspondingly, when the server needs to collect statistical information for multiple databases separately, a multi-threaded parallel approach can be adopted, with one thread corresponding to one database, to complete the collection of statistical information for all databases.
[0113] When collecting statistical information, the server uses the following Structured Query Language (SQL) statements:
[0114] The SQL statement is: {call dbms_stats.gather_table_stats(ownname=>user,tabname=>filename,no_invalidate=>false,cascade=>True,force=>true,degree=>4)}.
[0115] Additionally, when statistics are locked, the server does not need to collect the locked statistics.
[0116] See Figure 6 This is a logical schematic diagram of a statistical information collection method provided in an embodiment of this application. After the system receives a task trigger instruction, the server initiates statistical information collection in response to the task trigger instruction. Figure 6 As shown, the server first needs to determine the corresponding scheduling scheme, identifying which one or a combination of pre-collection scheduling, post-collection scheduling, and dynamic collection scheduling it is. Then, it needs to determine the triggering method for each scheduling method: zero triggering, forced triggering, or progressive triggering. Finally, based on the above scheduling scheme and triggering method, the statistical information collection in the TA system can be completed.
[0117] The above introduces a statistical information collection method provided by the embodiment of the application. In order to facilitate understanding of the specific application of the method in a TA system, the application of the method in the TA system will be specifically described below in combination with the specific services of the TA system in the manner of the above combinations.
[0118] Referring to Figure 7 FIG. 1 is a schematic diagram of an execution template for executing a system task of a TA system provided by the embodiment of the application. As shown in the figure, the execution template can adapt to the execution flow of all system tasks in the TA system. Wherein, the corresponding explanations of each step are as follows: Figure 7
[0119] 1. Obtaining task parameters. The parameters include: an execution task identifier, and a target scheduling scheme corresponding to the execution task identifier.
[0120] 2. Pre-processing, single thread. Performing a pre-processing step of the system, and preparing data.
[0121] 3. Pre-collection of statistical information. Adding a statistical information module here, obtaining parameter configurations that need to be collected according to the execution task identifier and the target scheduling scheme, and initiating pre-statistical information collection scheduling;
[0122] 4. Segmentation of business processing related tables. After the statistical information collection of the business processing related tables is completed, the tables are segmented according to 2000 pieces per batch according to the table data volume, and the batches are saved to a clearing task pair list;
[0123] 5. Initiating task execution messages. Sending a redis message to a clearing task processing receiver, the receiver receiving the message to obtain tasks to be processed from a clearing task queue, performing TA business processing, and updating the task execution state of the clearing task queue list. The state is divided into 01-pending processing, 02-executing, 03-execution success, and 04-execution failure.
[0124] 6. Step processor polling task execution results. Polling the clearing task queue execution state summary information every 1 minute, if the number of completed tasks remains unchanged for more than a threshold value, sending a short message to notify the technical department, if the number of task execution failures is greater than zero, then interrupting the step, throwing an exception, and task execution failure; if the number of tasks to be executed is greater than 0 and the number of completed tasks exceeds a certain threshold value, then updating the processing progress bar; if the number of incomplete tasks = 0, it indicates that all clearing tasks are executed, and the polling is ended.
[0125] 7. Dynamic collection of statistical information. When the number of tasks to be executed in the clearing task queue is greater than 0, initiating dynamic collection of statistical information processing, inputting the total number of segments, the number of completed segments, and calculating whether the number of result table records exceeds the trigger value of the table configuration. If it exceeds, initiate statistics, which is consistent with the above statistical strategy.
[0126] 8. Post-processing statistical information collection. For some business operations, the saved result table will be queried later. After the step processor finishes its polling task, post-processing statistical information collection is initiated. Based on the step processor and the current scheduling scheme name-AF, the parameter configuration of the table to be statistically analyzed is obtained, and the post-processing statistical information collection schedule is initiated.
[0127] 9. Subsequent processing by the step processor. After the statistical information is collected, the process of updating the status of the relevant tables is initiated.
[0128] like Figure 7 The scheduling configuration shown describes the embedding location and usage of the statistical information collection strategy within the business processing flow of the TA system. Configuring the statistical information collection module within a unified processing template reduces the need for extensive modifications to the TA business processing flow. Furthermore, compared to periodically collecting statistical information, it is more compatible with the TA business processing system, reducing resource waste. Since statistical tasks are not a primary function, they cannot preempt resources from operational processes. Moreover, through three different collection methods—pre-collection, dynamic collection, and post-collection—it covers almost all business scenarios, saving manpower and time costs associated with collecting statistical information. If new business scenarios are added later, statistical information collection can be completed simply by adding configuration table information, demonstrating strong scalability, greater user-friendliness for technical personnel, and low code intrusion.
[0129] The above describes the task execution templates for each system task in the TA system. For different services in the TA system, this application also provides corresponding statistical information scheduling schemes as examples.
[0130] 1. Daily Start: The daily start step deletes business data processed yesterday, including account applications and transaction applications, without requiring configuration of statistical information collection.
[0131] 2. Net Asset Value Entry: The net asset value entry process only involves net asset value and fund status. Each fund has one data entry per day, and the data volume is small, so there is no need to configure statistical information collection.
[0132] 3. Net Asset Value Verification: The steps for net asset value verification are the same as those for net asset value entry. Since the amount of data is small, there is no need to configure statistical information collection.
[0133] 4. Net Asset Value Export: The net asset value export step only queries the fund information configuration related tables, the data volume is small, and there is no need to configure the collection of statistical information.
[0134] 5. Data Import: The data import step moves the data from the parsed file's landing table to the application table. This involves large table migration, but this task is a common processing component. The data is not queried in this step after it is saved to the application table, so there is no need to configure statistical information collection in this step.
[0135] 6. Account clearing: the business processing logic of the account opening business in the account clearing is: first segment the account application table record by 2000 pieces per segment, query the application list of each segment, and simultaneously cache the fund account table information and the transaction account table information that have been opened, if the account has been processed, save it to the account confirmation table for opening failure, if the account has not been opened and meets the opening conditions, add the fund account table information and the transaction account table information, and save it to the account confirmation table for opening success.
[0136] According to the business processing logic, the statistical information collection is divided into three scenarios:
[0137] a) Pre-collection of statistical information before account clearing: configure the pre-collection of statistical information of the account application table, cumulative trigger, trigger value is 20000, when the current statistical information of the account application table is less than the trigger value, initiate collection. Avoid subsequent query of the account application table with low efficiency.
[0138] b) Dynamic collection of statistical information in the opening process: configure the dynamic collection of statistical information of the fund account table and the transaction account table, cumulative trigger, trigger value is 20000, after processing each batch, estimate the number of new records and the total number of records according to the segmented table, if the number of records of the account table information is greater than the trigger value, initiate collection of the statistical information of the fund account table and the transaction account table.
[0139] c) Post-collection of statistical information after account clearing: configure the post-collection of statistical information of the account confirmation table, cumulative trigger, trigger value is 20000, after all account applications are processed, if the current statistical information of the account confirmation table is less than the trigger value, initiate collection. Avoid time-consuming of the subsequent processing steps with query of the account confirmation table.
[0140] 7. Clearing pre-check: the processing logic of the clearing pre-check is: first segment the transaction application table record by 2000 pieces per segment, query the application list of each segment, check whether each application record meets the requirements, if not, update the application status to failure, if yes, update it to pre-check success.
[0141] According to the business processing logic, the statistical information collection only needs pre-collection: configure the pre-collection of statistical information of the transaction application table, cumulative trigger, trigger value is 20000, when the current statistical information of the transaction application table is less than the trigger value, initiate collection. Since the subsequent data volume will not change after the table is changed, only collect once.
[0142] 8. Clearing processing: the business processing logic of the purchase business in the clearing processing is: segment the transaction application table by 2000 pieces, query the segmented application list, and cache the available shares obtained from the transaction confirmation table of the day, if each purchase transaction application is processed successfully, save the transaction confirmation table as success, otherwise save the transaction confirmation table as failure.
[0143] According to the business processing logic, the statistical information collection is configured to be collected dynamically: firstly, the statistical information of the transaction application form has been collected in the previous step and does not need to be collected. The statistical information of the transaction confirmation form is configured to be collected dynamically, the progressive trigger is 20000, after each batch of the processing of the subscription is completed, the newly added record number and the total record number of the transaction confirmation form are estimated according to the segmented table, and if the record number is greater than the calculated trigger value, the collection of the statistical information of the transaction account confirmation form is initiated.
[0144] 9. Data aggregation: the processing logic of the data aggregation of the subscription business is that the transaction confirmation table is segmented by 2000, the segmented confirmation list is queried, the existing share data is cached, the existing share is updated or the non-existing share is newly added when each transaction confirmation is processed, and is saved to the share table and the share change log table is newly added for each transaction confirmation record.
[0145] According to the business processing logic, the statistical information collection is divided into three scenarios:
[0146] a) Pre-processing of data aggregation, pre-collection of statistical information: the transaction confirmation table is configured to pre-collect statistical information, the trigger is zero, and when the statistical information of the transaction confirmation table is zero, the collection is initiated. Because the statistical information of the transaction confirmation table is zero only when the statistical information of the transaction confirmation table is not collected in the previous step, if the statistical information of the transaction confirmation table has been collected in the previous step, the step does not collect the statistical information, and the record number of the subsequent confirmation table does not change, so the statistical information needs to be collected only once.
[0147] b) Update of share, dynamic collection of statistical information: the share aggregation table is configured to dynamically collect statistical information, the progressive trigger is 20000, after each batch of the processing of the confirmation table is completed, the newly added record number and the total record number of the share table are estimated according to the segmented table, and if the record number of the share table is greater than the trigger value, the collection of the statistical information of the share table is initiated. The efficiency of the caching of the share is reduced.
[0148] c) Post-processing of data aggregation, post-collection of statistical information: the share change log table is configured to post-collect statistical information, the trigger is zero, and after the processing of all the transaction confirmations is completed, if the statistical information of the share change log table is zero, the collection is initiated. Because the table is not used in the subsequent steps and the data volume of the table is large, the collection takes a long time, and only the first time the statistical information is zero, the collection is needed, and the efficiency can be ensured by the regular collection every night.
[0149] 10. Confirmation export: the statistical information of the related table has been collected in the previous step, and the statistical information collection does not need to be configured.
[0150] 11. Confirmation export: the statistical information of the related table has been collected in the previous step, and the statistical information collection does not need to be configured.
[0151] 12. End of day: The end of day step archives the data in the transaction library without the need to configure statistics collection.
[0152] By analyzing the execution logic of each service and the configuration mode of the statistical information, the statistical time and resource consumption are saved, and the system execution efficiency is improved.
[0153] Based on the same inventive concept, the embodiment of the present application also provides a statistical information collection device.
[0154] Referring to Figure 8 For the embodiment of the present application, a statistical information collection device is provided. The device can be a terminal device or a chip or integrated circuit in the device, and the device includes a module / unit / technical means for executing the method performed by the terminal device in the above method embodiment.
[0155] Exemplarily, the device 800 includes:
[0156] The receiving module 801 is configured to receive a task trigger instruction. The task trigger instruction at least includes an execution task identifier. The task trigger instruction is used to instruct the TA system to execute a system task corresponding to the execution task identifier.
[0157] The response module 802 is configured to, in response to the task trigger instruction, acquire a target scheduling scheme corresponding to the execution task identifier from a preset configuration file. The target scheduling scheme at least includes a target scheduling mode corresponding to the execution task identifier.
[0158] The collection module 803 is configured to collect at least one statistical information corresponding to the execution task identifier in the TA system based on the target scheduling mode included in the target scheduling scheme.
[0159] As an embodiment, Figure 8 The device discussed can be used to execute the method discussed in the above embodiment, and thus the functions and the like that can be achieved by each functional module of the device can refer to the description of the method in the above embodiment. Figure 2 The functions and the like that can be achieved by each functional module of the device can refer to the description of the embodiment of the method discussed in the above embodiment, and thus will not be repeated here. Figure 2
[0160] It should be noted that although several modules or sub-modules of the device are mentioned in the above detailed description, such division is only exemplary and not mandatory. In fact, according to the embodiment of the present application, the features and functions of two or more units described above can be embodied in one module. Conversely, the features and functions of one module described above can be further divided into several modules embodied.
[0161] As a possible product form of the above device, referring to Figure 9 This application also provides an electronic device 900, comprising:
[0162] At least one processor 901; and a communication interface 903 communicatively connected to the at least one processor 901; the at least one processor 901 causes the electronic device 900 to perform the method steps performed by any device in the above method embodiments through the communication interface 903 by executing instructions stored in the memory 902.
[0163] Optionally, the memory 902 is located outside the electronic device 900.
[0164] Optionally, the electronic device 900 includes the memory 902, which is connected to the at least one processor 901, and stores instructions executable by the at least one processor 901. (Appendix) Figure 9 The dashed line indicates that memory 902 is optional for electronic device 900.
[0165] The processor 901 and the memory 902 can be coupled through an interface circuit or integrated together; no restriction is imposed here.
[0166] This application embodiment does not limit the specific connection medium between the processor 901, memory 902, and communication interface 903. This application embodiment... Figure 9 The processor 901, memory 902, and communication interface 903 are connected via a bus 904. Figure 9 The connections between other components are shown in bold and are for illustrative purposes only, not as limiting information. The bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, Figure 9 The text uses only a single thick line to represent a bus, but this does not imply that there is only one bus or one type of bus. It should be understood that the processor mentioned in the embodiments of this application can be implemented in hardware or software. When implemented in hardware, the processor can be a logic circuit, integrated circuit, etc. When implemented in software, the processor can be a general-purpose processor, implemented by reading software code stored in memory.
[0167] The processor can be, for example, a Central Processing Unit (CPU), a general-purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or the like. The general-purpose processor can be a microprocessor, or the processor can be any conventional processor, etc.
[0168] It should be understood that the memory mentioned in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a Random Access Memory (RAM) used as an external cache. By way of example and not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).
[0169] It should be noted that when the processor is a general-purpose processor, a DSP, an ASIC, an FPGA, or other programmable logic device, a discrete gate or transistor logic, a discrete hardware component, the memory (storage module) can be integrated in the processor.
[0170] It should be noted that the memory described herein is intended to include, but not limited to, these and any other suitable types of memory.
[0171] As another possible product form, the embodiments of the present application also provide a computer readable storage medium for storing instructions, which, when executed, cause a computer to perform the method steps performed by any of the devices in the above method embodiments.
[0172] As another possible product form, the embodiments of the present application also provide a computer program product containing instructions, which, when run on a computer, cause the computer to perform the method steps performed by any of the devices in the above method embodiments.
[0173] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) containing computer usable program code.
[0174] The present application is described with reference to the flowcharts and / or block diagrams of the methods, apparatus (systems) and computer program products according to this application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions that are executed by the processor of the computer or other programmable data processing apparatus generate an apparatus that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in a flow or multiple flows and / or blocks Figure 1 The functions specified in a flow or multiple flows and / or blocks
[0175] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction apparatus that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in a flow or multiple flows and / or blocks Figure 1 The functions specified in a flow or multiple flows and / or blocks
[0176] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operational steps are performed on the computer or other programmable data processing apparatus to produce a computer implemented process, so that the instructions executed on the computer or other programmable data processing apparatus provide a process for implementing the functions specified in the flowcharts and / or block diagrams.Figure 1 one or more processes and / or functions specified in one or more blocks Figure 1 one or more processes and / or functions specified in one or more blocks
[0177] It will be apparent to those skilled in the art that various modifications and variations can be made to the present application without departing from the spirit or scope of the application. Thus, it is intended that the present application cover modifications and variations of this application provided they come within the scope of the appended claims and their equivalents.
Claims
1. A method of collecting statistical information, characterized by, The method is applied to an open fund registration and transfer TA system, and comprises the following steps: receiving a task triggering instruction; the task triggering instruction at least comprises an execution task identifier, and the task triggering instruction is used to instruct the TA system to execute a system task corresponding to the execution task identifier; in response to the task triggering instruction, executing the system task corresponding to the execution task identifier, and obtaining a target scheduling scheme corresponding to the execution task identifier from a preset configuration file; the target scheduling scheme at least comprises a target scheduling mode corresponding to the execution task identifier; the target scheduling mode comprises one or more of the following: a pre-collection scheduling, a post-collection scheduling, and a dynamic collection scheduling, which are determined based on a timing of executing the system task by the TA system; collecting at least one statistical information corresponding to the execution task identifier in the TA system based on the target scheduling mode included in the target scheduling scheme; when the target scheduling mode is the dynamic collection scheduling mode, the collecting of the at least one statistical information corresponding to the execution task identifier in the TA system comprises: obtaining a collection triggering condition included in a target triggering mode in the target scheduling scheme; based on at least one preset time node included in the collection triggering condition, obtaining a completed record number at each preset time node, which is used to indicate a quantity of the statistical information collected at the current time by the TA system, and calculating a triggering base and a progressive multiplier at the corresponding preset time based on the completed record number; based on the triggering base and the progressive multiplier, determining a dynamic threshold for triggering the collection of the statistical information, and based on the dynamic threshold, collecting at least one statistical information corresponding to the to-be-counted data indicated in the target scheduling scheme and stored in the TA system within a time range corresponding to the dynamic threshold.
2. The method of claim 1, wherein, The collecting of the at least one statistical information corresponding to the execution task identifier in the TA system based on the target scheduling mode included in the target scheduling scheme comprises: obtaining identification information of to-be-counted data corresponding to the execution task identifier from the target scheduling scheme; based on the identification information of the to-be-counted data, determining a storage address of the statistical information corresponding to the identification information of the to-be-counted data in the TA system; based on the target scheduling mode included in the target scheduling scheme, collecting at least one statistical information generated by the TA system when executing the system task from the storage address.
3. The method of claim 1 or 2, wherein: the pre-collection scheduling is used to instruct to collect the at least one statistical information after receiving the task triggering instruction and before executing the system task corresponding to the execution task identifier; the post-collection scheduling is used to instruct to collect the at least one statistical information after executing the system task corresponding to the execution task identifier; the dynamic collection scheduling is used to instruct to collect the at least one statistical information in real time when executing the system task corresponding to the execution task identifier.
4. The method of claim 1, wherein, The target scheduling scheme further includes a target triggering mode corresponding to the execution task identifier; The collection of the at least one statistical information in the TA system corresponding to the execution task identifier includes: An acquisition of a collection triggering condition included in the target triggering mode; When the TA system satisfies the collection triggering condition during the execution of the system task, at least one statistical information corresponding to the to-be-counted data indicated in the target scheduling scheme and stored in the TA system is collected.
5. The method of claim 4, wherein, The collection triggering condition included in the target triggering mode is that the number of statistical information stored in the TA system at the current time is a preset value; or, The collection triggering condition included in the target triggering mode is that the at least one statistical information is directly collected after the acquisition of the collection triggering condition; or, The collection triggering condition included in the target triggering mode is that the number of statistical information generated by the TA system during the execution of the system task is an integer multiple of a target threshold.
6. An apparatus for collecting statistical information, characterized by comprising: The device is applied to an open-end fund registration and transfer TA system, and includes: A receiving module configured to receive a task triggering instruction; the task triggering instruction at least includes an execution task identifier, and the task triggering instruction is used to instruct the TA system to execute a system task corresponding to the execution task identifier; A responding module configured to, in response to the task triggering instruction, execute the system task corresponding to the execution task identifier, and acquire a target scheduling scheme corresponding to the execution task identifier from a preset configuration file; wherein the target scheduling scheme at least includes a target scheduling mode corresponding to the execution task identifier; the target scheduling mode includes one or more of the following contents: a pre-collection scheduling, a post-collection scheduling, and a dynamic collection scheduling, which are determined based on a timing of the execution of the system task by the TA system; A collecting module configured to collect at least one statistical information in the TA system corresponding to the execution task identifier based on a target scheduling mode included in the target scheduling scheme; When the target scheduling mode is the dynamic collection scheduling mode, the collecting module is configured to, when collecting the at least one statistical information in the TA system corresponding to the execution task identifier, acquire a collection triggering condition included in a target triggering mode in the target scheduling scheme, acquire a completed record number used to indicate a number of the statistical information collected by the TA system at the current time at each preset time node based on at least one preset time node included in the collection triggering condition, calculate a triggering base number and a cumulative multiplier at the corresponding preset time based on the completed record number, determine a dynamic threshold used to trigger the collection of the statistical information based on the triggering base number and the cumulative multiplier, and collect at least one statistical information corresponding to the to-be-counted data indicated in the target scheduling scheme and stored in the TA system within a time range corresponding to the dynamic threshold based on the dynamic threshold.
7. The apparatus of claim 6, wherein, The collection module is configured to collect at least one statistical information corresponding to the execution task identifier in the TA system based on a target scheduling manner included in the target scheduling scheme, and specifically configured to: obtain, from the target scheduling scheme, identification information of to-be-counted data corresponding to the execution task identifier; determine, based on the identification information of the to-be-counted data, a storage address of statistical information corresponding to the identification information of the to-be-counted data in the TA system; collect, based on the target scheduling manner included in the target scheduling scheme, at least one statistical information generated by the TA system when executing the system task from the storage address.
8. The apparatus of claim 6 or 7, wherein the pre-collection schedule is configured to instruct to collect the at least one statistical information before receiving the task trigger instruction and before executing the system task corresponding to the execution task identifier; the post-collection schedule is configured to instruct to collect the at least one statistical information after executing the system task corresponding to the execution task identifier; and the dynamic-collection schedule is configured to instruct to collect the at least one statistical information in real time when executing the system task corresponding to the execution task identifier.
9. The apparatus of claim 6, wherein, The target scheduling scheme further includes a target trigger manner corresponding to the execution task identifier. The collection module is configured to collect at least one statistical information corresponding to the execution task identifier in the TA system based on a target scheduling manner included in the target scheduling scheme, and specifically configured to: obtain a collection trigger condition included in the target trigger manner; collect at least one statistical information corresponding to to-be-counted data indicated in the target scheduling scheme and stored in the TA system when the TA system satisfies the collection trigger condition when executing the system task.
10. The apparatus of claim 9, wherein, The collection trigger condition included in the target trigger manner is that the number of statistical information stored by the TA system at the current time is a preset value; or The collection trigger condition included in the target trigger manner is that the at least one statistical information is directly collected after the collection trigger condition is obtained; or The collection trigger condition included in the target trigger manner is that the number of statistical information generated by the TA system when executing the system task is an integer multiple of a target threshold.
11. An electronic device, comprising: comprise: at least one processor; and a memory in communication with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the at least one processor, by executing the instructions stored in the memory, performs the method of any one of claims 1-5.
12. A computer-readable storage medium, characterized in that, The computer-readable storage medium is configured to store instructions, which, when executed, cause the method of any one of claims 1-5 to be implemented.
13. A computer program product comprising instructions, characterized in that, The computer program product stores instructions, which, when executed on a computer, cause the computer to perform the method of any one of claims 1-5.
Citation Information
Patent Citations
Method and device for realizing statistical information collection
CN106339244A
Task scheduling method and device, computer equipment and storage medium
CN109743390A
Database statistical information management method, management platform and storage medium
CN116483835A