Methods, systems, electronic devices, and media for cleaning and processing pharmaceutical distribution data

Abstract

This disclosure provides a method, system, electronic device, and medium for cleaning and processing drug distribution data. The cleaning and processing method includes: in response to processing several initial drug distribution data sets using a database server, sampling to obtain usage rate data of a preset processor at different sampling times; acquiring the current usage rate change status data of the preset processor; based on the usage rate change status data, generating a matching task processing adjustment strategy to dynamically adjust the number threshold of concurrent channels used for task processing in the preset processor; and using the adjusted preset processor to perform data cleaning processing to obtain standardized target drug distribution data. This disclosure utilizes the high-performance resource configuration and stronger task concurrency processing capabilities of a database server to improve the system's low resource utilization and enhance system stability; it also includes CPU stress identification and adjustment functions, improving data processing efficiency, completion rate, and system stability.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and in particular to a method, system, electronic device, and medium for cleaning and processing pharmaceutical distribution data. Background Technology

[0002] Currently, the pharmaceutical retail industry is facing immense competitive pressure and changing consumer demands. As overall product retail prices decline, the pressure on drug production and distribution is also increasing. For pharmaceutical manufacturers and distributors, this necessitates meticulous management of marketing costs, requiring a high level of efficiency. This translates to a need for information management systems in the drug distribution process that are highly efficient and cost-effective.

[0003] However, existing solutions for information management and data services in the pharmaceutical distribution process generally suffer from problems such as insufficient system stability, low system resource utilization, and low efficiency in system function cleaning, thus failing to meet the high requirements of actual scenarios. Summary of the Invention

[0004] The technical problem to be solved by this disclosure is to overcome the defects of existing drug distribution data cleaning and processing solutions, such as insufficient system stability, low system resource utilization, and low system function cleaning efficiency. The purpose is to provide a drug distribution data cleaning and processing method, system, electronic device, and medium.

[0005] This disclosure solves the above-mentioned technical problems through the following technical solution:

[0006] In a first aspect, this disclosure provides a method for cleaning and processing pharmaceutical distribution data. The cleaning and processing method is implemented based on a database server in a data cleaning and processing system, and the database server is configured with preset data cleaning tools.

[0007] The cleaning method includes:

[0008] In response to the use of the database server to process several initial drug distribution data, the usage rate data of the preset processor in the database server at different sampling times is obtained based on the preset sampling frequency.

[0009] Based on the usage rate data at different sampling times, obtain the current usage rate change status data of the preset processor;

[0010] Based on the usage rate change data, a matching task processing adjustment strategy is generated.

[0011] Based on the task processing adjustment strategy, the number threshold of concurrent channels used for task processing in the preset processor is dynamically adapted and adjusted.

[0012] The adjusted preset processor performs data cleaning processing according to the preset data cleaning tool to obtain standardized target drug distribution data after cleaning each initial drug distribution data.

[0013] Optionally, the step of obtaining the current usage rate change status data of the preset processor based on the usage rate data at different sampling times includes:

[0014] The difference in the usage rate data corresponding to two consecutive adjacent sampling times of the preset processor is calculated;

[0015] The ratio between the difference and the time interval corresponding to two adjacent consecutive sampling times is obtained as the current usage rate change status data of the preset processor.

[0016] Optionally, the step of generating a matching task processing adjustment strategy based on the usage rate change status data includes:

[0017] In response to the decreasing trend of the usage rate change status data, based on the current threshold number of concurrent channels in the preset processor, a first number of concurrent channels are added;

[0018] In response to the usage rate change status data indicating a stable or upward trend, an upward slope less than a first preset value, and the usage rate data at the corresponding sampling time being less than a second preset value, a threshold for maintaining the current number of concurrent channels in the preset processor is determined.

[0019] In response to the usage rate change status data showing an upward trend with an upward slope greater than or equal to the first preset value; and / or the usage rate data at the corresponding sampling time being greater than or equal to the second preset value, based on the current number threshold of concurrent channels in the preset processor, the second number of concurrent channels is reduced;

[0020] In response to the usage rate change status data showing an upward trend, and the usage rate data at the corresponding sampling time being greater than a third preset value, it is determined to clear the concurrent channel in the preset processor; wherein, the third preset value is greater than the second preset value.

[0021] Optionally, the step of increasing the number of concurrent channels by a first number based on the current threshold number of concurrent channels in the preset processor includes:

[0022] Based on the degree of decline in the different usage rate change status data, and based on the pre-configured first mapping relationship, the first number of concurrent channels in the preset processor is increased to match the first number of concurrent channels.

[0023] The degree of the downward trend is positively correlated with the first quantity;

[0024] And / or,

[0025] The step of reducing the second number of concurrent channels based on the current threshold number of concurrent channels in the preset processor includes:

[0026] Based on the degree of upward trend of different usage rate change state data, and based on the pre-configured second mapping relationship, the number of concurrent channels matched by the second number is reduced based on the current number threshold of concurrent channels in the preset processor;

[0027] The degree of the upward trend is positively correlated with the second quantity.

[0028] Optionally, the cleaning method further includes:

[0029] Based on the usage rate change status data and the pre-configured third mapping relationship, a new preset sampling frequency that matches the usage rate change status data is dynamically determined, and subsequent sampling operations are performed using the updated new preset sampling frequency.

[0030] Optionally, the database server is configured with a global temporary table; wherein the global temporary table is used to store session-level temporary tables and / or transaction-level temporary tables;

[0031] For any set of initial drug distribution data to be cleaned, the cleaning process includes:

[0032] The initial drug distribution data is loaded into the global temporary table;

[0033] Using the preset standardized processing flow corresponding to the initial drug distribution data, corresponding read or write operations are performed based on the global temporary table to obtain the cleaned and standardized target drug distribution data.

[0034] Optionally, the cleaning method further includes:

[0035] Obtain several sets of the initial drug distribution data from different sources;

[0036] Based on pre-configured automatic file queuing rules, different initial drug distribution data are assigned to different file cleaning queue areas to determine the task processing batch corresponding to each file cleaning queue area.

[0037] File data cleaning operations are performed sequentially according to the order of the task processing batches.

[0038] Optionally, the cleaning method further includes:

[0039] Obtain external trigger operations;

[0040] Based on the external triggering operation, select several sets of the initial drug distribution data to be cleaned;

[0041] And / or,

[0042] Display the first associated data for each batch of tasks that are queued;

[0043] And / or,

[0044] The second associated data corresponding to each set of initial drug distribution data is displayed.

[0045] Optionally, the cleaning method further includes:

[0046] The cleaning process for different initial drug distribution data is performed concurrently in the database server;

[0047] And / or,

[0048] Resource isolation is implemented between the background tasks that process different drug distribution data in the database server and the main program in the database server.

[0049] And / or,

[0050] Disconnect the database server and application server in the data cleaning and processing system from communication.

[0051] In a second aspect, this disclosure provides a data cleaning and processing system for pharmaceutical distribution data. The data cleaning and processing system is implemented based on a database server in the data cleaning and processing system, and the database server is configured with preset data cleaning tools.

[0052] The cleaning system includes:

[0053] The data sampling module is used to process several initial drug distribution data using the database server and obtain the usage rate data of the preset processor in the database server at different sampling times based on the preset sampling frequency.

[0054] The change status acquisition module is used to acquire the current usage change status data of the preset processor based on the usage rate data at different sampling times;

[0055] The adjustment strategy determination module is used to generate a matching task processing adjustment strategy based on the usage rate change status data.

[0056] The adjustment and control module is used to dynamically adapt and adjust the threshold number of concurrent channels used for task processing in the preset processor based on the task processing adjustment strategy.

[0057] The data processing module is used to perform data cleaning processing using the adjusted preset processor and the preset data cleaning tool to obtain standardized target drug distribution data after cleaning each initial drug distribution data.

[0058] Optionally, the change state acquisition module includes:

[0059] The difference calculation unit is used to calculate the difference in the usage rate data corresponding to two adjacent consecutive sampling times of the preset processor.

[0060] The change state acquisition unit is used to acquire the ratio between the difference and the time interval corresponding to two adjacent consecutive sampling times, as the current usage rate change state data of the preset processor.

[0061] Optionally, the adjustment strategy determination module is further configured to:

[0062] In response to the decreasing trend of the usage rate change status data, based on the current threshold number of concurrent channels in the preset processor, a first number of concurrent channels are added;

[0063] In response to the usage rate change status data indicating a stable or upward trend, an upward slope less than a first preset value, and the usage rate data at the corresponding sampling time being less than a second preset value, a threshold for maintaining the current number of concurrent channels in the preset processor is determined.

[0064] In response to the usage rate change status data showing an upward trend with an upward slope greater than or equal to the first preset value; and / or the usage rate data at the corresponding sampling time being greater than or equal to the second preset value, based on the current number threshold of concurrent channels in the preset processor, the second number of concurrent channels is reduced;

[0065] In response to the usage rate change status data showing an upward trend, and the usage rate data at the corresponding sampling time being greater than a third preset value, it is determined to clear the concurrent channel in the preset processor; wherein, the third preset value is greater than the second preset value.

[0066] Optionally, the adjustment strategy determination module is further configured to:

[0067] Based on the degree of decline in the different usage rate change status data, and based on the pre-configured first mapping relationship, the first number of concurrent channels in the preset processor is increased to match the first number of concurrent channels.

[0068] The degree of the downward trend is positively correlated with the first quantity;

[0069] And / or,

[0070] The adjustment strategy determination module is also used for:

[0071] Based on the degree of upward trend of the different usage rate change status data, and based on the pre-configured second mapping relationship, the number of concurrent channels matched is reduced to the current number threshold of concurrent channels in the preset processor.

[0072] The degree of the upward trend is positively correlated with the second quantity.

[0073] Optionally, the cleaning system further includes:

[0074] The update module is used to dynamically determine a new preset sampling frequency that matches the usage rate change status data based on the usage rate change status data and a pre-configured third mapping relationship, and to perform subsequent sampling operations using the updated new preset sampling frequency.

[0075] Optionally, the database server is configured with a global temporary table; wherein the global temporary table is used to store session-level temporary tables and / or transaction-level temporary tables;

[0076] For any set of initial drug distribution data to be cleaned, the cleaning system further includes:

[0077] The data loading module is used to load the initial drug distribution data into the global temporary table;

[0078] The data processing module is used to perform corresponding read or write operations based on the global temporary table using a preset standardized processing flow corresponding to the initial drug distribution data, so as to obtain the cleaned and standardized target drug distribution data.

[0079] Optionally, the cleaning system further includes:

[0080] The data acquisition module is used to acquire several sets of the initial drug distribution data from different sources;

[0081] The data scheduling module is used to allocate different initial drug distribution data to different file cleaning queue areas based on pre-configured automatic file queuing rules, so as to determine the task processing batch corresponding to each file cleaning queue area.

[0082] The cleaning control module is used to perform file data cleaning operations sequentially according to the order of the task processing batches.

[0083] Optionally, the cleaning system further includes:

[0084] The external operation acquisition module is used to acquire external trigger operations;

[0085] The data selection module is used to select several sets of the initial drug distribution data to be cleaned based on the external trigger operation.

[0086] And / or,

[0087] The first display module is used to display the first associated data of each batch of tasks participating in the queue;

[0088] And / or,

[0089] The second display module is used to display the second associated data corresponding to each set of initial drug distribution data.

[0090] Optionally, the cleaning system further includes:

[0091] The concurrent execution module is used to concurrently perform the cleaning and processing of different initial drug distribution data in the database server;

[0092] And / or,

[0093] The resource isolation control module is used to isolate the resources between the background tasks that process different drug distribution data in the database server and the main program in the database server.

[0094] And / or,

[0095] The database server and application server in the data cleaning and processing system are disconnected from each other.

[0096] A third aspect of this disclosure provides an electronic device including a memory, a processor, and a computer program stored in the memory and for running on the processor, wherein the processor executes the computer program to implement the drug distribution data cleaning and processing method of the first aspect.

[0097] In a fourth aspect, this disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method for cleaning and processing pharmaceutical distribution data according to the first aspect.

[0098] In a fifth aspect, this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the drug distribution data cleaning and processing method of the first aspect.

[0099] Based on common knowledge in the field, the above-mentioned preferred conditions can be combined arbitrarily to obtain various preferred embodiments of this disclosure.

[0100] The positive and progressive effects of this disclosure are as follows:

[0101] In this disclosure, instead of using an application server for cleaning and processing drug distribution data, the database server is functionally improved to execute the standardized file cleaning process within the database server. By leveraging the high-performance resource configuration of the database server, it has stronger concurrent task processing capabilities and can significantly improve the problem of low system resource utilization, thereby increasing resource utilization and enhancing system stability and performance.

[0102] Instead of using a fixed concurrent task startup mechanism, this system samples the CPU (Central Processing Unit) usage in the database server in real time, dynamically, and continuously to determine its changing trends. This allows for precise identification of the CPU's data processing pressure. The system then dynamically adjusts the number of concurrent channels used for task processing within the CPU, flexibly and accurately adjusting the CPU's processing state to a suitable level. Based on the adjusted concurrent channels on the database server, the system performs cleaning to obtain standardized target drug distribution data. This is equivalent to configuring CPU pressure identification and adjustment capabilities, ensuring both data processing efficiency and system stability. Simultaneously, it effectively avoids abnormal states such as system crashes due to CPU exhaustion in the database server, ensuring smooth completion of cleaning processing for all drug distribution data without human intervention, thus reducing labor costs. Attached Figure Description

[0103] Figure 1 This is a flowchart of the drug distribution data cleaning and processing method according to Embodiment 1 of this disclosure;

[0104] Figure 2 This is a first flowchart of the drug distribution data cleaning and processing method of Embodiment 2 of this disclosure;

[0105] Figure 3 This is a second flowchart of the drug distribution data cleaning and processing method of Embodiment 2 of this disclosure;

[0106] Figure 4 This is a schematic diagram of the framework for the document cleaning process in Embodiment 2 of this disclosure;

[0107] Figure 5 This is a schematic diagram illustrating the configuration of the automatic file queuing rule in Embodiment 2 of this disclosure;

[0108] Figure 6 This is a schematic diagram illustrating the arrival and queuing of documents according to Embodiment 2 of this disclosure;

[0109] Figure 7 This is a schematic diagram showing the first data in Embodiment 2 of this disclosure;

[0110] Figure 8 This is a schematic diagram illustrating the second data display of Embodiment 2 of this disclosure;

[0111] Figure 9 This is a schematic diagram of the drug distribution data cleaning and processing system of Embodiment 3 of this disclosure;

[0112] Figure 10 This is a schematic diagram of the drug distribution data cleaning and processing system of Embodiment 4 of this disclosure;

[0113] Figure 11 This is a schematic diagram of the structure of the electronic device according to Embodiment 5 of this disclosure. Detailed Implementation

[0114] The present disclosure is further illustrated below by way of embodiments, but the present disclosure is not limited to the scope of the embodiments described herein.

[0115] The prefixes such as "first" and "second" used in this disclosure are merely for distinguishing different descriptive objects and do not limit the position, order, priority, quantity, or content of the described objects. The use of ordinal numbers and other prefixes used to distinguish descriptive objects in this disclosure does not constitute a limitation on the described objects. The description of the described objects is given in the claims or the context of the embodiments, and should not be construed as an unnecessary limitation. Furthermore, in the description of this embodiment, unless otherwise stated, "multiple" means two or more.

[0116] In this embodiment of the disclosure, the collection, storage, use, processing, transmission, provision, and disclosure of user personal information comply with relevant laws and regulations and do not violate public order and good morals.

[0117] The existing solutions for information management and data services in the pharmaceutical distribution process have the following problems:

[0118] (1) Insufficient system stability

[0119] In existing systems, file cleaning and processing operations primarily reside on the application server. Data required for these operations is read from the database disk and transferred to the application server, with each processing step's result then saved back to the database server. The current system uses a Java-Job scheduling mechanism, relying on Java applications to load cleaning tasks at a fixed time frequency, without considering the application server's available resources or workload. Furthermore, while it supports multi-threaded concurrency, there is no mechanism for detecting, terminating, or releasing resources when a thread is unexpectedly suspended (single task unresponsive). When all allowed threads are suspended, the entire data cleaning task is halted (suspended). Therefore, the existing system frequently experiences unexpected shutdowns of the "cleaning task scheduling" function due to resource contention or other exceptional events (such as premature termination of cleaning tasks in the early morning).

[0120] Furthermore, the application server performs more than just this one task; it may also be executing numerous other tasks interactively. Therefore, the result is that automated tasks frequently stop for unknown reasons, causing the cleaning function to "freeze," prematurely ending the planned automated cleaning process and failing to achieve the intended results. In one instance, the plan was to automatically clean from midnight until 8:30 AM, but in reality, the automatic cleaning process had already stopped long before.

[0121] (2) Low system resource utilization

[0122] The existing system achieves data file standardization and cleaning by scheduling tasks through the application server (mainly using CPU and memory). It primarily uses the resources of the application server for data computation and other processing, while making fewer resource requests to the database server. The existing database server does not have data computation capabilities and works with the application server for data scheduling and other processing. The resource configuration of the database server is often 2 to 4 times or even higher than that of the application server.

[0123] Furthermore, the existing system uses a fixed maximum thread concurrency mechanism, which cannot be dynamically adjusted according to the actual system pressure. The idle rate of overall hardware resources (including application servers and database servers) is over 30% to 70%. Under the premise of fixed resource costs (AWS resource monthly rental system), the cost-effectiveness ratio is very low.

[0124] (3) The system has low data cleaning efficiency.

[0125] (3.1) Existing system design defects lead to the following: Specifically, cleaning a single drug sales file requires 46 disk I / O (input / output) operations. When a large amount of data is continuously inserted into the same table, the table is repeatedly read and modified at high frequency, which can easily lead to increased competition for hot tables in the database and cause I / O (input / output) performance bottlenecks.

[0126] (3.2) The existing system sets a maximum of 5 threads for cleaning file data and uses application server resources (CPU and memory). Even if the CPU utilization is low, the number of concurrent threads will not be automatically expanded.

[0127] (3.3) Because there is no automatic recovery mechanism for suspended processes, once all threads stop working, the entire cleaning task will stop prematurely. This means that before the problem is discovered manually, the actual amount of work completed can only reach the expected 5%~20% (cleaning completion rate), which cannot guarantee the normal cleaning and processing of data and cannot meet the actual data cleaning needs.

[0128] Taking the cleaning of pharmaceutical sales flow documents as an example, the existing system involves about 46 steps in the entire cleaning process. The database operations used are mainly query (Select) and update (Update), with each action corresponding to a database read and write operation. On the one hand, new documents are constantly coming into the data table corresponding to the sales flow documents, while several concurrent tasks are constantly reading and modifying this table. This indicates that the data table is very busy (called a hot table in the database). The result is that the front-end operators feel that the cleaning is very slow. It is impossible to select multiple documents to enter the cleaning process at the same time. It is necessary to manually control the entry of small batches into the cleaning process and manually observe whether the progress is completed. However, too much manual intervention will inevitably consume human time, reduce the efficiency of cleaning, and make it difficult to determine the appropriate batch size for inclusion in the cleaning process, thus resulting in low overall cleaning efficiency.

[0129] Based on the aforementioned problems with current information management and data service solutions in the pharmaceutical distribution process, this solution proposes a novel pharmaceutical distribution data cleaning and processing scheme to improve resource utilization, data cleaning efficiency, and system stability. Specifically:

[0130] Example 1

[0131] The drug distribution data cleaning and processing method in this embodiment is implemented based on the database server in the data cleaning and processing system, and the database server is configured with preset data cleaning tools.

[0132] In this process, the database server is pre-programmed using a database technology language, enabling the configuration of various types of pre-set data cleaning tools on the database server to perform corresponding calculations on the input drug distribution data based on the database server.

[0133] Different types of preset data cleaning tools correspond to different data cleaning functions. The specific types of preset data cleaning tools can be configured, expanded, or adjusted according to the actual application scenario.

[0134] In addition, the database server and application server in the data cleaning and processing system are disconnected from each other, and network data communication between them is canceled. There is no communication between the application server and the database server. This cuts off the existing data processing process based on the application server and avoids multiple (e.g., 46 times) data interaction actions (frequent communication) between the application server and the database server for each drug distribution data cleaning, thus ensuring the efficiency of drug distribution data cleaning.

[0135] In this embodiment, instead of using the application server for cleaning and processing drug distribution data, the database server is functionally improved to perform the standardized file cleaning and processing process on the database server. By utilizing the high-performance resource configuration of the database server (CPU and memory are several times that of the application server), it has stronger task concurrency processing capabilities and can significantly improve the problem of low system resource utilization, thereby improving resource utilization, system stability, and system performance.

[0136] like Figure 1 As shown, the drug distribution data cleaning and processing method in this embodiment includes:

[0137] S101. In response to using a database server to process several initial drug distribution data, the usage rate data of the preset processor in the database server at different sampling times is obtained based on a preset sampling frequency.

[0138] The pharmaceutical distribution data includes data corresponding to documents such as pharmaceutical manufacturers' inventory documents, sales documents, and procurement documents.

[0139] The default processor includes, but is not limited to, the CPU in the database server.

[0140] The preset sampling frequency can be pre-set or adjusted based on experience or actual needs, such as collecting CPU usage data once every 1 minute.

[0141] S102. Based on the usage rate data at different sampling times, obtain the current usage rate change status data of the preset processor;

[0142] Among them, the usage rate change status data corresponds to the CPU usage rate increasing, decreasing, or remaining unchanged.

[0143] S103. Generate matching task processing adjustment strategies based on usage rate change status data;

[0144] S104. Based on the task processing adjustment strategy, dynamically adapt and adjust the threshold number of concurrent channels used for task processing in the preset processor.

[0145] The usage rate change data corresponds to the current data processing pressure state of the CPU. Different pressure states employ different task processing adjustment strategies to regulate the concurrent channels in the CPU. In this case, the threshold for the maximum number of concurrent channels of the CPU is being adjusted.

[0146] S105. The adjusted preset processor performs data cleaning processing according to the preset data cleaning tool to obtain standardized target drug distribution data after cleaning each initial drug distribution data.

[0147] For example, in the drug distribution information collection, processing and manufacturing system, through improvements such as intelligent algorithms and optimization of the system's backend disk read / write efficiency, the system's computing pressure is automatically checked, and the system's data processing throughput is adjusted in a timely manner. When the system pressure is low, the concurrent processing channels are automatically increased, and when the system pressure is high, the concurrent processing channels are appropriately reduced. From manual scheduling of production to undifferentiated automatic concurrent production, and then to the system's proactive adjustment of automated scheduling with pressure feedback, the system platform's data cleaning capabilities and automatic cleaning completion rate (number of data files that have been cleaned / number of data files that have been collected) have been significantly improved, such as from the initial 0-5%, to 20%, and finally to 65%-95%, thus far exceeding the level of manual scheduling and processing.

[0148] This solution abandons the fixed concurrent task startup mechanism and instead uses real-time, dynamic, and continuous sampling of the CPU utilization rate in the database server to determine its changing trends. This allows for precise identification of the CPU's data processing pressure, enabling dynamic adjustment of the number of concurrent channels used for task processing within the CPU. This flexibly and accurately adjusts the CPU's processing state to a matching level. Based on the adjusted concurrent channels in the database server, the initial drug distribution data to be cleaned is processed to obtain standardized target drug distribution data. This is equivalent to configuring CPU pressure identification and adjustment functions, ensuring both data processing efficiency and system stability. Simultaneously, it effectively avoids abnormal states such as system crashes caused by CPU exhaustion in the database server, ensuring smooth completion of cleaning processing for all drug distribution data without human intervention, thus reducing labor costs.

[0149] Example 2

[0150] The drug distribution data cleaning and processing method in this embodiment is a further improvement on Embodiment 1, specifically:

[0151] In a feasible solution, as shown in Table 1 below, the common types and corresponding relationships between different file types and the content to be cleaned are as follows:

[0152] Table 1

[0153]

[0154] In this example, the database server is configured with a global temporary table (GTT).

[0155] Global temporary tables are used to store session-level and / or transaction-level temporary tables. When a transaction or session ends, the Oracle database system (a type of database) can automatically ensure that resources are released automatically without requiring the developer to give a deletion command, thus avoiding resource leaks and increasing the stability of application functions in multi-threaded concurrency.

[0156] Specifically, the core of the Global Temporary Table (GTT) lies in its ability to reduce I / O and log overhead, achieved through its unique storage and data processing methods. By default, data in the GTT is stored in the Oracle database's temporary tablespace, not the user data tablespace. The temporary tablespace is typically configured to use memory (such as the sort area in the System Global Area (SGA)) or high-speed disk to optimize temporary data access, and it has the following characteristics:

[0157] (1) Memory priority: Oracle database will prioritize caching the data of the global temporary table GTT in memory, and only when the amount of data exceeds the preset memory capacity will it overflow into the temporary table space on the disk (such as when it exceeds the preset sorting area size of the SGA parameter).

[0158] (2) No redo log: Data modifications to the global temporary table GTT do not generate redo logs, reducing log writing overhead; among them, transaction-level GTT discards data directly upon commit, while session-level GTT is automatically cleaned up by the background process when the session ends.

[0159] (3) No Undo log (some scenarios): The rollback operation of transaction-level GTT directly discards the data and does not require the generation of undo log; the rollback of session-level GTT may involve a small amount of undo, but far less than that of ordinary tables.

[0160] Based on this, compared to operations based on ordinary database user tables, which target physical disks and have redo and undo logs for each write operation, providing a global temporary table tool in the database server prioritizes memory, eliminates redo and undo logs, and even if there is capacity overflow to disk under GTT conditions, the overall IO is at most 1 / 3 of the original processing operation, not to mention pure memory operations without GTT capacity overflow. This significantly improves the processing efficiency of drug distribution data and enhances the stability of the system in drug distribution data processing scenarios.

[0161] Taking sales file data as an example, performance can be improved by fully utilizing the high-performance operation features of Oracle database's global temporary tables, while also improving performance by reducing the number of I / O operations.

[0162] For any initial set of drug distribution data to be cleaned, the cleaning method in this embodiment includes:

[0163] Load the initial drug distribution data into a global temporary table;

[0164] The system employs a pre-defined standardized processing flow corresponding to the initial drug distribution data, performing corresponding read or write operations based on a global temporary table to obtain cleaned and standardized target drug distribution data.

[0165] The following section, based on different types of sales document data processing requirements, specifically illustrates the differences in data operation steps before and after providing a global temporary table (GTT) in the database server:

[0166] (1) The operation steps using a regular database user table are as follows:

[0167] [For cleaning unknown product-related data in sales documents] Steps 1 to n1, each step performs a read (Select) or write (Update) operation on the original physical table;

[0168] [For cleaning unknown customer-related data in sales documents] In steps n+1 to n2, each step performs a read (Select) or write (Update) operation on the original physical table;

[0169] [For cleaning data related to unknown batch numbers in sales documents] From step n2+1 to step 46, each step performs a read (Select) or write (Update) operation on the original physical table;

[0170] (2) The operation steps based on the global temporary table GTT are as follows:

[0171] [For cleaning data related to unknown products in sales files] Step 1: First, load the corresponding data in the current sales file into the global temporary table GTT, where the global temporary table GTT is automatically selected to reside in memory first; Steps 2 to m1 (less than n1): Each step only performs read (Select) or write (Update) operations on the global temporary table GTT, which are basically the corresponding read or write operations performed on memory.

[0172] [For cleaning unknown customer-related data in sales files] Steps m+1 to m2 (m2 < n2) each basically involve reading or writing to memory.

[0173] [For cleaning data related to unknown batch numbers in sales documents] Steps m2+1 to 19 are basically reading or writing operations to memory.

[0174] Therefore, compared with operations based on ordinary database user tables, the data operation process based on the global temporary table GTT has been significantly optimized, greatly improving the processing efficiency of drug distribution data and enhancing the stability of the system in drug distribution data processing scenarios.

[0175] Based on this, this embodiment implements physical disk read merging and operation step merging, specifically:

[0176] Disk read / write: The cleaning process before the system upgrade was reduced from 46 physical disk I / O operations to 2 operations after the upgrade (1 read operation + 1 write operation). All intermediate steps were completed in the global temporary table GTT, reducing physical read / write operations to 4.35% of the original.

[0177] Step merging: Multiple DML commands (Select / Update / DELETE / INSERT, etc., are all data management operation commands, collectively referred to as DML) are merged into a single DML statement. Under the premise of achieving the same goal, the number of DML commands issued is reduced as much as possible, thereby further improving the processing efficiency of drug distribution data.

[0178] In this embodiment, before file cleaning begins, the data corresponding to the file is loaded into a global temporary table. The data in the global temporary table is stored in a temporary table space (such as TEMP) rather than the user table space. Subsequent cleaning operations only target the global temporary table, that is, only in the temporary table space, avoiding the same physical storage area as ordinary table operations, thereby achieving the effect of avoiding hot blocks.

[0179] The cleaning process may involve cleaning, updating, and assigning values ​​to multiple fields. In the original functionality, each time a field was cleaned (i.e., each time the standardization of a field was completed), an Update command was executed to write back to the physical table. Ultimately, cleaning a single file involved a total of 46 executions of Select and Update commands on the original physical table.

[0180] In this embodiment, variables that can be temporarily stored during the intermediate cleaning process are not written back to the table. The standardized results of multiple fields are written back uniformly. Furthermore, these write-backs only reflect the global temporary table. When all the cleaning content is completed in the global temporary table, it is written back to the formal table all at once. As a result, there is only one initial query (Select) and one write-back (Update) operation for the original physical table.

[0181] In a feasible solution, such as Figure 2 As shown, step S102 includes:

[0182] S1021. Calculate the difference in the usage rate data of the preset processor at two consecutive adjacent sampling times;

[0183] S1022. Obtain the ratio between the difference and the time interval corresponding to two adjacent consecutive sampling times, and use it as the current usage rate change status data of the preset processor.

[0184] Specifically, the CPU utilization rate change at each sampling point is calculated using the central difference method. Specifically, the CPU utilization rate data at two sampling moments before and after the current sampling point i are obtained, and the corresponding instantaneous slope (i.e., ratio) is calculated using the following formula:

[0185]

[0186] Where m represents the instantaneous slope (i.e., the ratio). and For the two sampling times before and after the current sampling point i , The CPU utilization data below.

[0187] If the sampling frequency intervals are uniform (once per minute), then the time difference Δt = 2 minutes (1 minute before and after), and the above formula simplifies to (unit: percentage / minute):

[0188]

[0189] For example: CPU utilization at any given time =38%, CPU utilization at any given time =43% (current time i, not directly used for calculation); CPU utilization at any given time =57%, then The instantaneous slope calculated at each moment is m = (57% - 38%) / 2 = 9.5% / minute; the unit of instantaneous slope here is percentage / minute.

[0190] By analogy, the instantaneous slope corresponding to each sampling point is calculated. The sign of the instantaneous slope determines whether the CPU's data processing pressure increases, decreases, or remains unchanged. Specifically, a positive instantaneous slope indicates that the CPU's data processing pressure increases, a negative instantaneous slope indicates that the CPU's data processing pressure decreases, and an instantaneous slope of 0 indicates that the CPU's data processing pressure remains unchanged.

[0191] In this scheme, the CPU utilization rate is sampled in real time and dynamically at two consecutive sampling times to determine its changing trend, so as to accurately identify the data processing pressure status of the CPU at the current sampling point in the database server.

[0192] In one feasible solution, step S103 includes:

[0193] In response to the decreasing trend in usage rate status data, a first number of concurrent channels is added based on the current threshold number of concurrent channels in the preset processor.

[0194] In response to the stable or upward trend of the usage rate change status data, the upward slope is less than the first preset value, and the usage rate data at the corresponding sampling time is less than the second preset value, the current number threshold of concurrent channels in the preset processor is determined.

[0195] In response to the usage rate change status data showing an upward trend, with the upward slope being greater than or equal to a first preset value; and / or the usage rate data at the corresponding sampling time being greater than or equal to a second preset value, the system determines to reduce the number of concurrent channels by a second number based on the current threshold number of concurrent channels in the preset processor.

[0196] In response to the upward trend in the usage rate change status data, and the usage rate data at the corresponding sampling time being greater than the third preset value, it is determined to clear the concurrent channel in the preset processor; wherein, the third preset value is greater than the second preset value.

[0197] In this scheme, when the utilization rate change status data is negative, it indicates that the CPU utilization rate is declining and there are idle CPU resources. At this time, the number of concurrent channels used for task processing can be increased to schedule the CPU to process more data in a timely manner, thereby improving resource utilization and data processing efficiency.

[0198] When the utilization rate change status data is 0 or close to 0, it means that the CPU utilization rate is basically unchanged, and the number of concurrent channels in the CPU can be maintained at this time.

[0199] When the usage rate change status data is positive, it indicates that the CPU usage rate is on the rise and the available CPU resources are decreasing. At this time, the number of concurrent channels used for task processing can be reduced to reduce the amount of data processed by the CPU in a timely manner. This ensures data processing efficiency while avoiding CPU resource exhaustion and abnormalities, thus ensuring the smooth, stable and reliable operation of all drug distribution data.

[0200] For example, when the CPU utilization status data indicates a decrease in CPU utilization, then 'a' (e.g., 'a' is 2) concurrent channels are automatically added to the original concurrent channels for task processing.

[0201] When the CPU utilization status data indicates an increase in CPU utilization, the number of concurrent channels will be automatically reduced by b (e.g., b = 2) from the original number of concurrent channels for task processing.

[0202] More specifically, for example, when the CPU utilization at any sampling time is less than 80%, the current concurrent channels of the CPU are maintained for task processing; when the CPU utilization at any sampling time is greater than or equal to 80%, the current number of concurrent channels of the CPU is reduced; or, when the CPU utilization at any sampling time is greater than or equal to 80% but less than 95%, and the utilization change status data (instantaneous slope) is greater than 45°, the CPU utilization is continuously increasing and timely intervention is needed to slow down the processing, at which point the current number of concurrent channels of the CPU is automatically reduced; when the CPU utilization at any sampling time is greater than or equal to 95%, the allowed concurrent channels of the CPU are immediately cleared to ensure that the application does not "immediately start a new cleaning task" during job scheduling, avoiding abnormal situations such as "freezing" caused by CPU exhaustion in the database server, thus effectively ensuring the stability of the system while also ensuring the smooth and efficient processing of data.

[0203] The following is a further detailed explanation with reference to Table 2:

[0204] Table 2

[0205]

[0206] In one feasible approach, the step of determining, based on the current threshold number of concurrent channels in a preset processor, to increase a first number of concurrent channels includes:

[0207] Based on the degree of decline in different usage rate change status data, and on the basis of the current number threshold of concurrent channels in the preset processor, a first number of matching concurrent channels are added according to the pre-configured first mapping relationship. The first mapping relationship is pre-built based on experience or actual needs and can be reset or adjusted according to actual needs.

[0208] The degree of downward trend is positively correlated with the first quantity; see Table 3 for example:

[0209] Table 3

[0210]

[0211] Based on Table 3 above, specifically, if the degree of downward trend A1, B1, and C1 increases sequentially, the corresponding increase in concurrent channels i1, j1, and k1 also increases sequentially.

[0212] This solution can automatically match the number of concurrent channels that need to be increased based on the degree of decline in different usage rate change data, so as to further improve the rationality of CPU pressure adjustment in database services, and while ensuring system stability, further ensure the reliability of input drug distribution data cleaning and processing.

[0213] In one feasible approach, the step of reducing a second number of concurrent channels based on a current threshold number of concurrent channels in a preset processor includes:

[0214] The degree of upward trend of different usage rate change status data is determined by reducing the number of matching concurrent channels based on the current number threshold of concurrent channels in the preset processor, according to the pre-configured second mapping relationship. The second mapping relationship is pre-built based on experience or actual needs and can be reset or adjusted according to actual needs.

[0215] The degree of upward trend is positively correlated with the second quantity. See Table 4 for example:

[0216] Table 4

[0217]

[0218] Based on Table 4 above, specifically, if the degree of upward trend A2, B2, and C2 increases sequentially, the corresponding decrease in concurrent channels i2, j2, and k2 also increases sequentially.

[0219] This solution can automatically match the number of concurrent channels that need to be reduced based on the degree of upward trend of different usage rate change data, so as to further improve the rationality of CPU pressure adjustment in database services, and while ensuring system stability, further ensure the reliability of input drug distribution data cleaning and processing.

[0220] In one feasible solution, the cleaning method of this embodiment further includes:

[0221] Based on the usage rate change status data and the pre-configured third mapping relationship, a new preset sampling frequency that matches the usage rate change status data is dynamically determined, and subsequent sampling operations are performed using the updated new preset sampling frequency.

[0222] The third mapping relationship is pre-built based on experience or actual needs and can be reset or adjusted according to actual needs.

[0223] In this solution, the sampling frequency of CPU utilization is dynamically adjusted according to specific circumstances to ensure that the accuracy of CPU stress state identification is further improved, so as to make the system more stable and meet the higher requirements of drug distribution data processing scenarios.

[0224] Specifically, the usage rate change data, i.e., instantaneous efficiency, can be used as the trigger condition for updating the sampling frequency. For example, when the instantaneous efficiency is greater than 10%, the sampling frequency of once per minute will be adjusted to twice per minute (i.e., the next sampling time will be delayed by 30 seconds).

[0225] Of course, the sampling frequency can also be adjusted according to other triggering conditions, such as direct modification based on external interactive operations, without specific restrictions, as long as it can be achieved.

[0226] In a feasible solution, such as Figure 3 As shown, the cleaning method in this embodiment further includes:

[0227] S301. Obtain several initial drug distribution data from different sources;

[0228] Different sources can correspond to different pharmaceutical companies. The purpose is to process the data input from these pharmaceutical companies A, B, C, etc. through a series of processes (scheduling, cleaning, etc.) to obtain standardized target drug distribution data.

[0229] S303. Based on pre-configured automatic file queuing rules, different initial drug distribution data are assigned to different file cleaning queuing areas to determine the task processing batches corresponding to each file cleaning queuing area.

[0230] S304. Perform file data cleaning operations sequentially according to the order of task processing batches.

[0231] like Figure 4 As shown, all document data collected from pharmaceutical companies A, B, C, etc., first enter the document cleaning queue waiting area under the scheduling of the "automatic queuing controller". Then, under the scheduling of the job scheduling controller, they enter different work centers (document processing and cleaning workstations). The "processor" of each work center performs the document cleaning function. Then, according to the configuration requirements, it is determined whether to automatically import the cleaning results into the purchase, sales, and inventory ledgers, and finally reach the deliverable waiting quality inspection area (standardized data).

[0232] To achieve the above-mentioned allocation of data to different file cleaning queues, it is necessary to pre-configure automatic queuing rules in a configuration file. The specific rules can be configured according to actual needs.

[0233] For example, such as Figure 5 As shown, the configuration information for pre-configured automatic queuing rules generally includes, but is not limited to:

[0234] PROJECTCODE: Pharmaceutical company identifier, representing the data file of which pharmaceutical company;

[0235] FILETYPE: File type; where I represents inventory file, S represents sales file, and P represents purchase file;

[0236] STARTTIME: Start time. From this point onwards, all arriving data files can automatically enter the queuing area.

[0237] ENDTIME: Queue deadline. After this time, arriving files will no longer enter the queue. Once the files in the waiting area have been cleaned, no new data files will enter the automatic cleaning process.

[0238] IMPORTFLAG: Whether to automatically import the cleaned data file into the ledger (so that the quality inspection process can begin).

[0239] Of course, customized automatic cleaning plans can also be scheduled to meet specific needs. These needs are covered by special rules, as shown in Table 5 below. The configuration information for special automatic cleaning configuration rules for individual pharmaceutical plants can be as follows:

[0240] Table 5

[0241]

[0242] In addition, in this embodiment, before the introduction of the automatic file queuing rule function, the application system itself has an automatic task scheduling function, but it does not have an order queuing waiting function and directly starts file cleaning; after the introduction of the automatic file queuing rule function, the scheduling control program can still be used for scheduling. At this time, the scheduling function only realizes that the "arriving files" are sent into the "queuing area" according to the configuration and then ends, thus completing the queuing scheduling function.

[0243] like Figure 6 As shown, in the document arrival area, drug distribution data from dozens of pharmaceutical companies and thousands of drug distributors arrive at the "document arrival area" at different times according to their respective frequencies and conditions. For the "document queuing area," the queuing and processing workflow groups data into task batches based on three dimensions: pharmaceutical company, document type, and arrival time. These batches are then named according to the following naming rule: SD-yyyymmdd-hhmiss-MSJK, representing document type-year / month / day-hour / minute / second-pharmaceutical company identifier, respectively. The document types include SD, PD, ID, SM, and PM, representing daily sales, daily purchases, daily inventory, monthly sales, and monthly purchases, respectively.

[0244] In a feasible solution, the first associated data for each batch of tasks participating in the queuing is displayed;

[0245] The second set of associated data corresponding to each initial set of drug distribution data is displayed.

[0246] The first and second related data can be predetermined or adjusted according to actual needs, without specific restrictions.

[0247] In this solution, information related to the cleaned drug distribution data is displayed in real time and synchronously, allowing relevant personnel to view and understand it intuitively so that they can carry out further operations.

[0248] For example, see 7. Based on the pharmaceutical company identification, document type (business type), and collection frequency, the system can query and display which documents have arrived but have not yet entered the queuing area. For some special documents, although they have participated in the automated queuing and cleaning process, they have not been able to complete the data cleaning (the standardized cleaning of some information in some documents requires manual intervention). In this case, they will also return to this display platform so that they can be manually intervened and then enter the automated cleaning process.

[0249] See Figure 8It can display the queuing status of documents, making it easy for relevant personnel to view the currently queued batches of documents. To facilitate querying and display, sales documents, purchase documents, and inventory documents of a pharmaceutical company are created into separate batches. Each batch may contain multiple documents, and each document is assigned a unique queuing ID number.

[0250] This interface allows users to directly check the queue status without needing to specify any conditions.

[0251] Sequence number: The priority of the queue batch, indicating which batch it is and where it is in the queue.

[0252] Pharmaceutical manufacturer identification: Indicates which pharmaceutical manufacturer this batch of data files belongs to;

[0253] Queue batch: A string consisting of type + date + day sequence number + pharmaceutical company identifier. This string also reveals the information of the files contained in this batch.

[0254] Queue start time: The specific time (down to the second) at which each batch starts joining the queue, thus indicating the length of the waiting time.

[0255] Number of files: For a pharmaceutical factory, the number of files of one type (purchase, sales, inventory) will not be too many when automatically scheduling and queuing. It is usually a single digit number, and at most two digits.

[0256] Batch start order: Each file has a unique queue ID number, which indicates which ID number this batch starts with.

[0257] Batch cutoff order: This indicates the queue ID number at which this batch ends.

[0258] In one feasible solution, the cleaning method of this embodiment further includes:

[0259] Obtain external trigger operations;

[0260] Based on external triggering operations, select several initial drug distribution data sets to be cleaned.

[0261] This solution can be combined with external triggering operations to achieve more flexible data cleaning operations, so as to meet personalized cleaning requirements.

[0262] Specifically, in combination Figure 7 The system displays information such as which pharmaceutical companies' data files have arrived and which have not yet been queued, allowing relevant personnel to select different pharmaceutical companies at any time.

[0263] This function interface allows for manual control, enabling users to select individual batches of files to enter the queuing area.

[0264] The main configuration features of this console:

[0265] Data cleaning: Manually query a batch of files that need to be cleaned through the console interface, apply checkboxes, and add them to the queue in batches.

[0266] Clean All: This operation adds all files that meet the query criteria to the cleaning queue at once without requiring checkboxes.

[0267] Queue-jumping cleaning: Under normal circumstances, the queue is based on time order, and the first to arrive is cleaned. In emergency or special circumstances, people with super privileges can use this function to add the selected files to the front of the queue. Once the processing center has time, the cleaning process for this batch of data files will begin immediately.

[0268] Cleaning List: Used to view the current queuing order and the number of files. When intervention is needed, the priority order of already queued files can be forcibly changed (only roles with special permissions have this function).

[0269] In one feasible embodiment, the cleaning method further includes:

[0270] The cleaning and processing of different initial drug distribution data are performed concurrently on the database server.

[0271] Additionally, resource isolation is implemented between the background tasks that process different drug distribution data on the database server and the main program on the database server.

[0272] Specifically, the existing approach uses the automatic startup function of Java programs, with file cleaning performed on the application server, resulting in frequent data exchange between the database server and the application server. Each additional layer of hardware interoperability increases the probability of information disruption and loss (such as external attacks, large data uploads and downloads, or human intervention to halt the computation process due to special maintenance needs), thereby increasing instability risks.

[0273] In this embodiment, all cleaning actions are completed within the database server, without the need for communication between the database server and the application server, and are unaffected by various unexpected events on the application server. This requires the use of Oracle database's asynchronous task execution capabilities to achieve task scheduling and concurrent execution control.

[0274] This solution introduces an asynchronous execution scheduling mechanism, utilizing the Oracle database background process on the database server to directly perform the cleanup, effectively avoiding communication risks with the application server. Simultaneously, it requires multi-threaded concurrency, employing the Oracle database's DBMS_JOB.SUBMIT (database scheduled task) package to achieve dynamic control and concurrent execution of the task queue. The asynchronous scheduling based on the Oracle database has the following characteristics:

[0275] (1) Concurrent execution: DBMS_JOB.SUBMIT supports submitting multiple tasks to the background queue at the same time. The Oracle task scheduler dynamically allocates execution rights according to system resources (such as CPU, I / O) to achieve true parallel processing.

[0276] (2) Resource isolation: The background task is separated from the main program to avoid the long-running task occupying the main session resources and ensure the smoothness of the front-end business. Before the solution was improved, the front-end operation function would always be in a "busy suspended" state. After the solution was improved, the background database is responsible for scheduling and executing other front-end application operations without being affected.

[0277] Compared to synchronous execution, where N time-consuming tasks need to be completed serially, with a total time of N×T, asynchronous scheduling allows tasks to be executed concurrently. If system resources are sufficient, the total time is close to T (single task time), increasing resource utilization by M times, thereby effectively improving the processing efficiency of drug distribution data.

[0278] Example 3

[0279] Corresponding to the aforementioned embodiments of cleaning and processing drug distribution data, this disclosure also provides embodiments of a drug distribution data cleaning and processing system.

[0280] The drug distribution data cleaning and processing system in this embodiment is implemented based on a database server in the data cleaning and processing system, and the database server is configured with preset data cleaning tools.

[0281] like Figure 9 As shown, the drug distribution data cleaning and processing system of this embodiment includes:

[0282] Data sampling module 1 is used to process several initial drug distribution data using a database server and obtain the usage rate data of the preset processor in the database server at different sampling times based on a preset sampling frequency.

[0283] The change status acquisition module 2 is used to acquire the current usage change status data of the preset processor based on the usage rate data at different sampling times;

[0284] Adjustment strategy determination module 3 is used to generate matching task processing adjustment strategies based on usage rate change status data;

[0285] Adjustment control module 4 is used to dynamically adapt and adjust the threshold number of concurrent channels used for task processing in the preset processor based on the task processing adjustment strategy.

[0286] Data processing module 5 is used to perform data cleaning processing using a preset processor and a preset data cleaning tool to obtain standardized target drug distribution data after cleaning each initial drug distribution data.

[0287] This solution abandons the fixed concurrent task startup mechanism and instead uses real-time, dynamic, and continuous sampling of the CPU utilization rate in the database server to determine its changing trends. This allows for precise identification of the CPU's data processing pressure, enabling dynamic adjustment of the number of concurrent channels used for task processing within the CPU. This flexibly and accurately adjusts the CPU's processing state to a matching level. Based on the adjusted concurrent channels in the database server, the initial drug distribution data to be cleaned is processed to obtain standardized target drug distribution data. This is equivalent to configuring CPU pressure identification and adjustment functions, ensuring both data processing efficiency and system stability. Simultaneously, it effectively avoids abnormal states such as system crashes caused by CPU exhaustion in the database server, ensuring smooth completion of cleaning processing for all drug distribution data without human intervention, thus reducing labor costs.

[0288] For the system embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs.

[0289] Example 4

[0290] To improve the system's efficiency in cleaning files, enhance operational performance, increase resource utilization, reduce human involvement, and prevent automatic shutdowns, this embodiment proposes a novel system for cleaning drug distribution data (corresponding to data in pharmaceutical flow files).

[0291] like Figure 10 As shown, the drug distribution data cleaning and processing system in this embodiment is a further improvement on Embodiment 3, specifically:

[0292] In one feasible solution, the change state acquisition module 2 includes:

[0293] The difference calculation unit is used to calculate the difference in the usage rate data of the preset processor at two consecutive adjacent sampling times.

[0294] The change state acquisition unit is used to acquire the ratio between the difference and the time interval corresponding to two adjacent consecutive sampling times, as the preset processor's current usage rate change state data.

[0295] In one feasible solution, the adjustment strategy determination module 3 is also used for:

[0296] In response to the decreasing trend in usage rate status data, a first number of concurrent channels is added based on the current threshold number of concurrent channels in the preset processor.

[0297] In response to the stable or upward trend of the usage rate change status data, the upward slope is less than the first preset value, and the usage rate data at the corresponding sampling time is less than the second preset value, the current number threshold of concurrent channels in the preset processor is determined.

[0298] In response to the usage rate change status data showing an upward trend, with the upward slope being greater than or equal to a first preset value; and / or the usage rate data at the corresponding sampling time being greater than or equal to a second preset value, the system determines to reduce the number of concurrent channels by a second number based on the current threshold number of concurrent channels in the preset processor.

[0299] In response to the upward trend in the usage rate change status data, and the usage rate data at the corresponding sampling time being greater than the third preset value, it is determined to clear the concurrent channel in the preset processor; wherein, the third preset value is greater than the second preset value.

[0300] In one feasible solution, the adjustment strategy determination module 3 is also used for:

[0301] Based on the degree of decline in different usage rate change status data, and based on the pre-configured first mapping relationship, the first number of matching concurrent channels is increased on the basis of the current number threshold of concurrent channels in the preset processor;

[0302] Among them, the degree of the downward trend is positively correlated with the first quantity;

[0303] In one feasible solution, the adjustment strategy determination module 3 is also used for:

[0304] Based on the degree of upward trend of different usage rate change status data, and based on the pre-configured second mapping relationship, the number of matching second concurrent channels is reduced based on the current number threshold of concurrent channels in the preset processor;

[0305] Among them, the degree of upward trend is positively correlated with the second quantity.

[0306] In one feasible solution, the cleaning system also includes:

[0307] The update module is used to dynamically determine a new preset sampling frequency that matches the usage change status data based on the usage change status data and the pre-configured third mapping relationship, and to perform subsequent sampling operations using the updated new preset sampling frequency.

[0308] In one feasible solution, a global temporary table is configured in the database server; wherein the global temporary table is used to store session-level temporary tables and / or transaction-level temporary tables;

[0309] For any initial set of drug distribution data to be cleaned, the cleaning process system also includes:

[0310] Data import module 6 is used to import initial drug distribution data into a global temporary table;

[0311] Data processing module 5 is used to perform corresponding read or write operations based on a global temporary table using a preset standardized processing flow corresponding to the initial drug distribution data, so as to obtain the cleaned and standardized target drug distribution data.

[0312] In one feasible solution, the cleaning system also includes:

[0313] Data acquisition module 7 is used to acquire several initial drug distribution data from different sources;

[0314] Data scheduling module 8 is used to allocate different initial drug distribution data to different file cleaning queue areas based on pre-configured automatic file queuing rules, so as to determine the task processing batch corresponding to each file cleaning queue area;

[0315] The cleaning control module 9 is used to perform file data cleaning operations sequentially according to the order of task processing batches.

[0316] In one feasible solution, the cleaning system also includes:

[0317] External operation acquisition module 10 is used to acquire external trigger operations;

[0318] Data selection module 11 is used to select several sets of initial drug distribution data to be cleaned based on external trigger operations;

[0319] In one feasible solution, the first display module 12 is used to display the first associated data of each batch of task processing involved in the queue;

[0320] In one feasible solution, the second display module 13 is used to display the second associated data corresponding to each initial drug distribution data.

[0321] In one feasible solution, the cleaning system also includes:

[0322] Concurrent execution module 14 is used to concurrently perform cleaning and processing of different initial drug distribution data in the database server;

[0323] In one feasible solution, the resource isolation control module 15 is used to isolate the resources between the background tasks that process different drug distribution data in the database server and the main program in the database server.

[0324] In one feasible approach, the database server and application server in the data cleaning and processing system are disconnected from each other.

[0325] For the system embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs.

[0326] Example 5

[0327] Figure 11 This is a schematic diagram of the structure of an electronic device according to an example embodiment of the present disclosure. The electronic device includes a memory, a processor, and a computer program stored in the memory and used to run on the processor. When the processor executes the computer program, it implements the method described in any of the above embodiments. Figure 11 The electronic device 90 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0328] like Figure 11 As shown, the electronic device 90 can be manifested as a general-purpose computing device, such as a server device. The components of the electronic device 90 may include, but are not limited to: at least one processor 91, at least one memory 92, and a bus 93 connecting different system components (including memory 92 and processor 91).

[0329] Bus 93 includes a data bus, an address bus, and a control bus.

[0330] The memory 92 may include volatile memory, such as random access memory (RAM) 921 and / or cache memory 922, and may further include read-only memory (ROM) 923.

[0331] The memory 92 may also include a program tool 925 (or utility) having a set (at least one) program module 924, such program module 924 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0332] The processor 91 executes various functional applications and data processing, such as the methods provided in any of the above embodiments, by running computer programs stored in the memory 92.

[0333] Electronic device 90 can also communicate with one or more external devices 94 (e.g., keyboard, pointing device, etc.). This communication can be performed through input / output (I / O) interface 95. Furthermore, electronic device 90 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public network, such as the Internet) via network adapter 96. As shown, network adapter 96 communicates with other modules of electronic device 90 via bus 93. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with electronic device 90, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems.

[0334] It should be noted that although several units / modules or sub-units / modules of the electronic device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.

[0335] Example 6

[0336] This disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method provided in any of the above embodiments.

[0337] The readable storage medium may be more specifically adopted, including but not limited to: portable disk, hard disk, random access memory, read-only memory, erasable programmable read-only memory, optical storage device, magnetic storage device, or any suitable combination thereof.

[0338] Example 7

[0339] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the method described in any of the above embodiments.

[0340] The program code for executing the computer program product of this disclosure can be written in any combination of one or more programming languages, and the program code can be executed entirely on a user device, partially on a user device, as a stand-alone software package, partially on a user device and partially on a remote device, or entirely on a remote device.

[0341] While specific embodiments of this disclosure have been described above, those skilled in the art should understand that these are merely illustrative examples, and the scope of protection of this disclosure is defined by the appended claims. Those skilled in the art can make various changes or modifications to these embodiments without departing from the principles and essence of this disclosure, but all such changes and modifications fall within the scope of protection of this disclosure.

Claims

1. A method of cleansing pharmaceutical distribution data, characterized by, The cleaning processing method is realized based on a database server in a data cleaning processing system, the database server is disconnected from communication with an application server in the data cleaning processing system, and a preset data cleaning tool is configured in the database server; The database server is programmed and developed in advance through a database technology language to obtain different types of preset data cleaning tools, different types of the preset data cleaning tools correspond to different data cleaning functions, and an Oracle database is used in the database server; The cleaning processing method comprises: In response to processing a plurality of initial drug distribution data by using the database server, usage rate data of a preset processor in the database server at different sampling times is obtained based on a preset sampling frequency; According to the usage rate data at different sampling times, current usage rate change state data of the preset processor is obtained; The step of obtaining the current usage rate change state data of the preset processor according to the usage rate data at different sampling times comprises: The difference value of the usage rate data corresponding to two adjacent continuous sampling times of the preset processor is calculated; The ratio between the difference value and the time interval corresponding to the two adjacent continuous sampling times is obtained as the current usage rate change state data of the preset processor; Based on the usage rate change state data, a matching task processing adjustment strategy is generated; Based on the task processing adjustment strategy, the number threshold of concurrent channels for task processing in the preset processor is dynamically adapted and adjusted; The preset processor is used to perform data cleaning processing according to the preset data cleaning tool, so as to obtain standardized target drug distribution data after cleaning of each initial drug distribution data; The step of generating a matching task processing adjustment strategy based on the usage rate change state data comprises: In response to the usage rate change state data representing a downward trend, a first number of concurrent channels is added based on the current number threshold of concurrent channels in the preset processor; In response to the usage rate change state data representing a stable or upward trend, the upward slope being less than a first preset value, and the usage rate data at the corresponding sampling times being less than a second preset value, it is determined to maintain the current number threshold of concurrent channels in the preset processor; In response to the usage rate change state data representing an upward trend, the upward slope being greater than or equal to the first preset value, and / or the usage rate data at the corresponding sampling times being greater than or equal to the second preset value, a second number of concurrent channels is reduced based on the current number threshold of concurrent channels in the preset processor; In response to the usage rate change state data representing an upward trend and the usage rate data at the corresponding sampling times being greater than a third preset value, it is determined to clear the concurrent channels in the preset processor; wherein the third preset value is greater than the second preset value; The cleaning processing method further comprises: Based on the usage change state data and a pre-configured third mapping relationship, a new preset sampling frequency matching the usage change state data is dynamically determined, and a subsequent sampling operation is performed using the updated new preset sampling frequency; The database server is configured with a global temporary table; wherein the global temporary table is used to store session-level temporary tables and / or transaction-level temporary tables; For any one of the initial drug distribution data to be cleaned, the cleaning processing method comprises: The initial drug distribution data is called into the global temporary table; A preset standardization processing flow corresponding to the initial drug distribution data is used to perform corresponding read or write operations based on the global temporary table to obtain the cleaned and standardized target drug distribution data; The step of increasing the first number of concurrent channels based on the current number threshold of concurrent channels in the preset processor comprises: According to different degrees of downward trend of the usage change state data, a pre-configured first mapping relationship is used to increase the matching first number of concurrent channels based on the current number threshold of concurrent channels in the preset processor; Wherein, the degree of downward trend is positively correlated with the first number; And / or, The step of reducing the second number of concurrent channels based on the current number threshold of concurrent channels in the preset processor comprises: According to different degrees of upward trend of the usage change state data, a pre-configured second mapping relationship is used to reduce the matching second number of concurrent channels based on the current number threshold of concurrent channels in the preset processor; Wherein, the degree of upward trend is positively correlated with the second number.

2. The method of claim 1, wherein the data cleaning process is performed by a data cleaning program. The cleaning processing method further comprises: Obtaining several initial drug distribution data of different sources; Based on pre-configured file automatic queuing rules, the different initial drug distribution data is allocated to different file cleaning queuing areas to determine the task processing batch corresponding to each file cleaning queuing area; File data cleaning operations are performed in sequence according to the order of the task processing batch.

3. The method of claim 2, wherein the data cleaning process is performed by a data cleaning program. The cleaning processing method further comprises: Obtaining an external trigger operation; Based on the external trigger operation, several initial drug distribution data to be cleaned are selected for cleaning; And / or, Displaying the first associated data of each task processing batch participating in queuing; And / or, Displaying the second associated data corresponding to each initial drug distribution data.

4. The method of claim 1, wherein the data cleaning process is performed on the drug distribution data. The cleaning processing method further comprises: Concurrently executing cleaning processing of different initial drug distribution data in the database server; And / or, Isolating resources between the background task in the database server processing different drug distribution data and the main program in the database server.

5. A system for cleansing pharmaceutical distribution data, characterized by, The cleaning processing system is implemented based on the database server in the data cleaning processing system, the database server is disconnected from the application server in the data cleaning processing system, and a preset data cleaning tool is configured in the database server; The database server is programmed and developed in advance by a database technology language to obtain different types of preset data cleaning tools corresponding to different data cleaning functions, and the database server adopts an Oracle database; The cleaning processing system comprises: A data sampling module is configured to sample usage rate data of a preset processor in the database server at different sampling time points based on a preset sampling frequency in response to processing a plurality of initial drug distribution data by using the database server; A change state acquisition module is configured to acquire current usage rate change state data of the preset processor according to the usage rate data at different sampling time points; The change state acquisition module comprises: A difference calculation unit is configured to calculate a difference between the usage rate data of the preset processor at two adjacent continuous sampling time points; A change state acquisition unit is configured to acquire a ratio between the difference and a time interval corresponding to the two adjacent continuous sampling time points as the current usage rate change state data of the preset processor; An adjustment strategy determination module is configured to generate a matched task processing adjustment strategy based on the usage rate change state data; An adjustment control module is configured to dynamically adjust a number threshold of concurrent channels for task processing in the preset processor based on the task processing adjustment strategy; A data processing module is configured to perform data cleaning processing on the initial drug distribution data by using the preset processor after adjustment according to the preset data cleaning tool to obtain standardized target drug distribution data after cleaning of each initial drug distribution data; The adjustment strategy determination module is further configured to: in response to the usage rate change state data representing a downward trend, increase a first number of concurrent channels based on a current number threshold of concurrent channels in the preset processor; in response to the usage rate change state data representing a stable or upward trend, an upward slope being less than a first preset value, and the usage rate data at the corresponding sampling time points being less than a second preset value, maintain the current number threshold of concurrent channels in the preset processor; in response to the usage rate change state data representing an upward trend, the upward slope being greater than or equal to the first preset value, and / or the usage rate data at the corresponding sampling time points being greater than or equal to the second preset value, reduce a second number of concurrent channels based on the current number threshold of concurrent channels in the preset processor; in response to the usage rate change state data representing an upward trend and the usage rate data at the corresponding sampling time points being greater than a third preset value, clear the concurrent channels in the preset processor; the third preset value is greater than the second preset value; The cleaning processing system further comprises: An update module is configured to dynamically determine a new preset sampling frequency matched with the usage rate change state data based on the usage rate change state data and a third mapping relationship configured in advance, and perform a subsequent sampling operation by using the updated new preset sampling frequency. The database server is configured with a global temporary table; wherein the global temporary table is used to store a session-level temporary table and / or a transaction-level temporary table; For any one of the initial drug distribution data to be cleaned, the cleaning processing system further comprises: a data call-in module, configured to call in the initial drug distribution data into the global temporary table; a data processing module, configured to perform corresponding read operation or write operation based on the global temporary table by using a preset standardization processing procedure corresponding to the initial drug distribution data, to obtain the target drug distribution data after cleaning and standardization; The adjustment strategy determination module is further configured to: According to different descending trend degrees of the usage rate change state data, increase the matched first number of concurrent channels based on the current number threshold of the concurrent channels in the preset processor according to a preconfigured first mapping relationship; Wherein, the descending trend degree is positively correlated with the first number; And / or, The adjustment strategy determination module is further configured to: According to different ascending trend degrees of the usage rate change state data, reduce the matched second number of concurrent channels based on the current number threshold of the concurrent channels in the preset processor according to a preconfigured second mapping relationship; Wherein, the ascending trend degree is positively correlated with the second number.

6. An electronic device comprising a memory, a processor, and a computer program stored on the memory for running on the processor, characterized in that, The processor executes the computer program to implement the drug distribution data cleaning processing method in any one of claims 1 to 4.

7. A computer-readable storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the drug distribution data cleaning processing method in any one of claims 1 to 4.

8. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the drug distribution data cleaning processing method in any one of claims 1 to 4.

Images