Data processing method, device, equipment, medium and program product

Abstract

The present disclosure provides a data processing method that can be applied in the information security or financial fields. The method includes: determining a first storage area for each of N applications; determining a target production database corresponding to the first storage area of each application, wherein each of the M storage areas corresponds to at least one production database; and obtaining first application data from the target production database of each application as the test data, wherein the first application data includes application data stored in the first storage area for each application based on an association relationship. The present disclosure also provides a data processing apparatus, device, storage medium, and program product.

Description

Technical Field

[0001] The present disclosure relates to the field of information security or finance, and more specifically, to a data processing method, apparatus, device, medium, and program product. Background Art

[0002] During testing, data generated by the application under test in the production environment is often used for testing. The production environment is the operating environment where the application under test provides services to users. For example, data from the production environment can be restored to the test environment to improve test accuracy. Furthermore, since the test environment may have limited resources, it is possible to selectively restore data from the production environment to reduce the data footprint of the test environment compared to the production environment.

[0003] Each application can store its production data in a corresponding database cluster. When testing multiple applications with related data, you may arbitrarily select a database for each application and restore the data in the selected database to the test environment.

[0004] In the process of realizing the concept of the present disclosure, the inventors discovered that there are at least the following problems in the related technology: there is no unified data distribution rule when each application stores data in the production environment, and there is no unified data extraction rule when acquiring test data. The databases selected by each application may store unrelated data, and the unrelated data is not available when restored to the test environment, resulting in a waste of time and resources and reduced testing efficiency. Summary of the Invention

[0005] In view of the above problems, the present disclosure provides a data processing method, apparatus, device, medium and program product for improving the relevance of extracted data.

[0006] One aspect of an embodiment of the present disclosure provides a data processing method, including: determining a first storage area for each of N applications, wherein each application corresponds to M storage areas, the first storage area is any one of the M storage areas, and N and M are integers greater than or equal to 1; determining a target production database corresponding to the first storage area of each application, wherein each of the M storage areas corresponds to at least one production database; and obtaining first application data from the target production database of each application as the test data, wherein the first application data includes application data of each application stored in the first storage area according to an association relationship.

[0007] According to an embodiment of the present disclosure, before obtaining the first application data, each application also stores the first application data, specifically including: obtaining an association identifier from the first application data to be stored; based on the association identifier, determining a first storage area corresponding to the association identifier from the association relationship, wherein the association relationship includes a correspondence between at least one association identifier and a storage area; routing the first application data to be stored to the first storage area for storage in the target production database.

[0008] According to an embodiment of the present disclosure, determining the first storage area corresponding to the association identifier from the association relationship based on the association identifier includes: obtaining the corresponding hash value using a hash algorithm according to the association identifier; and determining the first storage area identifier corresponding to the hash value from a mapping table, wherein the mapping table includes the association relationship.

[0009] According to an embodiment of the present disclosure, the method further includes: for any one of the N applications, if the first storage area corresponds to multiple target production databases, restoring the first application data in multiple target production databases to the same test database.

[0010] According to an embodiment of the present disclosure, in the case where the first storage area corresponds to a single target production database, before obtaining the first application data from the target production database of each application, it also includes: if the number of storage areas corresponding to the target production database of any one of the N applications is multiple, obtaining the union of the storage areas corresponding to the target production databases of each of the N applications; for each application, determining the corresponding R production databases based on the storage areas in the union, wherein the R production databases include the target production database, and R is an integer greater than or equal to 2; and using the R production databases of each application as R new target production databases.

[0011] According to an embodiment of the present disclosure, obtaining the first application data from the target production database of each application includes: obtaining the first application data from R new target production databases; wherein, the method also includes: restoring the first application data in the R new target production databases to the same test database.

[0012] According to an embodiment of the present disclosure, before restoring the first application data to the same test database, it also includes: obtaining the auto-increment sequence of at least one database table in each of the target production databases; in the case where there is a duplicate auto-increment sequence between at least two of the target production databases, modifying the duplicate auto-increment sequence.

[0013] Another aspect of an embodiment of the present disclosure provides a data processing device, including: a first determination module, used to determine a first storage area for each of N applications, wherein each application corresponds to M storage areas, the first storage area is any one of the M storage areas, and N and M are integers greater than or equal to 1 respectively; a second determination module, used to determine a target production database corresponding to the first storage area of each application, wherein each of the M storage areas corresponds to at least one production database; a data acquisition module, used to acquire first application data from the target production database of each application as the test data, wherein the first application data includes application data of each application stored in the first storage area according to an association relationship.

[0014] Another aspect of an embodiment of the present disclosure provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the method described above.

[0015] Another aspect of the embodiments of the present disclosure further provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the method described above.

[0016] Another aspect of the embodiments of the present disclosure further provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0017] One or more of the above embodiments have the following beneficial effects: When storing application data, each application can store the application data in the same storage area based on associations. During data extraction, the target production database corresponding to each application's first storage area is determined based on the first storage area, thereby obtaining the first application data in the target production database. This allows for unified data extraction rules to establish associations between the first application data extracted from multiple applications, enabling testing of multiple applications in a test environment and improving testing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above contents and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0019] Figure 1 The following schematically illustrates an application scenario of a data processing method according to an embodiment of the present disclosure;

[0020] Figure 2 The following schematically shows a flow chart of a data processing method according to an embodiment of the present disclosure;

[0021] Figure 3 Schematically shows a flow chart of storing first application data according to an embodiment of the present disclosure;

[0022] Figure 4 Schematically shows a flow chart for determining a first storage area according to an embodiment of the present disclosure;

[0023] Figure 5 Schematically shows a flow chart for determining a new target production database according to an embodiment of the present disclosure;

[0024] Figure 6 Schematically shows a flow chart for preventing data conflicts according to an embodiment of the present disclosure;

[0025] Figure 7 The following schematically shows a structural block diagram of a data processing device according to an embodiment of the present disclosure;

[0026] Figure 8 The block diagram schematically shows an electronic device suitable for implementing the data processing method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0027] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In the detailed description below, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0028] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise," "include," etc. used herein indicate the presence of the features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0029] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0030] When expressions such as "at least one of A, B and C, etc." are used, they should generally be interpreted in accordance with the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0031] In the technical solution disclosed herein, the user-related application data involved is collected, stored, used, processed, transmitted, provided, disclosed and applied with the user's authorization, which complies with the provisions of relevant laws and regulations, takes necessary confidentiality measures, and does not violate public order and good morals.

[0032] For example, a user may use application A, application B, and application C. Each application can generate and store application data based on user operations. Each application can have multiple databases for storing data, as shown in Table 1.

[0033] Table 1

[0034]

[0035] Referring to Table 1, the related art fails to consider the testing process when storing data. There is no unified data distribution rule when storing data in a production environment. For example, User A's data may be stored in databases A1, B3, and C8. When extracting test data, there is no unified data extraction rule based on the data distribution rule, and the data may be extracted from databases A1, B1, B2, C1, C2, C5, and C6 (for example only). However, User A's data is only in A1. When testing with User A's data, the data in other databases is unavailable due to lack of relevance, wasting time and resources.

[0036] An embodiment of the present disclosure provides a data processing method. The method includes: determining the first storage area of each application in N applications, wherein each application corresponds to M storage areas, the first storage area is any one of the M storage areas, and N and M are integers greater than or equal to 1. Determine the target production database corresponding to the first storage area of each application, wherein each storage area in the M storage areas corresponds to at least one production database. Obtain first application data from the target production database of each application as test data, wherein the first application data includes application data stored in the first storage area for each application according to an association relationship.

[0037] According to the embodiments of the present disclosure, when storing application data, each application can store the application data in the same storage area based on the association relationship. When extracting data, the target production database corresponding to the first storage area of each application is determined based on the first storage area, thereby obtaining the first application data in the target production database. This allows the first application data extracted from multiple applications to be correlated through unified data extraction rules, allowing them to be used in a test environment to test multiple applications, thereby improving testing efficiency.

[0038] It should be noted that the data processing method, apparatus, device, medium, and program product disclosed herein can be used for product testing in the information security field, as well as in testing scenarios in the financial field or other fields. This disclosure does not limit the application field.

[0039] Figure 1 The application scenario diagram of the data processing method according to an embodiment of the present disclosure is schematically shown.

[0040] like Figure 1 As shown, the application scenario 100 according to this embodiment may include N application servers 111, 112, 113, ... 11N, where each application server may be deployed with an application, and each application server may store data in a corresponding database cluster, such as database clusters 121, 122, 123, ... 12N. The application scenario 100 may also include terminal devices 131, 132, 133, and a network 140.

[0041] Each database cluster may include one or more databases. Network 140 is used to provide a medium for communication links between terminal devices 131, 132, and 133 and any of the N application servers. Network 140 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0042] In some embodiments, the object under test can be one or more distributed applications, each hosted on a different application server, working together over the network to complete a task. These distributed applications can store interrelated user data. From a user perspective, a user's information is distributed and stored across different applications, further optimizing the integrity and consistency of data across these interrelated applications.

[0043] Users can use terminal devices 131, 132, and 133 to interact with any of the N application servers via network 140 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 131, 132, and 133, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples). It should be noted that users can use terminal devices 131, 132, and 133 to interact with the registration center server 120 and the message server 130, and this disclosure does not specifically limit this.

[0044] The terminal devices 131 , 132 , and 133 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0045] The N application servers may be servers that provide various services, such as background management servers (for example only) that support websites browsed by users using terminal devices 131, 132, and 133. The background management servers may analyze and process received data such as user requests, and feed back processing results (e.g., web pages, information, or data obtained or generated based on user requests) to the terminal devices.

[0046] According to the embodiment of the present disclosure, for example, when testing applications in application servers 111, 112, and 113, the database clusters 121, 122, and 123 corresponding to each application store application data in the production environment. Figure 1 Deploy the test environment with other servers and database clusters in the cluster for testing.

[0047] It should be understood that Figure 1 The number of terminal devices, networks, servers, and database clusters in the embodiment is merely illustrative. Any number of terminal devices, networks, servers, and database clusters may be provided as required.

[0048] The following will be based on Figure 1 The scene described by Figures 2 to 6 The data processing method of the embodiment of the present disclosure is described in detail.

[0049] Figure 2 The flowchart of the data processing method according to the embodiment of the present disclosure is schematically shown.

[0050] like Figure 2 As shown, the data processing method of this embodiment may include operations S210 to S230.

[0051] In operation S210 , a first storage area of each of N applications is determined, wherein each application corresponds to M storage areas, the first storage area is any one of the M storage areas, and N and M are integers greater than or equal to 1.

[0052] Referring to Table 2, Table 2 exemplarily shows the content of dividing the storage area of the database of each application in an embodiment of the present disclosure.

[0053] Table 2

[0054]

[0055] Taking storage area "1 / 8" as an example, it indicates that there are a total of 8 storage areas, and the order of this area is 1. It should be noted that the number of applications, production databases, and storage areas, as well as the corresponding relationship between storage areas and production databases in Table 2 are only examples and can be set according to actual conditions. This disclosure does not specifically limit them.

[0056] As shown in Table 2, each application's production database can be divided into eight storage areas. For example, one production database of application C corresponds to one storage area. Meanwhile, one database of application B corresponds to two storage areas, and one production database of application A corresponds to four storage areas. On the one hand, the storage area division can be considered to have the same number of areas for applications A, B, and C. On the other hand, the number of storage areas can be determined first based on application C, which has the largest amount of production data.

[0057] In operation S220 , a target production database corresponding to the first storage area of each application is determined, wherein each storage area of the M storage areas corresponds to at least one production database.

[0058] Exemplarily, "each storage area corresponds to at least one production database" means that at least one production database can be determined from Table 2 for each storage area. For example, the storage area labeled 1 / 8 is first determined from the storage area as the first storage area. Then, the target production database for application A is A1, the target production database for application B is B1, and the target production database for application C is C1.

[0059] In operation S230 , first application data is acquired from the target production database of each application as test data, wherein the first application data includes application data of each application stored in the first storage area according to an association relationship.

[0060] For example, application data in A1, B1, and C1 are obtained as test data. For example, the association relationship can be determined based on the user dimension. When processing application data related to user A, application A, application B, and application C can all store it according to the corresponding relationship between user A and the first storage area.

[0061] According to the embodiments of the present disclosure, when storing application data, each application can store the application data in the same storage area based on the association relationship. When extracting data, the target production database corresponding to the first storage area of each application is determined based on the first storage area, thereby obtaining the first application data in the target production database. This allows the first application data extracted from multiple applications to be correlated through unified data extraction rules, allowing multiple applications to be tested in a test environment, thereby improving testing efficiency.

[0062] Figure 3 The flowchart of storing first application data according to an embodiment of the present disclosure is schematically shown.

[0063] Before obtaining the first application data in operation S230, each application also stores the first application data, such as Figure 3 As shown, storing the first application data in this embodiment may include operations S310 to S330.

[0064] In operation S310 , an association identifier is obtained from the first application data to be stored.

[0065] For example, the association identifier may be a pre-set field, such as a user number, or a combination of a user number and a region, or a uuid number generated based on the user number, etc. The specific means of obtaining the field is not limited.

[0066] In operation S320 , based on the association identifier, a first storage area corresponding to the association identifier is determined from the association relationship, wherein the association relationship includes a correspondence between at least one association identifier and a storage area.

[0067] For example, if application A is a mobile banking application, user A initiates a transaction. Application A can store the transaction data by first obtaining user A's ID. Then, based on user A's ID, it searches for the corresponding storage area. Finally, it routes the transaction execution log to the searched storage area.

[0068] Figure 4 The flowchart of determining the first storage area in operation S320 according to an embodiment of the present disclosure is schematically shown.

[0069] like Figure 4As shown, determining the first storage area corresponding to the association identifier from the association relationship based on the association identifier in operation S320 may include operations S410 to S420.

[0070] In operation S410, a corresponding hash value is obtained using a hash algorithm according to the association identifier.

[0071] In operation S420, a first storage area identifier corresponding to the hash value is determined from a mapping table, wherein the mapping table includes an association relationship.

[0072] For example, from a user perspective, association identifiers for different users may be duplicated. If the same association identifier corresponds to different storage areas, this can lead to confusion of user data or routing errors during storage. The hash value serves as a unique identifier for each user. The first storage area identifier can be used to obtain the address of the database in the first storage area.

[0073] According to an embodiment of the present disclosure, a mapping table with association relationships written therein may be pre-set, wherein when each application stores data in a production environment, the content in the mapping table is used as a unified data distribution rule.

[0074] For example, first, a corresponding storage area is allocated to each user of the application. For example, users whose user numbers end in 1 correspond to storage area 1 / 8 in Table 2, users whose user numbers end in 2 correspond to storage area 2 / 8 in Table 2, and so on. Users whose user numbers end in 8, 9, and 0 correspond to storage area 8 / 8 in Table 2. Then, the user numbers of each user are hashed, or a field is added after the user number, such as a field for the number of the corresponding storage area, and a hash value is obtained using a hash algorithm. Finally, the correspondence between the hash value and the storage area is written in the mapping table. The preset algorithm may include a consistent hashing algorithm, an MD5 algorithm, etc.

[0075] In operation S330 , the first application data to be stored is routed to the first storage area to be stored in the target production database.

[0076] Exemplarily, when one storage area corresponds to one production database, the first application data may be directly sent to the production database.

[0077] For example, when a storage area corresponds to multiple production databases, storage can be performed based on a load balancing strategy, such as considering the storage space available in each production database. After a user's application data is stored in a production database, it can be directly sent to the production database later.

[0078] In related technologies, each application may have its own specific data distribution rules when storing application data, making it difficult to effectively extract data related to each application. The data processing method of the disclosed embodiment can set a mapping table as a unified data distribution rule for each application, thus storing data related to each application in a traceable manner. This facilitates the development of effective data extraction rules based on the data distribution rules, improving the availability of test data.

[0079] It should be noted that the above description of the data distribution rules of the embodiment of the present disclosure based on the user dimension is only an example. Data distribution can also be performed from the business dimension, time dimension, etc., and the present disclosure does not make specific limitations.

[0080] Figure 5 The flowchart of determining a new target production database according to an embodiment of the present disclosure is schematically shown.

[0081] According to an embodiment of the present disclosure, in the case where the first storage area corresponds to a single target production database, before obtaining the first application data from the target production database of each application, the method further includes: if the number of storage areas corresponding to the target production database of any application among the N applications is multiple, executing the following Figure 5 Operations S510 to S530 in the process are used to determine a new target production database.

[0082] Referring to Table 2, the first storage area corresponding to a single target production database means that if the storage area is determined first and then the target production database is determined based on the storage area, then for each application, each storage area can correspond to a corresponding target production database. The fact that any application has multiple target production databases corresponding to multiple storage areas means that if a single database is determined first, multiple storage areas may be generated based on that database.

[0083] For example, if the storage area 1 / 8 is determined first, A1, B1, and C1 can be obtained in applications A, B, and C respectively. If A1 is determined first, multiple storage areas can be obtained, such as 1 / 8, 2 / 8, 3 / 8, and 4 / 8.

[0084] In operation S510 , a union of storage areas corresponding to target production databases of respective applications in N applications is obtained.

[0085] For example, referring to Table 2, first, storage area 1 / 8 is determined from the storage area as the first storage area. Then, the target production database of application A is A1, the target production database of application B is B1, and the target production database of application C is C1.

[0086] Next, reverse-locate the corresponding storage areas based on each target production database. A1 corresponds to four storage areas, such as 1 / 8, 2 / 8, 3 / 8, and 4 / 8. B1 corresponds to two storage areas, such as 1 / 8 and 2 / 8. C1 corresponds to one storage area, such as 1 / 8.

[0087] Therefore, the union of the storage areas includes 1 / 8, 2 / 8, 3 / 8, and 4 / 8.

[0088] In operation S520 , for each application, corresponding R production databases are determined according to the storage areas in the union, where the R production databases include the target production database, and R is an integer greater than or equal to 2.

[0089] After obtaining the union of storage areas, reversely identify the R production databases corresponding to the unioned storage areas. The value of R can be different for each application. For example, application A has one target production database A1, application B has two target production databases B1, and application C has four target production databases C1 through C4.

[0090] In operation S530 , the R production databases of each application are used as R new target production databases.

[0091] According to the embodiments of the present disclosure, the purpose of determining a new target production database is to improve data availability. Specifically, if application data is obtained from A1, B1, and C1, only the application data routed according to storage area 1 / 8 will be relevant. However, A1 also stores application data routed according to 2 / 8, 3 / 8, and 4 / 8, and this data may not be useful for testing. Therefore, by determining a new target production database, the extracted data from each application is correlated, thereby improving data availability.

[0092] In some embodiments, the database storage location in the production environment can be simulated, and the data of the R production databases of each application can be restored one by one to the corresponding R test databases.

[0093] In other embodiments, obtaining the first application data from the target production database of each application includes: obtaining the first application data from R new target production databases for each application. In addition to operations S210 to S230, the data processing method of the disclosed embodiment may further include: restoring the first application data from the R new target production databases to the same test database.

[0094] For example, data from C1 to C4 can be restored to a test database under application C, and data from B1 and B2 can be restored to a test database under application B. This effectively reduces the number of test databases. Furthermore, even if each application has a different number of production databases, maintaining a consistent number of test databases for each application ensures database reduction within the test environment, reduces data usage, and ensures data validity.

[0095] According to an embodiment of the present disclosure, for any one of N applications, when the first storage area corresponds to multiple (eg, more than two) target production databases, the first application data in the multiple target production databases is restored to the same test database.

[0096] Referring to Table 3, Table 3 exemplarily shows the content of dividing the storage area of the database of each application according to another embodiment of the present disclosure.

[0097] Table 3

[0098]

[0099] As shown in Table 3, each application's production database can be divided into four storage areas. For example, one storage area for application C corresponds to two production databases. If storage area 1 / 4 is selected as the first storage area, application C will target two production databases, namely C1 and C2. Therefore, the data in C1 and C2 can be restored to the same test database, effectively reducing the number of test databases. In addition, even if each application has a different number of production databases, if the number of test databases for each application is consistent, the requirement for rapid test environment downsizing can be met while also ensuring data validity.

[0100] Figure 6 The flowchart of preventing data conflict according to an embodiment of the present disclosure is schematically shown.

[0101] like Figure 6 As shown, before restoring the first application data in multiple target production databases to the same test database, operations S610 to S620 may be further included to prevent data conflicts.

[0102] In operation S610 , an auto-increment sequence of at least one database table in each target production database is obtained.

[0103] In operation S620 , when duplicate auto-increment sequences exist between at least two target production databases, the duplicate auto-increment sequences are modified.

[0104] For example, the more production databases selected for data recovery for each application, the more application data that needs to be restored, and the more likely it is to improve data validity. Data slimming and data validity are contradictory. If data from multiple production databases is restored to the same test database, under the premise of satisfying data slimming, the improvement of data validity is analyzed. On the one hand, from the perspective of production databases, each production database is independent of each other, and data is stored in only one database, so there is no data conflict in the data of each database. On the other hand, there is an auto-increment sequence in the database table, and there is data conflict in the auto-increment sequence of each database.

[0105] For example, data conflicts in auto-increment sequences mean that for each production database, when defining the primary key of the database table, a new ID field is created that automatically increases, and each database table uses the automatically generated number as the primary key. When the various production databases are integrated, such as restoring them to the same test database, there may be primary key conflicts between tables from different production databases, resulting in data unavailability. The purpose of modifying the repeated auto-increment sequences is to modify the primary keys of the database tables that have primary key conflicts to overcome the conflict problem and improve the validity of the data.

[0106] Based on the above data processing method, the present disclosure also provides a data processing device. Figure 7 The device is described in detail.

[0107] Figure 7 The structural block diagram of the data processing device according to an embodiment of the present disclosure is schematically shown.

[0108] like Figure 7 As shown, the data processing device 700 of this embodiment includes a first determination module 710 , a second determination module 720 and a data acquisition module 730 .

[0109] The first determination module 710 can perform operation S210 to determine a first storage area for each of N applications, where each application corresponds to M storage areas, the first storage area is any one of the M storage areas, and N and M are integers greater than or equal to 1 respectively.

[0110] The second determining module 720 may perform operation S220 to determine a target production database corresponding to the first storage area of each application, wherein each storage area of the M storage areas corresponds to at least one production database.

[0111] The data acquisition module 730 may perform operation S230 to acquire first application data from the target production database of each application as test data, wherein the first application data includes application data of each application stored in the first storage area according to an association relationship.

[0112] The data processing device 700 may further include a data storage module. The data storage module may perform operations S310 to S330 to obtain an association identifier from the first application data to be stored before the data acquisition module 730 acquires the first application data. Based on the association identifier, a first storage area corresponding to the association identifier is determined from an association relationship, where the association relationship includes a correspondence between at least one association identifier and a storage area. The first application data to be stored is routed to the first storage area for storage in the target production database.

[0113] The data storage module may perform operations S410 to S420 to obtain a corresponding hash value using a hash algorithm according to the association identifier, and determine a first storage area identifier corresponding to the hash value from a mapping table, wherein the mapping table includes an association relationship.

[0114] The data processing device 700 may further include a third determination module. The third determination module may perform operations S510 to S530. If the number of storage areas corresponding to the target production database of any application among the N applications is multiple, the third determination module is configured to obtain a union of the storage areas corresponding to the target production databases of each application in the N applications. For each application, based on the storage areas in the union, the corresponding R production databases are determined, where the R production databases include the target production database, and R is an integer greater than or equal to 2. The R production databases of each application are used as the R new target production databases.

[0115] The data processing device 700 may further include a modification module. The modification module may perform operations S610 to S620 to obtain an auto-increment sequence for at least one database table in each target production database. If duplicate auto-increment sequences exist between at least two target production databases, the duplicate auto-increment sequences may be modified.

[0116] According to the embodiments of the present disclosure, data processing device 700 selects a portion of the database in each application's production environment and extracts data according to data extraction rules to achieve data slimming, thereby improving data validity. This effectively conserves test environment resources, improves data recovery efficiency, and saves manpower and time costs.

[0117] It should be noted that the implementation methods, technical problems solved, functions implemented, and technical effects achieved of each module in the device part embodiment are the same or similar to the implementation methods, technical problems solved, functions implemented, and technical effects achieved of each corresponding step in the method part embodiment, and will not be repeated here.

[0118] According to an embodiment of the present disclosure, any multiple modules among the first determination module 710, the second determination module 720, and the data acquisition module 730 may be combined into a single module for implementation, or any one of these modules may be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules may be combined with at least part of the functionality of other modules and implemented in a single module.

[0119] According to an embodiment of the present disclosure, at least one of the first determination module 710, the second determination module 720, and the data acquisition module 730 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware in any other reasonable manner of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in an appropriate combination of any of them. Alternatively, at least one of the first determination module 710, the second determination module 720, and the data acquisition module 730 can be at least partially implemented as a computer program module, which can perform the corresponding function when the computer program module is executed.

[0120] Figure 8 The block diagram schematically shows an electronic device suitable for implementing the data processing method according to an embodiment of the present disclosure.

[0121] like Figure 8 As shown, the electronic device 800 according to an embodiment of the present disclosure includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage part 808 into a random access memory (RAM) 803. The processor 801 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 801 may also include an onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0122] Various programs and data required for the operation of the electronic device 800 are stored in the RAM 803. The processor 801, ROM 802, and RAM 803 are connected to each other via a bus 804. The processor 801 executes the various operations of the method flow according to the embodiment of the present disclosure by executing the programs in the ROM 802 and / or RAM 803. It should be noted that the programs may also be stored in one or more memories other than the ROM 802 and RAM 803. The processor 801 may also execute the various operations of the method flow according to the embodiment of the present disclosure by executing the programs stored in the one or more memories.

[0123] According to an embodiment of the present disclosure, the electronic device 800 may further include an input / output (I / O) interface 805, which is also connected to the bus 804. The electronic device 800 may further include one or more of the following components connected to the I / O interface 805: an input portion 806 including a keyboard, a mouse, etc.; an output portion 807 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage portion 808 including a hard disk; and a communication portion 809 including a network interface card such as a LAN card or a modem. The communication portion 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed in the drive 810 as needed, so that a computer program read therefrom can be installed into the storage portion 808 as needed.

[0124] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments, or may exist independently and not be incorporated into the device / apparatus / system. The computer-readable storage medium carries one or more programs, and when executed, implements the method according to the embodiments of the present disclosure.

[0125] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM 802 and / or RAM 803 described above and / or one or more memories other than ROM 802 and RAM 803.

[0126] The embodiments of the present disclosure also include a computer program product, which includes a computer program containing program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the method provided by the embodiments of the present disclosure.

[0127] The computer program executes the above functions defined in the system / device of the embodiment of the present disclosure when the processor 801 executes the computer program. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0128] In one embodiment, the computer program may be stored on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may be transmitted and distributed in the form of a signal on a network medium, downloaded and installed via the communication portion 809, and / or installed from a removable medium 811. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0129] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 809, and / or installed from a removable medium 811. When the computer program is executed by the processor 801, the above-described functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the systems, devices, means, modules, units, etc. described above can be implemented by computer program modules.

[0130] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computer programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages include, but are not limited to, languages such as Java, C++, python, "C" or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect via the Internet).

[0131] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0132] Those skilled in the art will appreciate that the features described in the various embodiments and / or claims of this disclosure may be combined and / or coupled in various ways, even if such combinations and / or couplings are not explicitly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure may be combined and / or coupled in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or couplings are intended to fall within the scope of this disclosure.

[0133] The embodiments of the present disclosure are described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although each embodiment has been described separately above, this does not mean that the measures in each embodiment cannot be used in combination to advantage. The scope of the present disclosure is defined by the appended claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A data processing method, comprising: Determine a first storage area for each of N applications, wherein each application corresponds to M storage areas, the first storage area is any one of the M storage areas, and N and M are integers greater than or equal to 1; Determining a target production database corresponding to the first storage area of each application, wherein each storage area in the M storage areas corresponds to at least one production database; Acquiring first application data from a target production database of each application as test data includes: using a unified data extraction rule to associate first application data extracted from multiple applications, and testing multiple applications in a test environment, wherein the first application data includes application data stored in the first storage area for each application based on the association relationship.

2. The method according to claim 1, wherein Before obtaining the first application data, each application stores the first application data, specifically including: Obtaining an association identifier from the first application data to be stored; Based on the association identifier, determining a first storage area corresponding to the association identifier from the association relationship, wherein the association relationship includes a correspondence between at least one association identifier and a storage area; The first application data to be stored is routed to the first storage area to be stored in the target production database.

3. The method according to claim 2, wherein: The determining, based on the association identifier, from the association relationship, a first storage area corresponding to the association identifier includes: According to the association identifier, a corresponding hash value is obtained using a hash algorithm; Determine a first storage area identifier corresponding to the hash value from a mapping table, wherein the mapping table includes the association relationship.

4. The method according to claim 1, wherein The method further includes: for any one of the N applications, In a case where the first storage area corresponds to a plurality of the target production databases, the first application data in the plurality of the target production databases are restored to the same test database.

5. The method according to claim 1, wherein In a case where the first storage area corresponds to a single target production database, before obtaining the first application data from the target production database of each application, the method further includes: If the number of storage areas corresponding to the target production database of any of the N applications is multiple, Obtaining a union of storage areas corresponding to target production databases of each of the N applications; For each of the applications, determining corresponding R production databases according to the storage areas in the union, wherein the R production databases include the target production database, and R is an integer greater than or equal to 2; The R production databases of each application are used as R new target production databases.

6. The method according to claim 5, wherein: The acquiring of the first application data from the target production database of each application comprises: for each application, Acquire first application data from R new target production databases; The method further comprises: The first application data in the R new target production databases is restored to the same test database.

7. The method according to any one of claims 4 or 6, wherein Before restoring the first application data to the same test database, also include: Obtaining an auto-increment sequence of at least one database table in each of the target production databases; In the case that duplicate auto-increment sequences exist between at least two of the target production databases, the duplicate auto-increment sequences are modified.

8. A data processing device comprising: A first determining module is configured to determine a first storage area for each of N applications, wherein each application corresponds to M storage areas, the first storage area is any one of the M storage areas, and N and M are integers greater than or equal to 1; a second determining module, configured to determine a target production database corresponding to the first storage area of each application, comprising: correlating first application data extracted from multiple applications using a unified data extraction rule, and testing the multiple applications in a test environment, wherein each of the M storage areas corresponds to at least one production database; The data acquisition module is used to acquire first application data from the target production database of each application as test data, wherein the first application data includes the application data of each application stored in the first storage area according to the association relationship.

9. An electronic device comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors are enabled to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to perform the method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images