A data operation method based on a multi-system, a terminal device, and a storage medium

Abstract

The application relates to a multi-system-based data operation method, a terminal device and a storage medium, and the method comprises the following steps: initializing configuration information of databases corresponding to each system; generating a token according to encrypted user information after a user logs in; encrypting the token, generating a temporary key, and distributing the temporary key to each system for storage; when a data operation request sent by a system is received, the data operation request is distributed to the system by using load balancing; the system verifies the key in the data operation request, and when the verification is passed, data operation is performed on the corresponding database according to the data operation request, and the data obtained after the data operation is assembled according to a return data format, and then the system initiating the data operation request is returned. The application can improve the efficiency, security and stability of data operation between business systems.

Description

Technical Field

[0001] This invention relates to the field of data processing, and in particular to a data operation method, terminal device and storage medium based on multiple systems. Background Technology

[0002] With the rapid development of information technology, it has become a crucial driving force for the diversification of economic structures. While bringing enormous economic and social benefits, information technology also presents increasing threats to enterprises, posing new challenges to data security and privacy protection. Currently, enterprises are diversifying their services, including private clouds and public clouds. System architectures are becoming increasingly complex; in private cloud architectures, systems are increasingly overlapping and interacting; and different business models have their own data processing methods, resulting in diverse data processing formats, management difficulties, maintenance challenges, and high risks of data security threats between systems. Furthermore, different business models' data processing methods are incompatible with different data sources, making data access between systems difficult. Therefore, establishing a secure, efficient, and convenient unified data gateway system to meet the needs of different business models is of great significance.

[0003] Currently, many enterprise systems have their own data storage and processing methods for multiple products, creating a significant demand for data calls and responses between these systems. Traditional calling methods, which rely on verifying user information for each system or using unverified HTTP methods, are extremely insecure and vulnerable to attacks, potentially causing production losses and damage to the enterprise. Furthermore, the lack of a unified data format between systems leads to inefficient and difficult integration methods and poor compatibility. Specific shortcomings include:

[0004] 1) Data requirement development is inefficient, as each requirement requires the development of a corresponding function, resulting in a significant waste of manpower.

[0005] 2) The lack of a unified request distribution system can easily lead to service crashes.

[0006] 3) It cannot cope with the demand for large amounts of data in different scenarios.

[0007] 4) Low security and vulnerable to spoofed request attacks.

[0008] 5) The lack of a unified data format makes inter-system communication difficult, resulting in high maintenance costs in the later stages.

[0009] 6) Inflexible and inefficient data retrieval and response. Summary of the Invention

[0010] To address the aforementioned problems, this invention proposes a data operation method, terminal device, and storage medium based on multiple systems.

[0011] The specific plan is as follows:

[0012] A data manipulation method based on multiple systems includes the following steps:

[0013] S1: Initialize the configuration information of the databases corresponding to each system;

[0014] S2: After a user logs in, the user information is encrypted according to the first encryption rule. The encrypted user information is then matched with the authorized user information stored in the authentication and authorization database to determine whether the user information is valid. If valid, a token is generated based on the encrypted user information.

[0015] S3: Encrypt the token using the second encryption rule, generate a temporary key, and distribute the temporary key to each system for storage;

[0016] S4: When a data operation request is received from the system, determine whether the user corresponding to the data operation request is an authorized user based on the key in the data operation request. If so, grant the system access permission corresponding to the data operation request and set the user's access limit for the system. Each time a data operation request from the user to the system is received, check whether the number of requests exceeds the access limit. If so, stop receiving data operation requests from the user to the system; otherwise, use HAproxy load balancing to distribute the data operation request to the system corresponding to the data operation request.

[0017] S5: When the system receives a data operation request, it parses the parameters of the data operation request to obtain the key in the data operation request. It then verifies the key with the temporary key stored in the system. If the verification is successful, it performs data operation from the corresponding database according to the data operation request. Finally, it assembles the data obtained after the data operation according to the returned data format and returns it to the system that initiated the data operation request.

[0018] Furthermore, the system's corresponding databases include: HBase, MySQL, Elasticsearch, Hive, ClickHouse, and Hadoop.

[0019] Furthermore, the database configuration information includes: system name, IP address, port, username, login password, database connection, database version number, package path information, and returned data format.

[0020] Furthermore, the data operation request includes: key, operation option, operation statement optionValue, system name of the system to be operated on, databaseType of the database to be operated on, data to be operated on, and version number.

[0021] Furthermore, the access limit is the maximum number of times you can access the site per day.

[0022] Furthermore, the method for performing data operations from the corresponding database based on the data operation request is as follows:

[0023] S501: Parse the data operation request and obtain the contents of the six fields: databaseType, option, optionValue, data, system, and version;

[0024] S502: Obtain the corresponding version of the database based on the system, databaseType, and version fields, and obtain the database connection information from the database's corresponding configuration information;

[0025] S503: Select the corresponding version of the database JAR file and the required connection method under the JAR file based on the system, databaseType and version fields;

[0026] S504: Establish a connection to the database based on the database connection information, JAR package, and connection method;

[0027] S505: Perform database operations based on the option and optionValue fields and the data field.

[0028] A data operation terminal device based on multiple systems includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the method described above in the embodiments of the present invention.

[0029] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described above in the embodiments of the present invention.

[0030] The present invention adopts the above technical solution and has the following beneficial effects:

[0031] (1) Multi-business systems are based on a unified identity authentication method under single sign-on and provide a unified data verification gateway service to improve the efficiency, security and stability of data between business systems.

[0032] (2) A unified load balancing scheduling method is used to provide stability for data flow between various business systems, and to smooth out peaks during business periods, thereby improving the speed and stability of data flow between systems.

[0033] (3) It supports custom database operations, parameter configuration, and custom return data format of various mainstream databases and versions, which greatly increases flexibility and reduces the cost of manual secondary development. It can meet the needs of large amounts of data in different scenarios and improve enterprise efficiency.

[0034] (4) In terms of maintenance, the technology and code are unified, which greatly reduces the maintenance cost and can quickly meet the needs of different businesses. It has the advantages of high security, stable data, high concurrency and flexible dynamic configuration. Attached Figure Description

[0035] Figure 1 The diagram shown is a flowchart of Embodiment 1 of the present invention.

[0036] Figure 2 The diagram shown is a schematic of a MySQL query request in Embodiment 1 of the present invention.

[0037] Figure 3 The diagram shown is a schematic of a MySQL insert request in Embodiment 1 of the present invention.

[0038] Figure 4 The diagram shown is a schematic of a MySQL update request in Embodiment 1 of the present invention.

[0039] Figure 5 The diagram shown is a schematic of a MySQL delete request in Embodiment 1 of the present invention. Detailed Implementation

[0040] To further illustrate the various embodiments, the present invention provides accompanying drawings. These drawings are part of the disclosure of the present invention, primarily used to illustrate the embodiments, and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these drawings, those skilled in the art should be able to understand other possible implementations and the advantages of the present invention.

[0041] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments.

[0042] Example 1:

[0043] This invention provides a data manipulation method based on multiple systems, such as... Figure 1 As shown, the method includes the following steps:

[0044] S1: Initialize the configuration information of the databases corresponding to each system.

[0045] This embodiment is used to operate on multiple systems (such as WeChat, QQ, etc.), with each system corresponding to one or more databases.

[0046] The database types included in this embodiment are: HBase, MySQL, Elasticsearch, Hive, ClickHouse, and Hadoop. The configuration information for each database includes: system name (system), IP address, port (port), username (username), password (passport), database URL, database version number (version), package path (jarPath), and return data format (dataType).

[0047] S2: After a user logs in, the user information is encrypted according to the first encryption rule. The encrypted user information is then matched with the authorized user information stored in the authentication and authorization database to determine whether the user information is valid. If valid, a token is generated based on the encrypted user information.

[0048] In this embodiment, users log in through a unified user login interface.

[0049] The first encryption rule can be any commonly used data encryption rule, and no specific restrictions are imposed here.

[0050] The authentication and authorization database is used to store encrypted user information of authorized users.

[0051] If a match is found, the user information is deemed valid. If invalid, the process ends.

[0052] S3: Encrypt the token using the second encryption rule to generate a temporary key, and distribute the temporary key to each system for storage.

[0053] In this embodiment, the second encryption rule uses the AES-CBC algorithm.

[0054] A temporary key can cache an encrypted token with the username as the key for a user, which can be used for login-free verification when switching between systems. In other words, after verifying the temporary key, users can switch between different application systems without logging in.

[0055] S4: When a data operation request is received from the system, determine whether the user corresponding to the data operation request is an authorized user based on the key in the data operation request. If so, grant the system access permission corresponding to the data operation request and set the user's access limit for the system. Each time a data operation request for the system is received from the user, determine whether the number of requests exceeds the access limit. If so, stop receiving data operation requests for the system from the user; otherwise, use HAproxy load balancing to distribute the data operation request to the system corresponding to the data operation request.

[0056] If the user is determined not to be an authorized user, the data operation request is stopped, and the system waits to receive the next data operation request.

[0057] In this embodiment, the data operation request is a JSON formatted string, such as... Figure 2-5 Therefore, the content of the data operation request includes: key (the temporary key generated in step S3 above), the operation to be performed (such as add, delete, modify, search, etc.) option, operation statement optionValue, system name of the requested operation system, database type of the requested operation databaseType, data to be operated data and version number version.

[0058] Access limits are used to restrict the number of requests made within a fixed time period, such as the maximum number of accesses allowed per day. By limiting access limits, the pressure on the system server can be alleviated, while preventing malicious attacks by hackers and improving the stability of the application system.

[0059] By using HAProxy load balancing to distribute requests, the pressure and traffic on each system can be reduced.

[0060] S5: When the system receives a data operation request, it parses the parameters of the data operation request to obtain the key in the data operation request. It then verifies the key with the temporary key stored in the system. If the verification is successful, it performs data operation from the corresponding database according to the data operation request. Finally, it assembles the data obtained after the data operation according to the data format recorded in the data operation request and returns it to the system that initiated the request.

[0061] In this embodiment, the method for performing data operations from the corresponding database based on a data operation request is as follows:

[0062] S501: Parse the data operation request and obtain the contents of the six fields: databaseType, option, optionValue, data, system, and version.

[0063] like Figure 2 The parsed data yielded the following six fields: mysql, select, select usernamefrom user_info where id=1, cloudApi, data, and 5.7.1.

[0064] S502: Obtain the corresponding version of the database based on the system, databaseType, and version fields, and obtain the database connection information from the database's corresponding configuration information.

[0065] Connection information includes: IP address, port number, username, password, and database connection URL.

[0066] S503: Select the corresponding database JAR package (each version corresponds to a different JAR package) based on the system, databaseType, and version fields, as well as the required connection method under that JAR package.

[0067] S504: Establish a connection to the database based on the database connection information, JAR package, and connection method.

[0068] S505: Perform database operations based on the option and optionValue fields and the data field.

[0069] For example, the operation statement performed by optionValue is "insert into userInfo(column1,column2,..)VALUES()". VALUES() requires adding data by assembling the data "{values1,values2,..}" into the standard SQL insert syntax "insert into userInfo(column1,column2,..)VALUES(values1,values2,..)".

[0070] Different database types (e.g., HBase, Hadoop, Elasticsearch, Hive) have different syntax for the option value and data fields.

[0071] The returned data format, dataType, is obtained from the database configuration information.

[0072] The method described in this embodiment of the invention can achieve the following technical effects:

[0073] (1) Systems in different business forms within an enterprise have unified authentication permissions for login and single sign-on.

[0074] (2) Single-point token authentication is used for data calls between systems with different business models.

[0075] (3) Unify the data processing method to change the scenario where data processing forms are diverse, management is difficult, and maintenance is difficult between systems.

[0076] (4) Improve the security and stability of data between systems.

[0077] (5) Configurable data can greatly reduce the cost of secondary business development for developers and enterprise costs.

[0078] (6) Multiple data sources and multiple versions solve the compatibility problem between systems.

[0079] Example 2:

[0080] The present invention also provides a data operation terminal device based on multiple systems, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps in the method embodiment described above in Embodiment 1 of the present invention.

[0081] Furthermore, as an executable solution, the multi-system data operation terminal device can be a desktop computer, laptop, mobile phone, PDA, or other computing device. The multi-system data operation terminal device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that the above-described structure of the multi-system data operation terminal device is merely an example and does not constitute a limitation on the multi-system data operation terminal device. It may include more or fewer components than described above, or combine certain components, or different components. For example, the multi-system data operation terminal device may also include input / output devices, network access devices, buses, etc., and this embodiment of the invention does not limit this.

[0082] Furthermore, as an executable solution, the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. The processor is the control center of the multi-system-based data operation terminal device, connecting all parts of the multi-system-based data operation terminal device via various interfaces and lines.

[0083] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the multi-system data operation terminal device by running or executing the computer programs and / or modules stored in the memory and calling the data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created based on the use of the mobile phone, etc. In addition, the memory may include high-speed random access memory and non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0084] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the method described in the embodiments of the present invention.

[0085] If the modules / units integrated in the multi-system data operation terminal device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), and software distribution media, etc.

[0086] Although the invention has been specifically shown and described in conjunction with preferred embodiments, those skilled in the art should understand that various changes in form and detail may be made to the invention without departing from the spirit and scope of the invention as defined in the appended claims, all of which shall be within the scope of protection of the invention.

Claims

1. A data manipulation method based on multiple systems, characterized in that, Includes the following steps: S1: Initialize the configuration information of the databases corresponding to each system; S2: After a user logs in, the user information is encrypted according to the first encryption rule. The encrypted user information is then matched with the authorized user information stored in the authentication and authorization database to determine whether the user information is valid. If valid, a token is generated based on the encrypted user information. S3: Encrypt the token using the second encryption rule, generate a temporary key, and distribute the temporary key to each system for storage; S4: When a data operation request is received from the system, determine whether the user corresponding to the data operation request is an authorized user based on the key in the data operation request. If so, grant the system call permission corresponding to the data operation request and set the user's access limit to the system. Each time a data operation request from the user to the system is received, determine whether the number of requests exceeds the access limit. If so, stop receiving data operation requests from the user to the system. Otherwise, HAproxy load balancing is used to distribute data operation requests to the systems that perform the corresponding data operation requests; S5: When the system receives a data operation request, it parses the parameters of the data operation request to obtain the key in the data operation request. It then verifies the key with the temporary key stored in the system. When the verification is successful, it performs data operation from the corresponding database according to the data operation request. After the data operation is completed, it assembles the data obtained according to the returned data format and returns it to the system that initiated the request. The data operation request includes: key, operation option, operation statement optionValue, system name of the system to be operated on, databaseType of the database to be operated on, data to be operated on, and version number; The method for performing data operations from the corresponding database based on a data operation request is as follows: S501: Parse the data operation request and obtain the contents of the six fields: databaseType, option, optionValue, data, system, and version; S502: Obtain the corresponding version of the database based on the system, databaseType, and version fields, and obtain the database connection information from the database's corresponding configuration information; S503: Select the corresponding version of the database JAR file and the required connection method under the JAR file based on the system, databaseType and version fields; S504: Establish a connection to the database based on the database connection information, JAR package, and connection method; S505: Perform database operations based on the option and optionValue fields and the data field.

2. The data operation method based on multiple systems according to claim 1, characterized in that: The system uses the following databases: HBase, MySQL, Elasticsearch, Hive, ClickHouse, and Hadoop.

3. The data operation method based on multiple systems according to claim 1, characterized in that: The database configuration information includes: system name, IP address, port, username, login password, database connection, database version number, package path information, and returned data format.

4. The data operation method based on multiple systems according to claim 1, characterized in that: The access limit is the maximum number of times you can access the site per day.

5. A data operation terminal device based on multiple systems, characterized in that: It includes a processor, a memory, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the method as described in any one of claims 1 to 4.

6. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 4.

Images