Database migration method, device and equipment based on credential transformation and medium

Abstract

The invention relates to the technical field of databases, and discloses a database migration method and device based on credential transformation, equipment and a medium. The method comprises the following steps: traversing a source database to obtain all objects, distributing the objects to a plurality of target databases according to a business model, and decoupling dependent objects; based on an artificial intelligence large model of an integrated intelligent agent and retrieval enhancement generation module, comparing grammar differences, and identifying and rewriting objects incompatible with grammar; synchronizing the source database traffic to the target database, and comparing and feeding back the traffic to verify the object compatibility degree; building a target database cluster environment; and migrating the verified object, data and traffic to the target database cluster according to the business model. According to the method, the compatibility problem of database migration in credential reconstruction is solved, the compatibility, stability and efficiency of database migration in credential reconstruction are improved, the method is particularly suitable for database migration of key scenes such as a credit approval system and an electronic medical record system in the financial and medical fields, and it is guaranteed that the migration process is safe and reliable.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of database, and in particular to a database migration method, device and equipment based on Xingchuang reconstruction and a medium. BACKGROUND

[0002] In order to get rid of the dependence on foreign technology system, build a safe and reliable information technology base in key fields such as finance, medical treatment and government affairs, and then cope with the changes in internal and external situations and protect national information security, information technology application innovation and reconstruction have emerged as the times require.

[0003] As the core infrastructure of information system, the self-containment of database is a key link of Xingchuang reconstruction. However, the Xingchuang reconstruction of database faces multiple challenges: on the one hand, the database technology itself has high complexity, involving core modules such as storage architecture and transaction mechanism, and domestic systems have deep dependence on foreign leading technology standards; on the other hand, the product maturity and compatibility of domestic database manufacturers are uneven, which makes it difficult for users to match business scenarios when selecting products, and in the process of migrating from foreign technology stack to domestic technology stack, business risks are often caused by insufficient syntax adaptation, substandard performance, data consistency rupture and other problems, especially in fields such as financial transactions and medical data management, which involve the national economy and people's livelihood. Such risks may directly affect the continuity of the system and the security of the data.

[0004] Therefore, there is an urgent need to provide a database migration scheme suitable for Xingchuang scenarios to systematically solve the compatibility, stability and risk control problems in the migration of domestic technology stacks, and to ensure the safe transition and self-controllable operation of core systems in key fields such as finance and medical treatment. SUMMARY

[0005] The present application provides a database migration method, device, equipment and medium based on Xingchuang reconstruction to solve the technical problem that in the Xingchuang reconstruction of database, the lack of product adaptability leads to syntax, performance and data consistency problems during migration, causing business risks in key fields, and lacking a systematic solution.

[0006] In a first aspect, the present application provides a database migration method based on Xingchuang reconstruction, comprising:

[0007] traversing a source database to obtain all objects of the source database, distributing each object to a plurality of preset target databases according to a business model to which the source database belongs, and decoupling objects having a dependency relationship between the plurality of target databases;

[0008] An artificial intelligence large model based on an integrated intelligent agent module and a retrieval enhancement generation module, compares syntax rule differences between the target database and the source database, identifies objects incompatible with the syntax of the target database, and rewrites the syntax of the objects incompatible with the syntax of the target database to adapt to the target database.

[0009] Synchronously input traffic input to the source database into a plurality of target databases for running, and compare the traffic feedback of the source database with the traffic feedback of the target databases to verify the compatibility of each object in the target database to which it belongs.

[0010] According to a preset database cluster deployment architecture, a cluster environment of the target database is built.

[0011] According to the business model to which each object belongs in the source database, each verified object and corresponding data are migrated to the target database of the cluster environment, and traffic input to the source database is migrated to the target database of the cluster environment.

[0012] In a second aspect, a database migration device based on a signal creation reconstruction is provided, comprising:

[0013] A database object allocation module is configured to traverse a source database to obtain all objects of the source database, allocate each object to a plurality of preset target databases according to the business model to which each object belongs in the source database, and decouple objects having a dependency relationship between a plurality of target databases.

[0014] A database object compatibility detection module is configured to use an artificial intelligence large model based on an integrated intelligent agent module and a retrieval enhancement generation module to retrieve syntax differences between the target database and the source database, identify objects incompatible with the syntax of the target database, and rewrite the syntax of the objects incompatible with the syntax of the target database to adapt to the target database.

[0015] A database testing module is configured to synchronously input traffic input to the source database into a plurality of target databases for running, and compare the traffic feedback of the source database with the traffic feedback of the target databases to verify the compatibility of each object in the target database to which it belongs.

[0016] A database cluster deployment module is configured to build a cluster environment of the target database according to a preset database cluster deployment architecture.

[0017] A database migration module is configured to migrate each verified object and corresponding data to the target database of the cluster environment according to the business model to which each object belongs in the source database, and migrate traffic input to the source database to the target database of the cluster environment.

[0018] In a third aspect, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the database migration method based on the XG reform when executing the computer program.

[0019] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program, wherein the computer program is executable on a processor to implement the steps of the database migration method based on the XG reform.

[0020] The database migration method based on the XG reform, device, equipment and medium provided by the application can avoid the coordination disorder caused by object cross-database dependency and lay an orderly foundation for subsequent migration by traversing the source database to obtain all objects, distributing them to multiple target databases according to the business model and decoupling dependent objects. The artificial intelligence large model based on the integrated intelligent agent module and the search enhancement generation module compares the syntax rule differences, identifies and rewrites the objects with incompatible syntax, which can efficiently solve the problem of insufficient syntax adaptation and reduce the errors and inefficiency of manual processing. The source database traffic is synchronized to the target database and compared with the feedback traffic, which can accurately verify the compatibility of the objects in the target database and ensure data consistency and performance adaptability. The target database cluster environment is built according to the preset architecture, and the objects, data and traffic verified by the business model migration can build a stable high-availability cluster and reduce the migration risk.

[0021] The database migration method based on the XG reform can solve the problems of syntax, performance and data consistency caused by deep dependence on foreign technology and insufficient adaptability of domestic products in the database XG reform, as well as the technical problems of causing business risks in key fields such as financial business and medical business and lacking systematic solutions, which can improve the efficiency and security of database XG reform, ensure the stable operation of core systems in key fields such as finance and medicine, and help to realize the self-controlling of core technology.

[0022] These and other aspects of the application will become more apparent in the following description of embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the application, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the application. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0024] Figure 1An application environment schematic diagram of the database migration method based on the Xinchuang reconstruction provided by the embodiment of the present application is shown.

[0025] Figure 2 A flow schematic diagram of the database migration method based on the Xinchuang reconstruction provided by the embodiment of the present application is shown.

[0026] Figure 3 A flow schematic diagram of the step S100 is shown. Figure 2 A flow schematic diagram of the step S100 is shown.

[0027] Figure 4 A flow schematic diagram of the step S100 is shown. Figure 2 A flow schematic diagram of the step S100 is shown.

[0028] Figure 5 A flow schematic diagram of the step S300 is shown. Figure 2 A flow schematic diagram of the step S300 is shown.

[0029] Figure 6 A flow schematic diagram of the step S500 is shown. Figure 2 A flow schematic diagram of the step S500 is shown.

[0030] Figure 7 A flow schematic diagram of the step S500 is shown. Figure 2 A flow schematic diagram of the step S500 is shown.

[0031] Figure 8 A structure schematic diagram of the database migration device based on the Xinchuang reconstruction provided by the embodiment of the present application is shown.

[0032] Figure 9 A structure schematic diagram of the computer device in the embodiment of the present application is shown.

[0033] Figure 10 A structure schematic diagram of the computer device in the embodiment of the present application is shown. DETAILED DESCRIPTION

[0034] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all the other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0035] The database migration method based on the Xinchuang reconstruction provided by the embodiment of the present application can be applied in, for example, Figure 1application environment, the application environment includes an execution end 10 (such as a server deployed with a migration management platform), a source database cluster 20 constructed based on a foreign technology stack, and a plurality of domestic target database nodes 30 (to be migrated targets, in communication with the source database cluster through a network). The execution end 10 serves as a migration scheduling core and is connected with the source database cluster 20 and the target database nodes 30 through the network respectively, so as to overall plan the whole process of database migration, syntax rewriting, traffic synchronization, cluster building, and data migration. The source database cluster 20 stores business data to be migrated, and the target database nodes 30 complete relevant processing under the control of the execution end 10, and finally undertake the migrated business data and traffic.

[0036] Exemplarily, in the financial field, the database migration method based on the Xinyuan reconstruction provided by the embodiment of the application can be used to migrate the source database cluster data related to the credit approval system, account management system and the like constructed based on foreign databases to the domestic target database nodes, so as to guarantee the stable operation of core businesses. In the medical and health field, the database migration method based on the Xinyuan reconstruction provided by the embodiment of the application can be applied to the reconstruction of the electronic medical record system, information management system, inpatient charge management system and the like of a hospital, and the related information in the old foreign source database cluster of the hospital is migrated to the domestic target database nodes, so as to meet the requirements of medical data privacy protection and self-controllability and guarantee the stable development of medical businesses.

[0037] Figure 2 A flowchart of the database migration method based on the Xinyuan reconstruction provided by the embodiment of the application is shown, as shown in Figure 2 The database migration method based on the Xinyuan reconstruction provided by the embodiment of the application includes:

[0038] Step S100: The source database is traversed to obtain all objects of the source database, the objects are distributed to a plurality of preset target databases according to the business models to which the objects belong in the source database, and the objects having a dependency relationship between the plurality of target databases are decoupled.

[0039] Optionally, the object information of the source database includes definition statements, associated tables and call relationships of the objects, and forms a structured object list. When all objects are obtained by traversing the source database, a data type classification traversal method can be used, such as first traversing structured objects such as table structures and stored procedures, and then traversing auxiliary objects such as views and indexes, to ensure that no object is missed. The business model refers to an independent module divided according to business functions, such as account management, transaction settlement and risk assessment modules in the financial field. Each module contains specific business logic and data processing flow, and corresponds to a group of associated objects in the source database. When allocating objects, the access frequency and data size of the business model are combined to allocate objects with high-frequency access and strong data correlation to the same target database to reduce cross-database interaction overhead. Specifically, before decoupling processing, the dependency relationship between objects is combed to intuitively present the dependency path between objects, thereby providing clear guidance for decoupling operations. Thus, a large source database is split into multiple small target databases, a single large table with large data volume and high access frequency in the source database is split into multiple small tables with the same structure according to business dimensions (such as time range, regional division, business type, etc.), and is respectively stored in the corresponding target database, so as to improve query efficiency and data management flexibility.

[0040] Step S200: Based on the artificial intelligence model of the integrated intelligent agent module and the retrieval enhancement generation module, the syntax rule differences between the target database and the source database are compared, the objects incompatible with the syntax of the target database are identified, and the syntax of the objects incompatible with the syntax of the target database is rewritten to adapt to the target database.

[0041] Optionally, the integrated intelligent agent module (AGENT) is a process scheduling technology responsible for process control of syntax recognition and rewriting, including determining the priority of the object to be processed, calling relevant tools to perform syntax comparison tasks, etc., and the retrieval and enhancement generation module (RAG) is a knowledge retrieval-based enhancement generation technology used to quickly retrieve syntax difference information between the target database and the source database based on the constructed syntax knowledge system, and to provide accurate knowledge support for the artificial intelligence large model. The embodiments of the present application integrate AGENT and RAG technologies into the artificial intelligence large model, making the artificial intelligence large model more targeted and efficient when processing syntax adaptation. Specifically, the syntax manuals and official compatibility instructions of the target database and the source database can be collected in advance to construct a syntax difference knowledge base for the large model to learn; when incompatible objects are identified, syntax detection can be performed on objects such as stored procedures and triggers related to core business to ensure that key function adaptation is prioritized; after rewriting the syntax, the rewriting effect can be verified through unit testing to check for syntax errors or logical deviations. Through step S200, the embodiments of the present application identify and rewrite the syntax in the source database that depends on a specific PKG, converting it to a syntax form that adapts to the target database, eliminating the dependence on the PKG unique to foreign databases. PKG (stored procedure package) is a database object unique to foreign databases, which is a collection of related database objects such as stored procedures, functions, variables, and cursors, used to encapsulate multiple logically related database operations together to achieve modular management and reuse of code. Since PKG is an object unique to foreign databases, there may be no corresponding syntax support in domestic target databases, so the large model will identify the syntax in the source database that depends on a specific PKG (such as call statements for stored procedures in PKG). Subsequently, the RAG module retrieves the syntax difference knowledge base to obtain adaptation rules, and under the process control of AGENT, the syntax that depends on PKG is rewritten into a form supported by the target database (such as splitting into independent stored procedures or functions), thereby achieving the effect of removing PKG and eliminating the dependence on PKG unique to foreign databases, and completing the domestic adaptation of the syntax level.

[0042] Step S300: Synchronize the traffic input to the source database to the plurality of target databases, and compare the traffic feedback from the source database and the traffic feedback from the target databases to verify the compatibility of each object in the target database.

[0043] Optionally, by adopting the verification scheme of flow replication, the request flow of the source database is replicated and replayed in the target database, which can comprehensively verify the consistency of the function and performance of the target database, ensure the compatibility of the objects after splitting in the target database, and ensure the normal operation of the business logic after splitting the large database into small databases and the large table into small tables, and guarantee the consistency of data interaction. Specifically, when synchronizing the flow, the flow filtering rules can be set, and only the business requests related to the objects to be verified are synchronized to the target database, reducing irrelevant flow interference; when comparing the feedback flow, the difference results can be recorded, and the objects and specific positions where the differences occur are marked, providing a basis for subsequent optimization.

[0044] Step S400: According to the preset database cluster deployment architecture, the cluster environment of the target database is built.

[0045] Optionally, when building the cluster environment of the target database, the core parameters such as the upper limit of the connection number and the cache size can be pre-configured in combination with the performance requirements and business scenarios of the target database, thereby laying a foundation for cluster operation; the building process can adopt a phased node initialization strategy, complete the deployment of the master node first to establish the core architecture, and then gradually expand the slave nodes to build a complete cluster topology, while the cluster health check tool is used to monitor the node communication state and data synchronization in real time, ensuring that the cluster architecture is consistent with the preset deployment requirements.

[0046] It should be noted that the "target database" involved in steps S100-S300 is not the actual running production environment cluster, but a phased carrier prepared for migration: the target database in step S100 is a logical level division, which is used to allocate objects according to business models and implement large database splitting into small databases and large table splitting into small tables, and only carries the objects and data after preliminary splitting; the target database in step S200 is a syntax verification environment, which relies on the AI large model integrated with AGENT and RAG technologies to complete the identification and modification of incompatible SQL; the target database in step S300 is a flow verification carrier, which verifies the compatibility of the objects in actual operation through flow replication and replay, but still does not form a cluster architecture. The target database cluster environment built in step S400 is a production-level running environment consistent with the preset architecture, which provides a stable carrying platform for the objects, data and flow verified in the previous stage through node function division, has high availability, scalability and cross-node collaboration ability, and is the final running carrier for database localization migration.

[0047] Step S500: According to the business model to which each object belongs in the source database, each object and corresponding data after verification are migrated to the target database of the cluster environment, and the flow input to the source database is migrated to the target database of the cluster environment.

[0048] Optionally, when migrating the objects of the source database, for the objects that have been split according to the business model in step S100, the objects are synchronized in batches to the target database of the cluster environment built in step S400 according to the association relationship of the business model to which the objects belong. Since the large source database is split into multiple small target databases by allocating the objects of the source database to multiple target databases in step S100, the objects of the source database are split according to the business boundary and then migrated to multiple corresponding target databases. During the migration process, the corresponding relationship between the business model and the target database needs to be strictly followed to avoid migration exceptions caused by cross-database data dependencies. When migrating the traffic, the traffic verification result in step S300 is referred to, and the traffic input to the source database is migrated to the target database of the cluster environment built in step S400. During the process, the real-time data synchronization states of the source database and the target cluster are compared to ensure data consistency and guarantee seamless switching of the business.

[0049] In some embodiments, the database migration method based on the Xinchuang reconstruction provided by the embodiments of the present application, the source database is an Oracle database, and the target database is a TiDB database. The embodiments of the present application split the Oracle database into three TiDB databases according to the business model by reasonable planning, adopt a vertical split database strategy, reduce the module coupling degree, and migrate in batches according to the database to reduce the migration risk. In the compatibility reconstruction, an artificial intelligence large model is used to complete the identification and reconstruction of incompatible SQL syntax, thereby saving the labor cost. A traffic replication verification scheme is adopted to replicate the request traffic of the Oracle database, replay in the TiDB database, verify the consistency of the function and performance of the TiDB database, and further save the test cost in the migration process.

[0050] The database migration method based on the Xinchuang reconstruction provided by the embodiments of the present application can avoid the coordination disorder caused by cross-database dependencies by traversing the source database to obtain all objects, allocating the objects to multiple target databases according to the business model, and decoupling dependent objects, thereby laying an orderly foundation for subsequent migration. The artificial intelligence large model based on the integrated intelligent agent module and the retrieval enhancement generation module compares the syntax rule differences, identifies and rewrites the objects with incompatible syntax, can efficiently solve the problem of insufficient syntax adaptation, and reduce the error and inefficiency of manual processing. The source database traffic is synchronized to the target database and the feedback traffic is compared, which can accurately verify the compatibility of the objects in the target database, guarantee data consistency and performance adaptability. The target database cluster environment is built according to the preset architecture, and the objects, data and traffic verified according to the business model are combined, which can construct a stable high-availability cluster and reduce the migration risk.

[0051] The database migration method based on the Xinyuan reconstruction can solve the problems of syntax, performance and data consistency caused by deep dependence on foreign technology and insufficient adaptability of domestic products in the database Xinyuan reconstruction, and the technical problems of causing risks in key fields such as financial business and medical business, and lacking of systematic solutions, has the effects of improving the efficiency and safety of the database Xinyuan reconstruction, ensuring the stable operation of the core system in the key fields such as finance and medical treatment, and assisting in realizing the self-controlling of core technology.

[0052] In some embodiments, Figure 3 A flowchart of step S100 in some embodiments is shown in FIG. 1, and as shown in FIG. 1, the step of decoupling the objects having the dependency relationship between the multiple target databases in step S100 of the embodiments of the present application comprises: Figure 2 Figure 3 The step S110: identifying the objects having the dependency relationship between the target databases, and determining the dependent objects and the dependent objects. Optionally, the dependency paths across the target databases (such as the storage process of the A target database calling the table field of the B target database) can be sorted out by analyzing the association table, the calling relationship and the definition statement of each object of the source database, and the dependency hierarchy can be presented by a visual chart to clearly show the corresponding relationship between the dependent objects (such as the table field of the B database) and the dependent objects (such as the storage process of the A database), so as to ensure that there is no omission in the dependency relationship identification.

[0053] The step S120: redundantly storing the fields of the dependent objects from the target database to which the dependent objects belong to the target database to which the dependent objects belong, and establishing a synchronization mechanism to synchronize the fields of the two target databases. Optionally, the core fields (such as user ID, transaction number, etc.) of the dependent objects with high frequency access can be filtered during the redundant storage to avoid data redundancy expansion caused by full redundancy, and the field redundancy range can be determined according to the business query scene, for example, only the key fields necessary for cross-database association query are retained, which can meet the access demand of the dependent objects for data and can also reduce the redundant data to the greatest extent. Through this field redundancy method, the direct dependency relationship between the target databases can be effectively eliminated, the decoupling of the business logic and the data storage is realized, and the flexibility and maintainability of the system are improved.

[0054] In some embodiments,

[0055] Another flowchart of step S100 in some embodiments is shown in FIG. 2, and as shown in FIG. 2, the step of decoupling the objects having the dependency relationship between the multiple target databases in step S100 of the embodiments of the present application further comprises: Figure 4 Figure 2 Figure 4

[0056] ​​​​Step S130: synchronously store the plurality of depended objects and the plurality of dependent objects to an elastic search data heterogeneous model, so that the target database to which the dependent object belongs acquires the depended object in the elastic search data heterogeneous model. Optionally, the elastic search data heterogeneous model, i.e., the ES data heterogeneous model, is a cross-database data integration model constructed based on an elastic search (Elasticsearch). The depended objects and the dependent object data scattered in a plurality of target databases are extracted, converted, and then uniformly stored, so as to form a heterogeneous data layer independent of the original database architecture, and support the aggregation query of cross-source data. In addition to decoupling through field redundancy, the ES data heterogeneous model is also set to decouple. The model does not need to store the field redundancy between the target databases, but constructs a unified data access portal, so that the target database to which the dependent object belongs can directly acquire the required depended object information from the heterogeneous model, avoiding the direct interaction between the target databases. When storing, the data can be structurally reorganized according to the business query dimension, so as to optimize the cross-database query efficiency, and at the same time, ensure that the data synchronized to the ES data heterogeneous model is consistent with the source database, which meets the needs of the complex correlation query scene, and together with the field redundancy mode, forms a multi-dimensional decoupling scheme, improving the flexibility and adaptability of system decoupling.

[0057] In some embodiments, Figure 5 A flowchart of step S300 in the method is shown in FIG. 3. Figure 2 The step S300 of the embodiment of the present application includes the following steps: Figure 5 The step S300 of the embodiment of the present application includes the following steps:

[0058] Step S310: acquire the traffic input to the source database and the traffic feedback by the source database, and pre-process the traffic input to the source database and the traffic feedback by the source database to generate pre-processed data.

[0059] Step S320: encapsulate the pre-processed data into a pre-processing request message, and send the pre-processing request message to a message queue.

[0060] Step S330: when the pre-processing request message triggers a preset traffic replication service, input the traffic input to the source database contained in the pre-processed data to a plurality of target databases to run.

[0061] Step S340: acquire the traffic feedback by the target database and compare it with the traffic feedback by the source database, to verify the compatibility of each object in the target database to which the object belongs.

[0062] When the traffic feedback of the source database exceeds the preset first threshold traffic, the traffic feedback of the source database is replaced by a fixed-length MD5 algorithm value; and when the traffic feedback of the target database exceeds the preset second threshold traffic, the traffic feedback of the target database is replaced by a fixed-length MD5 algorithm value.

[0063] Optionally, in step S310, the old application generates the traffic input to the source database, which can be accurately captured by setting an interceptor, and the traffic feedback of the source database to the old application is also intercepted, and then the captured request traffic is preprocessed. Specifically, the purpose of preprocessing is to make the captured traffic be sent to the target database, and the present application embodiment does not limit the preprocessing process. For example, as an embodiment, the preprocessing process includes combing and integrating the request header information (such as request time, data format identifier, etc.) and core parameters, clearly recording the request mode, accessed resource path (uri) and specific input parameters, and ensuring that the complete characteristics of the request are accurately retained; for the traffic feedback of the source database, if the data volume is too large (exceeding the preset first threshold traffic), the traffic is encrypted by MD5 algorithm (Message-Digest Algorithm 5, Message Digest Algorithm 5) to generate a fixed-length hash value to replace it. The standardized data obtained through the above preprocessing is encapsulated as a preprocessed request message conforming to the message queue format requirement in step S320, the message contains complete request characteristic information and processed feedback data, and then is sent to the message queue for temporary storage, waiting for subsequent processing; in step S330, when the preprocessed request message in the message queue triggers the preset traffic replication service, the service extracts the preprocessed input traffic from the message, and synchronously inputs it into multiple target databases according to the consistent request mode, resource path and input parameters of the source database, simulates the processing scene of the source database, and runs in the multiple target databases; in step S340, the traffic feedback of the target database after running is obtained, if the data volume exceeds the preset second threshold traffic, a hash value is generated by MD5 algorithm, and then the feedback result of the target database is compared with the corresponding result of the source database, the consistency of the request response is verified by checking, the compatibility of each object in the target database is verified, and the function stability of the migrated object is ensured.

[0064] In some embodiments, the database migration method based on the Xinda reconstruction provided by the present application embodiment includes the following steps:

[0065] An application access layer for connecting the database cluster of the application end and a plurality of database clusters deployed in at least two cross-regional machine rooms.

[0066] Each database cluster includes a cluster control component, database service nodes, and distributed storage nodes.

[0067] Data is synchronized between distributed storage nodes through a consistency protocol, and cross-regional data synchronization is achieved between database clusters in different data centers through a data synchronization component.

[0068] Optionally, the application is a software program that directly provides services to users, such as various mobile applications and web applications. Users initiate data access and business processing requests by operating the application, such as checking account balances and submitting orders, which is the source of data traffic. The request traffic generated by the application is further processed through the application access layer before entering the database cluster. The database cluster is the physical deployment form of the target database. The cluster control component is the "scheduling center" of the target database, used to manage the topology and data distribution; the database service node is the "business interface" of the target database, used to receive and execute requests from the application; the distributed storage node is the "data warehouse" of the target database, used to store all data. The preset database cluster deployment architecture of this embodiment configures multiple data centers and multiple database clusters to achieve a multi-replica deployment mode. Each distributed storage node relies on a consistency protocol to achieve data synchronization, ensuring the consistency and integrity of data within a single cluster; database clusters in different regions complete data interaction through a data synchronization component. Combined with a multi-replica storage strategy, when a single data center or single cluster fails, it can quickly switch to replica data in other data centers or clusters, greatly reducing the risk of data loss and ensuring business continuity.

[0069] In some embodiments, Figure 6 It shows Figure 2 A flowchart of step S500 is shown below. Figure 6 As shown, in step S500 of this embodiment of the invention, the step of migrating traffic input to the source database to the target database in the cluster environment includes:

[0070] Step S510: Gradually migrate the read request traffic input to the source database to the target database, and synchronously transmit the data generated by the write request traffic processed by the source database to the target database.

[0071] Step S520: After all read request traffic input to the source database has been migrated to the target database, write request traffic input to the source database is gradually migrated to the target database, and the data generated by the target database in processing read request traffic and write request traffic is synchronously transmitted to the source database.

[0072] Step S530: After all write request traffic input to the source database has been migrated to the target database, the synchronous transmission of data generated by the target database's processing of read request traffic and write request traffic to the source database is terminated.

[0073] Optionally, the step-by-step migration of the read request flow in step S510 can be promoted in a service module priority layer, and non-core query services (such as log query) are migrated first. The accuracy is verified by comparing the return results of the target database and the source database in real time, and the incremental synchronization mechanism is used when synchronously transmitting the write request flow of the source database to reduce resource consumption; the write request flow migration in step S520 can be switched step by step according to the interface call frequency from small to large, and the bidirectional verification mechanism is enabled when synchronizing the target database data to the source database to ensure that the old and new database data are consistent, and at the same time, the flow back switching channel is reserved, and if the target database appears abnormal, the request can be quickly switched back to the source database; after step S530 terminates the synchronization, the full data of the target database is verified, the integrity and accuracy of the migrated data are confirmed by comparing the historical synchronization records and the current data state, and finally the smooth offline of the source database is completed, and the safety and business continuity of the entire flow migration process are ensured.

[0074] In some embodiments, Figure 7 It is shown Figure 2 Another flowchart of step S500 in the embodiment is shown, as Figure 7 As shown in step S500 of the embodiment of the present application, the step of migrating the flow input to the source database to the target database in the cluster environment includes:

[0075] Step S540: synchronously transmitting the write request flow input to the source database to the source database and the target database, and transmitting the read request flow input to the source database to the source database, so as to ensure that the same write request is sent to the source database and the target database for execution at the same time, the return states of the two databases are compared in real time, and the data writing behavior is ensured to be consistent.

[0076] Step S550: when the processing result of the write request flow of the source database is consistent with the processing result of the write request flow of the target database, the read request flow input to the source database is gradually migrated to the target database. Optionally, the read request migration can be layered according to the business scenarios (such as ordinary query, key transaction query), and the low-risk scenario is started to be switched step by step. After each batch is switched, the return results of the target database and the source database are sampled and verified, so as to confirm the consistency and expand the migration range.

[0077] Step S560: when the read request flow input to the source database is all migrated to the target database, the transmission of the write request flow to the source database is terminated. Optionally, before the transmission of the write request flow is terminated, the write request processing performance of the target database can be continuously monitored at any time, and if an exception occurs, the write request can be quickly rolled back to be executed only by the source database. After the read request is fully migrated and the target database is stable, the source database write request receiving is stopped, the double-write switching is completed, and the smooth transition of migration is ensured.

[0078] The database migration method based on the Xinchuang reconstruction provided by the embodiment of the application provides two traffic migration schemes for the step of migrating the traffic input to the source database to the target database of the cluster environment in step S500. Specifically, step S510-step S530 provides a read-write separation switching scheme by a step-by-step strategy of switching read-only traffic first, verifying the consistency of functions and performance, and then switching write traffic, in combination with the interface dimension data source switching implemented by the dynamic database reconstruction, so as to accurately control the database switching rhythm, disperse the risk in different stages, ensure that each migration is promoted on the basis of verification, and greatly reduce the risk of business interruption caused by one-time full switching. Step S540-step S560 provides a double-write switching scheme as a backup scheme, which maintains the synchronization of the data in the source database and the target database by synchronously transmitting the write request to the source database and the target database, verifies the consistency, and then migrates the read request, so as to maintain the synchronization of the data in the source database and the target database during the migration process, provide a stable transition environment for traffic switching, and guarantee the continuity and accuracy of the business data. The two schemes provided by the embodiment of the application are complementary to each other, and can be flexibly selected according to the actual business scene (such as data sensitivity and business complexity), so as to ensure the stability, safety and efficiency of the traffic migration to the target database as a whole, and provide a solid traffic switching guarantee for the database migration based on the Xinchuang reconstruction.

[0079] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the application.

[0080] In some embodiments, the embodiment of the application also provides a database migration device based on Xinchuang reconstruction, Figure 8 The schematic diagram of the database migration device based on Xinchuang reconstruction provided by the embodiment of the application is shown in Figure 8 As shown in the figure, the device comprises:

[0081] The database object allocation module 100 is configured to traverse the source database to obtain all objects of the source database, allocate each object to a plurality of preset target databases according to the business model to which the source database belongs, and decouple the objects having a dependency relationship between the plurality of target databases.

[0082] The database object compatibility detection module 200 is configured to retrieve the syntax difference between the target database and the source database based on the artificial intelligence large model of the integrated intelligent agent module and the retrieval enhancement generation module, identify the object incompatible with the syntax of the target database, and rewrite the syntax of the object incompatible with the syntax of the target database to adapt to the target database.

[0083] The database test module 300 is configured to synchronize the traffic input to the source database to the plurality of target databases, and compare the feedback traffic of the source database with the feedback traffic of the target databases to verify the compatibility of each object in the target database.

[0084] The database cluster deployment module 400 is configured to build a cluster environment of the target database according to a preset database cluster deployment architecture.

[0085] The database migration module 500 is configured to migrate each object and corresponding data verified to the target database in the cluster environment according to the business model of the source database, and migrate the traffic input to the source database to the target database in the cluster environment.

[0086] In some embodiments, when the database object allocation module 100 is used to decouple the objects having a dependency relationship between the plurality of target databases, the database object allocation module 100 specifically comprises the following steps.

[0087] The objects having a dependency relationship between the target databases are identified, and the dependent objects and the dependent objects are determined.

[0088] The fields of the dependent objects are redundantly stored from the target database to which the dependent objects belong to the target database to which the dependent objects belong, and a synchronization mechanism is established to synchronize the fields of the two target databases.

[0089] In some embodiments, when the database object allocation module 100 is used to decouple the objects having a dependency relationship between the plurality of target databases, the database object allocation module 100 specifically further comprises the following steps.

[0090] The plurality of dependent objects and the plurality of dependent objects are synchronously stored to the elastic search data heterogeneous model, so that the target database to which the dependent objects belong obtains the dependent objects in the ES data heterogeneous model.

[0091] In some embodiments, the database test module 300 is specifically configured to:

[0092] The traffic input to the source database and the feedback traffic of the source database are obtained, and the traffic input to the source database and the feedback traffic of the source database are preprocessed to generate preprocessed data.

[0093] The preprocessed data is encapsulated into a preprocessed request message, and the preprocessed request message is sent to a message queue.

[0094] When the preprocessed request message triggers a preset traffic replication service, the traffic input to the source database contained in the preprocessed data is input to the plurality of target databases.

[0095] The traffic fed back by the target database is acquired and compared with the traffic fed back by the source database to verify the compatibility of each object in the target database.

[0096] When the traffic fed back by the source database exceeds a preset first threshold traffic, the traffic fed back by the source database is replaced by a fixed-length MD5 algorithm value, and when the traffic fed back by the target database exceeds a preset second threshold traffic, the traffic fed back by the target database is replaced by a fixed-length MD5 algorithm value.

[0097] In some embodiments, the database cluster deployment module 400, the preset database cluster deployment architecture specifically includes:

[0098] An application access layer for connecting the database cluster of the application end and a plurality of database clusters deployed in at least two cross-regional machine rooms.

[0099] Each database cluster includes a cluster control component, a database service node, and a distributed storage node.

[0100] The distributed storage nodes are synchronized through a consistency protocol, and the database clusters in each machine room are synchronized through a data synchronization component.

[0101] In some embodiments, the database migration module 500 is used to migrate the traffic input to the source database to the target database in the cluster environment, and specifically includes:

[0102] The read request traffic input to the source database is gradually migrated to the target database, and the data generated by the source database processing the write request traffic is synchronously transmitted to the target database.

[0103] When the read request traffic input to the source database is all migrated to the target database, the write request traffic input to the source database is gradually migrated to the target database, and the data generated by the target database processing the read request traffic and the write request traffic is synchronously transmitted to the source database.

[0104] When the write request traffic input to the source database is all migrated to the target database, the data generated by the target database processing the read request traffic and the write request traffic is terminated to be synchronously transmitted to the source database.

[0105] In some embodiments, the database migration module 500 is used to migrate the traffic input to the source database to the target database in the cluster environment, and specifically further includes:

[0106] The write request traffic input to the source database is synchronously transmitted to the source database and the target database, and the read request traffic input to the source database is transmitted to the source database.

[0107] When the processing result of the write request traffic of the source database is consistent with the processing result of the write request traffic of the target database, the read request traffic input to the source database is gradually migrated to the target database.

[0108] When all the read request traffic input to the source database is migrated to the target database, the transmission of the write request traffic to the source database is terminated.

[0109] The database migration device based on the Xinchuang reconstruction provided by the embodiment can avoid the coordination disorder caused by object cross-database dependency, lay an orderly foundation for subsequent migration, and can efficiently solve the problem of insufficient syntax adaptation, reduce the error and inefficiency of manual processing, and can accurately verify the compatibility of objects in the target database, guarantee data consistency and performance adaptability.

[0110] The database migration device based on the Xinchuang reconstruction can solve the problems of syntax, performance, and data consistency caused by deep dependence on foreign technology and insufficient domestic product adaptability in the database Xinchuang reconstruction, and can solve the technical problems of causing business risks in key fields such as financial business and medical business and lacking systematic solutions, has the effects of improving the efficiency and security of the database Xinchuang reconstruction, guaranteeing the stable operation of core systems in key fields such as finance and medicine, and helping to realize the self-controllability of core technology.

[0111] The specific limitations of the database migration device based on the Xinchuang reconstruction can be referred to the limitations of the database migration method based on the Xinchuang reconstruction in the above, and will not be repeated here. Each module in the above database migration device based on the Xinchuang reconstruction can be realized by software, hardware, and combinations thereof. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to each module.

[0112] In one embodiment, the computer device provided by the embodiment can be a server, and its internal structure diagram can be as shown in Figure 9As shown in the figure. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes non-volatile and / or volatile storage media, internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with the external client through the network connection. The computer program is executed by the processor to realize the functions or steps of the server side of the database migration method based on the XG modified database.

[0113] In one embodiment, the embodiment of the present application provides a computer device which can be a client, and the internal structure diagram thereof can be as shown in the figure. Figure 10 As shown in the figure. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes non-volatile storage media, internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with the external server through the network connection. The computer program is executed by the processor to realize the functions or steps of the client side of the database migration method based on the XG modified database.

[0114] In one embodiment, the embodiment of the present application provides a computer device including a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to realize the following steps:

[0115] Traverse the source database to obtain all objects of the source database, distribute each object to a plurality of preset target databases according to the business model to which the source database belongs, and decouple the objects between the plurality of target databases that have a dependency relationship;

[0116] Based on the artificial intelligence large model of the integrated intelligent agent module and the retrieval enhancement generation module, the syntax rule differences between the target database and the source database are compared, incompatible target database syntax objects are identified, and the syntax of the incompatible target database syntax objects is rewritten to adapt to the target database;

[0117] Synchronize the traffic input to the source database to the plurality of target databases for running, and compare the traffic feedback of the source database and the traffic feedback of the target database to verify the compatibility of each object in the target database to which it belongs;

[0118] According to the preset database cluster deployment architecture, a cluster environment of the target database is built.

[0119] According to the business model to which each object belongs in the source database, the verified objects and corresponding data are migrated to the target database in the cluster environment, and the traffic input to the source database is migrated to the target database in the cluster environment.

[0120] In one embodiment, a computer readable storage medium is provided, and the computer program is stored on the computer readable storage medium. When the computer program is executed by a processor, the following steps are implemented:

[0121] The source database is traversed to obtain all objects of the source database, each object is allocated to a plurality of preset target databases according to the business model to which each object belongs in the source database, and objects having a dependency relationship between the plurality of target databases are decoupled;

[0122] Based on the artificial intelligence large model of the integrated intelligent agent module and the retrieval enhancement generation module, the syntax rule differences between the target database and the source database are compared, incompatible objects of the target database syntax are identified, and the syntax of the incompatible objects of the target database is rewritten to adapt to the target database;

[0123] The traffic input to the source database is synchronized to run in the plurality of target databases, and the traffic feedback of the source database and the traffic feedback of the target database are compared to verify the compatibility of each object in the target database to which the object belongs.

[0124] According to the preset database cluster deployment architecture, a cluster environment of the target database is built.

[0125] According to the business model to which each object belongs in the source database, the verified objects and corresponding data are migrated to the target database in the cluster environment, and the traffic input to the source database is migrated to the target database in the cluster environment.

[0126] It should be noted that the functions or steps that the computer readable storage medium or the computer device can implement described above can be referred to the related description of the server side and the client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0127] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, storage, database or other medium used in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0128] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.

[0129] The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, but not limit it. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features. The modification or replacement does not make the essence of the corresponding technical solution deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A database migration method based on information technology innovation transformation, characterized in that, include: Traverse the source database to obtain all objects in the source database, allocate each object to multiple preset target databases according to the business model to which each object belongs in the source database, and decouple objects that have dependencies between the multiple target databases; Based on an AI big model integrating an intelligent agent module and a retrieval enhancement generation module, the differences in syntax rules between the target database and the source database are compared to identify objects that are incompatible with the syntax of the target database, and the syntax of the objects that are incompatible with the syntax of the target database is rewritten to adapt to the target database. The traffic input to the source database is synchronously input to multiple target databases for operation, and the traffic returned by the source database and the traffic returned by the target database are compared to verify the compatibility of each object in its respective target database. The cluster environment of the target database is built according to the preset database cluster deployment architecture; Based on the business model to which each of the objects belongs in the source database, the verified objects and their corresponding data are migrated to the target database in the cluster environment, and the traffic input to the source database is migrated to the target database in the cluster environment.

2. The database migration method based on information technology innovation as described in claim 1, characterized in that, The step of decoupling objects that have dependencies among multiple target databases includes: Identify objects that have dependencies on each of the target databases, and determine the dependent objects and the objects that depend on them; The fields of the dependent object are redundantly stored from its target database to the target database to which the dependent object belongs, and a synchronization mechanism is established to ensure that the fields of the two target databases are updated synchronously.

3. The database migration method based on information technology innovation as described in claim 2, characterized in that, The step of decoupling objects that have dependencies among multiple target databases further includes: Multiple dependent objects and multiple dependent objects are synchronously stored in the elastic search data heterogeneous model so that the target database to which the dependent object belongs can obtain the dependent object in the elastic search data heterogeneous model.

4. The database migration method based on information technology innovation as described in claim 1, characterized in that, The step of synchronously inputting traffic to the source database to multiple target databases for operation, and comparing the traffic returned by the source database and the traffic returned by the target databases to verify the compatibility of each object in its respective target database includes: The system acquires the traffic input to the source database and the traffic returned by the source database, and preprocesses the traffic input to the source database and the traffic returned by the source database to generate preprocessed data. The preprocessed data is encapsulated into a preprocessing request message, and the preprocessing request message is sent to a message queue; When the preprocessing request message triggers the preset traffic replication service, the traffic contained in the preprocessed data that was input to the source database is input to multiple target databases for processing. The traffic returned by the target database is obtained and compared with the traffic returned by the source database to verify the compatibility of each object with its respective target database. Specifically, when the traffic reported by the source database exceeds a preset first threshold traffic, the traffic reported by the source database is replaced with a fixed-length MD5 algorithm value; when the traffic reported by the target database exceeds a preset second threshold traffic, the traffic reported by the target database is replaced with a fixed-length MD5 algorithm value.

5. The database migration method based on information technology innovation transformation as described in claim 1, characterized in that, The preset database cluster deployment architecture includes: The application access layer for connecting the database clusters to the application end and multiple database clusters deployed in at least two cross-regional data centers; Each of the database clusters includes a cluster control component, database service nodes, and distributed storage nodes; The distributed storage nodes synchronize data through a consistency protocol, and the database clusters in the data centers achieve cross-regional data synchronization through a data synchronization component.

6. The database migration method based on information technology innovation transformation as described in claim 1, characterized in that, The step of migrating traffic input to the source database to the target database in the cluster environment includes: The read request traffic input to the source database is gradually migrated to the target database, and the data generated by the write request traffic processed by the source database is synchronously transmitted to the target database; After all read request traffic input to the source database is migrated to the target database, write request traffic input to the source database is gradually migrated to the target database, and the data generated by the target database in processing read request traffic and write request traffic is synchronously transmitted to the source database. Once all write request traffic input to the source database has been migrated to the target database, the synchronous transmission of data generated by the target database's read and write request traffic to the source database is terminated.

7. The database migration method based on information technology innovation as described in claim 1, characterized in that, The step of migrating traffic input to the source database to the target database in the cluster environment includes: Write request traffic input to the source database is synchronously transmitted to both the source database and the target database, and read request traffic input to the source database is transmitted to the source database. When the processing result of the write request traffic by the source database is consistent with the processing result of the write request traffic by the target database, the read request traffic input to the source database will be gradually migrated to the target database. Once all read request traffic input to the source database has been migrated to the target database, the transmission of write request traffic to the source database is terminated.

8. A database migration device based on information technology innovation transformation, characterized in that, include: The database object allocation module is used to traverse the source database to obtain all objects in the source database, allocate each object to multiple preset target databases according to the business model to which each object belongs in the source database, and decouple objects that have dependencies between the multiple target databases. The database object compatibility detection module is used to retrieve the grammatical differences between the target database and the source database based on the large artificial intelligence model of the integrated intelligent agent module and the retrieval enhancement generation module, identify objects that are incompatible with the syntax of the target database, and rewrite the syntax of the objects that are incompatible with the syntax of the target database to adapt to the target database. The database testing module is used to synchronously input the traffic input to the source database to multiple target databases for operation, and compare the traffic returned by the source database and the traffic returned by the target database to verify the compatibility of each object in its respective target database. The database cluster deployment module is used to build the cluster environment of the target database according to the preset database cluster deployment architecture; The database migration module is used to migrate the verified objects and their corresponding data to the target database in the cluster environment according to the business model to which each object belongs in the source database, and to migrate the traffic input to the source database to the target database in the cluster environment.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the database migration method based on information technology innovation as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the database migration method based on information technology innovation as described in any one of claims 1 to 7.

Images