A migration method, device and medium of an oracle database large object

Abstract

The embodiment of the specification discloses a migration method and device of an oracle database large object and a medium, relates to the technical field of data migration, and is used for solving the problem that the existing oracle large object is difficult to migrate to PostgreSQL. The method comprises the following steps: detecting a creation SQL statement on a server side to determine whether a current table column has a large object column based on the creation SQL statement; if yes, creating a corresponding large object secondary table implicitly based on a large object secondary table structure, generating locator information of large object data in a large object primary table based on the large object secondary table; returning the locator information to a client side to receive large object data request information uploaded by the client side; determining current locator information of to-be-queried large object data based on the large object data request information, positioning a data position of the large object secondary table based on the current locator information, and returning large object data corresponding to the data position to the client side.

Description

Technical Field

[0001] This manual relates to the field of large object data, and in particular to a method, device, and medium for migrating large objects in an Oracle database. Background Technology

[0002] Databases are widely used in enterprise IT infrastructure development. Oracle databases, with their powerful features, high reliability, and broad application scenarios, are widely adopted by many large enterprises. However, as business grows and changes, situations may arise where database adjustments and optimizations are necessary, requiring the migration of large object data from Oracle databases to PostgreSQL databases.

[0003] In existing migration techniques, because all large objects in the Oracle database are stored in a single system table, this table becomes a performance bottleneck when the number of large objects is excessive. Furthermore, migrating large objects from Oracle to PostgreSQL involves creating large object table columns, inserting and updating large objects, etc. However, because PostgreSQL lacks a standard SQL large object data type, Oracle table creation statements cannot be directly used in PostgreSQL, nor can data be inserted into it. This makes a smooth migration of large objects from Oracle to PostgreSQL impossible. Summary of the Invention

[0004] To address the aforementioned technical problems, this specification provides one or more embodiments of a method, device, and medium for migrating large objects in an Oracle database.

[0005] One or more embodiments of this specification employ the following technical solutions:

[0006] This specification provides one or more embodiments of a method for migrating large objects in an Oracle database, the method including:

[0007] The server-side creation SQL statement is detected to determine whether the currently created table column has a large object column based on the creation SQL statement;

[0008] If so, then based on the large object second-level table structure, a corresponding large object second-level table is implicitly created, and based on the large object second-level table, the locator information of the large object data is generated and stored in the large object first-level table;

[0009] The locator information is returned to the client to receive the large object data request information uploaded by the client;

[0010] Based on the large object data request information, the current locator information of the large object data to be queried is determined, and the data location of the second-level table of the large object is located based on the current locator information, so as to return the large object data corresponding to the data location to the client.

[0011] Optionally, in one or more embodiments of this specification, detecting the server-side creation SQL statement to determine whether the currently created table column has a large object column based on the creation SQL statement specifically includes:

[0012] The SQL statement to be created is parsed to generate an abstract syntax tree for the SQL statement to be created.

[0013] The abstract syntax tree is traversed to check whether the data type of each column is a preset large object type;

[0014] If not, then create a regular table based on the SQL statement described above;

[0015] If so, then determine that the currently created table column has a large object column.

[0016] Optionally, in one or more embodiments of this specification, a corresponding large object second-level table is implicitly created based on the large object second-level table structure to generate the locator information of the large object, specifically including:

[0017] Obtain the unique identifier of the first-level table of the large object, and generate the table name of the second-level table of the large object based on the unique identifier of the first-level table of the large object;

[0018] Based on the large object second-level table structure, the fixed fields of the large object second-level table are determined to create a large object second-level table containing the fixed fields, and the dependency relationship between the large object first-level table and the large object second-level table is established based on the table name of the large object second-level table.

[0019] Based on the second-level table of the large object, a unique identifier corresponding to each large object data is determined, and based on the dependency relationship, each large object data is bound to the first-level table of the large object;

[0020] Based on the current computing resource constraints, the corresponding page size is determined, and a corresponding sharding identifier field is added to the large object secondary table according to the page size, so as to shard the large object data and realize distributed storage.

[0021] The unique identifier corresponding to each large object, the fragment identifier field corresponding to the large object data, and the data fragment position are encapsulated based on a preset format to obtain the locator information of the large object.

[0022] Optionally, in one or more embodiments of this specification, a corresponding page size is determined based on current computing resource constraints, and a corresponding sharding identifier field is added to the large object secondary table according to the page size, so as to shard the large object data and realize distributed storage, specifically including:

[0023] Based on current computing resource constraints, determine the migration scenarios for large Oracle objects; these migration scenarios include: high memory, low data transfer scenarios and low memory, high concurrency scenarios.

[0024] Determine the page size that matches the current computing resource limit data within the page range corresponding to the migration scenario;

[0025] Based on the page size and page number, a sharding rule is determined, and based on the sharding rule, the sharding position of the large object secondary table is determined. Then, a corresponding sharding identifier field is added to the large object secondary table according to the sharding position.

[0026] Large object data located in the same shard identifier field within the secondary table of the large object are stored in a distributed storage physical table, and the mapping relationship between the data shard location and the large object data is recorded.

[0027] Optionally, in one or more embodiments of this specification, after implicitly creating a corresponding large object second-level table based on the large object second-level table structure, and storing the locator information of the large object data in the large object first-level table based on the large object second-level table, the method further includes:

[0028] Receive updated data from the first-level table of the large object, and determine whether to trigger the large object data insertion process based on the updated data;

[0029] If the large object data insertion process is triggered, the locator information is inserted into the large object first-level table, and the specified large object column corresponding to the updated data is determined.

[0030] Based on the locator information and the specified large object column, the updated data is directly added to the large object sub-table corresponding to the specified large object column;

[0031] After determining whether to trigger the large object data insertion process based on the updated data, the method further includes:

[0032] If the large object data insertion process is triggered, a preset interface is called to determine the large object secondary table corresponding to the updated data based on the locator information and update it.

[0033] Optionally, in one or more embodiments of this specification, returning the large object locator information to the client to receive the large object data request information uploaded by the client specifically includes:

[0034] The large object locator information is returned to the client so that the client can generate large object data request information based on a preset remote function call protocol and receive the large object data request information uploaded by the client; wherein, the preset remote function call protocol is a preset remote FuncCall.

[0035] Optionally, in one or more embodiments of this specification, determining the current locator information of the large object to be queried based on the large object data request information, locating the data position of the large object's secondary table based on the current locator information, and returning the large object data corresponding to that data position to the client, specifically includes:

[0036] Based on the large object data request information, the current locator information of the large object to be queried is determined to verify the validity of the current locator information;

[0037] If the verification result is determined to be valid, then based on the unique identifier corresponding to the large object in the current locator information, the dependency relationship between the large object first-level table and the large object second-level table is associated, and the large object second-level table to be queried is determined based on the dependency relationship.

[0038] Based on the shard identifier field and data shard location of the current locator information, the distributed storage physical table of the second-level table of the large object to be queried is sequentially retrieved to determine the data location corresponding to the large object data to be queried, so as to return the large object data corresponding to the data location to the client.

[0039] Optionally, in one or more embodiments of this specification, returning the large object data corresponding to the data location to the client specifically includes:

[0040] Based on the client's runtime environment data and network data, the return type of the large object data is determined; wherein, the return type includes: chunked transmission type and compressed processing type;

[0041] If the return type is a chunked transmission type, the large object data is sliced ​​based on the page size to obtain each large object data to be transmitted;

[0042] If the return type is a compression processing type, then existing compression algorithms are selected based on the runtime environment data, and the large object data is compressed according to the existing compression algorithms to obtain the large object data to be transmitted.

[0043] The large object data to be transmitted is encapsulated based on a pre-defined remote function call protocol and returned to the client.

[0044] This specification provides one or more embodiments of an Oracle database large object migration device, the device comprising:

[0045] At least one processor; and,

[0046] A memory communicatively connected to the at least one processor; wherein,

[0047] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform any of the methods described above.

[0048] This specification provides one or more embodiments of a non-volatile computer storage medium storing computer-executable instructions, the computer-executable instructions being configured to execute any of the methods described above.

[0049] The above-described at least one technical solution adopted in the embodiments of this specification can achieve the following beneficial effects: Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0051] Figure 1 A flowchart illustrating a method for migrating large objects in an Oracle database, provided as an embodiment of this specification.

[0052] Figure 2 This is a schematic diagram illustrating a scenario for migrating large objects in an Oracle database, as provided in an embodiment of this specification.

[0053] Figure 3 A flowchart illustrating the implicit creation of a second-level table for a large object, provided as an embodiment of this specification;

[0054] Figure 4 This is a schematic diagram of the structure of a large object migration device for an Oracle database, provided as an embodiment of this specification.

[0055] Figure 5 This is a schematic diagram of the structure of a non-volatile storage medium provided in the embodiments of this specification. Detailed Implementation

[0056] This specification provides an embodiment of a method, device, and medium for migrating large objects in an Oracle database.

[0057] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0058] like Figure 1 As shown in the diagram, this specification provides a flowchart illustrating a method for migrating large objects in an Oracle database. Figure 1 As can be seen from one or more embodiments of this specification, a method for migrating large objects in an Oracle database includes:

[0059] S101: Detect the creation SQL statement on the server side to determine whether the currently created table column has a large object column based on the creation SQL statement.

[0060] Large object data types can store large amounts of data and allow direct insert and update operations on large object columns. Furthermore, large object data can be modified and queried through interfaces. However, the lack of a standard SQL large object data type in the PostgreSQL database system makes migrating large object data from Oracle to PostgreSQL extremely difficult. Therefore, to detect the migration bottleneck of Oracle databases—namely, large object data—this specification's embodiments detect the server-side creation SQL statements to determine whether the currently created table columns contain large object columns. This process, by detecting the server-side creation SQL statements to determine the existence of large object columns, accurately identifies the key factors that may cause migration difficulties—namely, large object data—early in the database migration process. This facilitates the subsequent implicit creation of corresponding large object sub-tables and the generation of locator information, ensuring the proper storage and access of large object data in the new database and guaranteeing the availability and manageability of large object data after migration.

[0061] Specifically, in one or more embodiments of this specification, detecting the server-side creation SQL statement to determine whether the currently created table column has a large object column based on the creation SQL statement includes the following process:

[0062] First, a pre-built SQL parser, such as ANTLR or JavaCC, can be used to parse the creation SQL statement, breaking it down into nodes and generating an abstract syntax tree (AST). Then, this AST is traversed to check if the data type of each column is a pre-defined large object type. If not, then... Figure 3 The example shown demonstrates the creation of a regular table based on a SQL statement. However, if the default large object type is used, then the table creation will be confirmed to have large object columns. This process automatically identifies large object columns by parsing the SQL, achieving automated identification of Oracle large object columns and laying the foundation for subsequent large object data migration. By separating large object data from regular data, the lack of a standard large object type in PostgreSQL is resolved, ensuring the feasibility and efficiency of the Oracle to PostgreSQL migration.

[0063] S102: If so, then based on the large object second-level table structure, a corresponding large object second-level table is implicitly created, so that the locator information of the large object data is generated and stored in the large object first-level table based on the large object second-level table.

[0064] Based on step S101 above, if it is determined that the currently created table column has a large object column, then a corresponding large object secondary table can be implicitly created according to the large object secondary table structure. Then, the locator information for the large object data is generated from the large object secondary table and stored in the large object primary table. This process, by introducing the large object secondary table structure, can simulate Oracle's large object storage through the secondary table, allowing upper-layer applications to migrate smoothly without being aware of the underlying database differences. By generating locator information and storing it in the large object primary table, the large object data is split into independent secondary tables, with the primary table only storing locator information, thus effectively reducing the IO overhead of the primary table. Therefore, the introduction of the large object secondary table and locator information effectively solves the problem of PostgreSQL lacking a standard large object type. Furthermore, the large object secondary table in this process adopts a distributed scheme, meaning that each table containing a large object column has a unique secondary large object table, while the native system uses a centralized storage method for large objects, storing all large objects in system tables. This distributed secondary table large object scheme significantly improves the performance of large object data processing in high-concurrency scenarios.

[0065] Specifically, in one or more embodiments of this specification, a corresponding large object second-level table is implicitly created based on the large object second-level table structure, and the locator information of the large object is generated based on the large object second-level table, specifically including:

[0066] First, to quickly identify the corresponding second-level table for the large object in subsequent operations, the unique identifier of the large object's first-level table is obtained. The table name of the second-level table is then generated based on this unique identifier. Next, according to the structure of the large object's second-level table, fixed fields are determined to create a second-level table containing these fixed fields. Finally, the dependency relationship between the large object's first-level and second-level tables is established based on the table name. It should be noted that the large object's second-level table structure contains three fixed fields: lobid, pageno, and data columns. lobid is the unique identifier for the large object, pageno is the data slice page number, and data is the slice data for pageno.

[0067] Then, to ensure that each large object data has a clear correspondence in the secondary table, it is necessary to determine the unique identifier corresponding to each large object data based on the secondary table, and bind the large object data to the primary table based on the dependency relationship, so that the large objects can be accurately associated and identified in both the primary and secondary tables. To balance storage efficiency and computing performance, this embodiment determines the corresponding page size based on the current computing resource limitations, and adds a corresponding shard identifier field to the secondary table based on the page size, thus sharding the large object data and achieving distributed storage. The unique identifier corresponding to each large object, the shard identifier field corresponding to the large object data, and the data shard position are encapsulated based on a preset format to obtain the locator information of the large objects.

[0068] Furthermore, in one or more embodiments of this specification, the corresponding page size is determined based on the current computing resource limitation data, and a corresponding sharding identifier field is added to the large object secondary table according to the page size to shard the large object data and realize distributed storage. Specifically, the process includes the following:

[0069] Based on current computing resource constraints, the migration scenarios for large Oracle objects are determined. These scenarios include high-memory, low-transfer scenarios and low-memory, high-concurrency scenarios. The page size within the corresponding page range for each migration scenario is determined, matching the current computing resource constraints. For example, in a high-memory, low-transfer scenario, due to relatively abundant memory resources and limited transfer capacity, a larger page size is chosen to reduce the number of transfers and fully utilize the advantages of high memory. Conversely, in a low-memory, high-concurrency scenario, with limited memory resources but high concurrency requirements, a smaller page size is chosen to adapt to the low memory constraints and support more concurrent operations. Then, sharding rules are determined based on the page size and page number. Based on these rules, the sharding positions of the second-level tables for large objects are determined, and corresponding sharding identifier fields are added to the second-level tables according to their sharding positions. Finally, large object data within the second-level tables that share the same sharding identifier field is stored in a distributed storage physical table, and the mapping relationship between the data sharding positions and the large object data is recorded. In other words, compared to the traditional method of inserting all large objects into the system table, this process uses a distributed storage method in high-concurrency scenarios. By sharding data, large objects are distributed into multiple physical tables according to the user table, which enables query requests to be processed in parallel, thereby breaking through the original single-table resource limitations and improving the overall throughput of the system.

[0070] Furthermore, in one or more embodiments of this specification, based on the large object second-level table structure, a corresponding large object second-level table is implicitly created. After the locator information of the large object data is generated and stored in the large object first-level table based on the large object second-level table, in order to solve the problem that it is difficult to freely insert or update data into a specified large object column in the existing method, the method further includes:

[0071] It receives updated data from the large object first-level table to determine whether to trigger the large object data insertion process. If the large object data insertion process is triggered, it inserts the locator information into the large object first-level table and determines the specified large object column corresponding to the updated data. For example... Figure 2 As shown, based on the locator information and the specified large object column, the updated data is directly added to the second-level table of the large object corresponding to the specified large object column. After determining whether to trigger the large object data insertion process based on the updated data, the method further includes: if the large object data insertion process is triggered, then calling a pre-defined interface such as... Figure 2The diagram shows how to update the large object's second-level table based on the locator information. There are two methods for inserting data into the large object table: one is direct data insertion, where data can be directly specified for insertion into the table. This method generates large object locator information before data insertion. Since we can locate the large object data in the second-level table using only the large object's second-level table OID and the large object's unique identifier OID, these two OIDs form the locator information. The locator structure is as follows: (16628, 888), where 16628 represents the large object's second-level table OID and 888 is the large object's OID. The generated locator is then inserted into the large object's first-level table column, specifying that the inserted data is sharded and stored in the second-level table. The second method utilizes an interface to modify and append data to the large object's data through the large object locator.

[0072] During this process, locator information for large object data is generated and stored in the first-level table of large objects. The locator information plays a crucial positioning role, containing essential information pointing to the second-level table of large objects and the location of its sharded data, facilitating subsequent access and management of the large object data. When the large object data insertion process is triggered, the locator information is first inserted into the first-level table of large objects. This step records the positioning information related to the large object data in the first-level table. The system then determines the specific large object column corresponding to the updated data, clarifying the exact location where the data should be inserted into the first-level table. Based on the locator information previously stored in the first-level table and the determined specific large object column, the system directly adds the updated data to the second-level table corresponding to the specified large object column. This ensures accurate writing of data from the first-level table to the second-level table, guaranteeing correct data storage and management. Adding data to the second-level table based on the locator and updating by calling pre-defined interfaces enables flexible insertion and update operations for large object data, improving the efficiency and convenience of data management.

[0073] S103: Return the locator information to the client to receive the large object data request information uploaded by the client.

[0074] To facilitate client-side queries of large object columns, in this embodiment of the specification, after obtaining the locator information based on the above steps, as follows: Figure 2 As shown, the locator information will be returned to the client to receive requests for large object data uploaded by the client. This allows the client to subsequently retrieve the large object data via remote FuncCall.

[0075] Specifically, in one or more embodiments of this specification, returning large object locator information to the client to receive large object data request information uploaded by the client specifically includes:

[0076] The server returns the locator information for large objects to the client, enabling the client to generate large object data request information based on a pre-defined remote function call protocol and to receive the large object data request information uploaded by the client. It should be noted that the pre-defined remote function call protocol is the pre-defined remote FuncCall. The server's return of the locator information to the client clarifies the storage location and related mapping relationships of the large object data. When receiving data requests uploaded by the client, the server can quickly locate the corresponding large object's secondary table and shard location for storage based on the locator information, avoiding data corruption and duplicate processing, and improving the accuracy and efficiency of data management.

[0077] S104: Determine the current locator information of the large object data to be queried based on the large object data request information, locate the data position of the second-level table of the large object based on the current locator information, and return the large object data corresponding to the data position to the client.

[0078] After determining the current locator information of the large object data to be queried based on the large object data request information, the server in this embodiment locates the data position of the large object in the secondary table based on the current locator information, and returns the large object data corresponding to that data position to the client. For example, in a certain scenario, when the server receives a remote FuncCall query for large object data from the client, it will locate the position of the large object in the secondary table based on the locator information, then retrieve all the large object data and return it to the client. That is, the insertion and querying of large objects are inextricably linked to the secondary table. When inserting data into a large object, the secondary table stores the specific inserted large object data. During a query, the locator information stored in the primary table is used to locate the data item of the large object in the secondary table, and then the large object data retrieved from the secondary table is returned to the querying end. Among these, the locator plays a crucial role. The locator information generated when the large object is inserted is stored in the primary table, and the large object data is retrieved by using the obtained locator when querying the table.

[0079] Specifically, in one or more embodiments of this specification, the current locator information of the large object to be queried is determined based on the large object data request information, the data location of the second-level table of the large object is located based on the current locator information, and the large object data corresponding to the data location is returned to the client. The specific process includes the following:

[0080] The process begins by determining the current locator information for the large object to be queried based on the large object data request information, and then verifying the validity of this information. If the verification result is valid, the dependency relationship between the first-level and second-level tables of the large object is established based on the unique identifier corresponding to the large object according to the current locator information. The second-level table of the large object to be queried is then determined based on this dependency relationship. Next, based on the shard identifier field and data shard location in the current locator information, the distributed storage physical tables of the second-level table of the large object to be queried are sequentially retrieved to determine the data location corresponding to the large object data. The large object data corresponding to that data location is then returned to the client. Throughout this process, the large object data is processed according to the locator. This locator design not only solves the problem that traditional data types in PostgreSQL databases cannot exceed 1GB of data volume, but also addresses the issues of traditional large object data not being able to be directly inserted or queried.

[0081] Specifically, in one or more embodiments of this specification, returning the large object data corresponding to the data location to the client includes:

[0082] First, the return type of the large object data is determined based on the client's runtime environment data and network data. The return type includes: chunked transmission type and compressed processing type; for example, the return type can be determined based on thresholds corresponding to the runtime environment data and network data, respectively. If the return type is determined to be chunked transmission, the large object data is sliced ​​according to the page size to obtain individual large object data to be transmitted. If the return type is compressed processing, existing compression algorithms are selected based on the runtime environment data, and the large object data is compressed using these algorithms to obtain the large object data to be transmitted. Then, the large object data to be transmitted is encapsulated using a pre-defined remote function call protocol and returned to the client. This process, by determining the return type based on the client's runtime environment and network conditions, allows for flexible adaptation to different conditions. For example, compressed transmission can be enabled under weak network conditions, and chunked transmission can be used when client storage is limited, improving data transmission efficiency and availability. Encapsulating data using a pre-defined remote function call protocol ensures transmission standardization and compatibility, facilitating client parsing and processing.

[0083] like Figure 4 As shown in the diagram, this specification provides a structural schematic of a migration device for large objects in an Oracle database. Figure 4 As can be seen from one or more embodiments of this specification, an Oracle database large object migration device includes:

[0084] At least one processor; and,

[0085] A memory communicatively connected to the at least one processor; wherein,

[0086] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform any of the methods described above.

[0087] like Figure 5 As shown in the diagram, this specification provides a schematic diagram of the structure of a non-volatile storage medium. Figure 5 As can be seen, in one or more embodiments of this specification, a non-volatile storage medium stores computer-executable instructions 501, which are capable of executing any of the methods described above.

[0088] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and non-volatile computer storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0089] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0090] The above description is merely one or more embodiments of this specification and is not intended to limit this specification. Various modifications and variations can be made to the one or more embodiments of this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of one or more embodiments of this specification should be included within the scope of the claims of this specification.

Claims

1. A method for migrating large objects in an Oracle database, characterized in that, The method includes: The server-side creation SQL statement is detected to determine whether the currently created table column has a large object column based on the creation SQL statement; If so, then based on the large object second-level table structure, a corresponding large object second-level table is implicitly created, and based on the large object second-level table, the locator information of the large object data is generated and stored in the large object first-level table; The locator information is returned to the client to receive the large object data request information uploaded by the client; Based on the large object data request information, the current locator information of the large object data to be queried is determined, so as to locate the data position of the second-level table of the large object based on the current locator information, and return the large object data corresponding to the data position to the client; Based on the large object second-level table structure, a corresponding large object second-level table is implicitly created to generate the locator information of the large object based on the large object second-level table, specifically including: Obtain the unique identifier of the first-level table of the large object, and generate the table name of the second-level table of the large object based on the unique identifier of the first-level table of the large object; Based on the large object second-level table structure, the fixed fields of the large object second-level table are determined to create a large object second-level table containing the fixed fields, and the dependency relationship between the large object first-level table and the large object second-level table is established based on the table name of the large object second-level table. Based on the second-level table of the large object, a unique identifier corresponding to each large object data is determined, and based on the dependency association, each large object data is bound to the first-level table of the large object; Based on the current computing resource constraints, the corresponding page size is determined, and a corresponding sharding identifier field is added to the large object secondary table according to the page size, so as to shard the large object data and realize distributed storage. The unique identifier corresponding to each large object, the fragment identifier field corresponding to the large object data, and the data fragment position are encapsulated based on a preset format to obtain the locator information of the large object.

2. The method for migrating large objects in an Oracle database according to claim 1, characterized in that, The server-side creation SQL statement is detected to determine whether the currently created table column has a large object column, specifically including: The SQL statement to be created is parsed to generate an abstract syntax tree for the SQL statement to be created. The abstract syntax tree is traversed to check whether the data type of each column is a preset large object type; If not, then create a regular table based on the SQL statement described above; If so, then determine that the currently created table column has a large object column.

3. The method for migrating large objects in an Oracle database according to claim 1, characterized in that, Based on the current computing resource limitations, the corresponding page size is determined. A corresponding sharding identifier field is added to the second-level table of the large object according to the page size, thereby sharding the large object data to achieve distributed storage. Specifically, this includes: Based on current computing resource constraints, determine the migration scenarios for large Oracle objects; these migration scenarios include: high memory, low data transfer scenarios and low memory, high concurrency scenarios. Determine the page size that matches the current computing resource limit data within the page range corresponding to the migration scenario; Based on the page size and page number, a sharding rule is determined, and the sharding position of the large object second-level table is determined based on the sharding rule. A corresponding sharding identifier field is added to the large object second-level table according to the sharding position. Large object data located in the same shard identifier field within the secondary table of the large object are stored in a distributed storage physical table, and the mapping relationship between the data shard location and the large object data is recorded.

4. The method for migrating large objects in an Oracle database according to claim 1, characterized in that, Based on the large object second-level table structure, a corresponding large object second-level table is implicitly created. After the locator information of the large object data is generated and stored in the large object first-level table based on the large object second-level table, the method further includes: Receive updated data from the first-level table of the large object, and determine whether to trigger the large object data insertion process based on the updated data; If the large object data insertion process is triggered, the locator information is inserted into the large object first-level table, and the specified large object column corresponding to the updated data is determined. Based on the locator information and the specified large object column, the updated data is directly added to the large object sub-table corresponding to the specified large object column; After determining whether to trigger the large object data insertion process based on the updated data, the method further includes: If the large object data insertion process is triggered, a preset interface is called to determine the large object secondary table corresponding to the updated data based on the locator information and update it.

5. The method for migrating large objects in an Oracle database according to claim 1, characterized in that, Returning locator information to the client to receive large object data request information uploaded by the client, specifically including: The locator information is returned to the client so that the client can generate large object data request information based on a preset remote function call protocol and receive the large object data request information uploaded by the client; wherein, the preset remote function call protocol is a preset remote FuncCall.

6. A method for migrating large objects in an Oracle database according to claim 3, characterized in that, Based on the large object data request information, the current locator information of the large object to be queried is determined, and the data location of the second-level table of the large object is located based on the current locator information. The large object data corresponding to the data location is then returned to the client. Specifically, this includes: Based on the large object data request information, the current locator information of the large object to be queried is determined to verify the validity of the current locator information; If the verification result is determined to be valid, then based on the unique identifier corresponding to the large object in the current locator information, the dependency relationship between the large object first-level table and the large object second-level table is associated, and the large object second-level table to be queried is determined based on the dependency relationship. Based on the shard identifier field and data shard location of the current locator information, the distributed storage physical table of the second-level table of the large object to be queried is sequentially retrieved to determine the data location corresponding to the large object data to be queried, so as to return the large object data corresponding to the data location to the client.

7. The method for migrating large objects in an Oracle database according to claim 1, characterized in that, Returning the large object data corresponding to the data location to the client, specifically including: Based on the client's runtime environment data and network data, the return type of the large object data is determined; wherein, the return type includes: chunked transmission type and compressed processing type; If the return type is a chunked transmission type, the large object data is sliced ​​based on the page size to obtain each large object data to be transmitted; If the return type is a compression processing type, then existing compression algorithms are selected based on the runtime environment data, and the large object data is compressed according to the existing compression algorithms to obtain the large object data to be transmitted. The large object data to be transmitted is encapsulated based on a pre-defined remote function call protocol and returned to the client.

8. A migration device for large objects in an Oracle database, characterized in that, The device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method described in any one of claims 1-7.

9. A non-volatile storage medium storing computer-executable instructions, characterized in that, The computer-executable instructions are capable of performing the method described in any one of claims 1-7.

Images