An Optimization Method and System for PostgreSQL PL / Java Based on Adaptive JVM Warm-up and Zero-Copy Data Exchange

Abstract

The application provides a PostgreSQL PLJava optimization method and system based on adaptive JVM preheating and zero-copy data exchange, which comprises the following steps: constructing a load-aware hierarchical JVM instance pool, dividing the Java execution environment into three levels of instances, i.e., cold state, warm state and hot state; establishing a zero-copy data exchange channel based on the mapping of the PostgreSQL dynamic shared memory and the off-heap memory of JVM, realizing cross-language direct reading and direct writing through the memory layout alignment protocol and the memory barrier synchronization mechanism; deploying an adaptive dynamic call routing engine, routing the request to the corresponding instance based on the call portrait and load prediction; and implementing transaction boundary-aware cooperative GC scheduling, triggering concurrent garbage collection by using the low activity time window of the database. The application further discloses a corresponding optimization system. The application can significantly reduce the PL / Java first call delay, reduce the cross-language data copy overhead, realize adaptive management of memory resources, and eliminate the interference of garbage collection on database transactions.

Description

Technical Field

[0001] This invention relates to the fields of database technology and Java Virtual Machine optimization technology, and in particular to a PostgreSQL PLJava optimization method and system based on adaptive JVM warm-up and zero-copy data exchange. Background Technology

[0002] PostgreSQL databases, through PL / Java extension modules, support the direct execution of Java-written stored procedures and functions within the database kernel. Current mainstream implementations typically employ an embedded JVM architecture, loading the JVM into the database backend process space via dynamic link libraries, initializing a single JVM instance or maintaining a fixed number of JVM connection pools. Parameter passing and result return between Java functions and the PostgreSQL backend process rely on field-level read / write APIs provided by the JNI interface for serialization and deserialization operations. Garbage collection strategies generally employ the default generational collector, triggering collection based on fixed thresholds.

[0003] However, the aforementioned existing technologies have the following drawbacks: First, the first call to a Java function requires JVM initialization, core library loading, and JIT compilation warm-up, resulting in significantly high cold start latency, making it difficult to meet the low latency requirements of high-frequency, short-query scenarios. Second, the PostgreSQL backend and JVM reside in different memory spaces, and complex data structures require multiple serializations and memory copies via JNI, leading to high CPU utilization and memory bandwidth bottlenecks. Third, a resident full JVM instance consumes a large amount of heap memory, resulting in resource waste during idle periods, while on-demand startup sacrifices response speed; fixed GC strategies are prone to triggering global pauses during peak transaction periods, blocking database query links. Fourth, existing PL / Java implementations use static instance allocation and unified weight configuration, which cannot dynamically adjust the execution environment based on function call frequency, data volume, and load waveform, leading to a technical contradiction of insufficient throughput under high concurrency and resource idleness in low-frequency scenarios. Summary of the Invention

[0004] To address the aforementioned technical problems in related technologies, this invention proposes an optimization method and system for PostgreSQL PL Java based on adaptive JVM warm-up and zero-copy data exchange, which can overcome the above-mentioned shortcomings of existing technologies.

[0005] To achieve the above-mentioned technical objectives, the technical solution of the present invention is implemented as follows: An optimization method for PostgreSQL PL Java based on adaptive JVM warm-up and zero-copy data exchange; This PostgreSQL PLJava optimization method based on adaptive JVM warm-up and zero-copy data exchange includes the following steps: Construct a load-aware, tiered JVM instance pool: Based on function call frequency characteristics and warm-up status, maintain a three-level JVM instance pool containing cold instances, warm instances, and hot instances. The cold instances run based on pre-compiled native executables, the warm instances run in a lightweight JVM with pre-loaded core class libraries, and the hot instances run in a standard JVM with just-in-time compilation optimization enabled. Establish a zero-copy data exchange channel: allocate a PostgreSQL dynamic shared memory segment, map the shared memory segment to the JVM off-heap memory, define a cross-language memory layout alignment protocol including the metadata area and the data payload area, and establish a synchronization mechanism based on memory barriers, so that the PostgreSQL backend process and the JVM can directly read and write the shared memory segment through the physical address space to exchange function parameters and execution results; Deploy an adaptive dynamic call routing engine: collect the call profile of the target Java function in real time, predict the load probability of the next cycle based on the call profile, and dynamically route the current call to the corresponding instance in the hierarchical JVM instance pool for execution according to the adaptive calibration threshold and prediction results; Implement coordinated GC scheduling: Register PostgreSQL transaction manager callback hooks, listen for database transaction boundary events and background worker idle signals, identify low-activity time windows, and trigger the current JVM instance to perform concurrent garbage collection within the time window.

[0006] Furthermore, the tiered JVM instance pool is maintained by the instance pool manager, which manages the health status, load count, and circuit breaker policy of each instance. When an instance experiences consecutive timeouts or memory overflows, the instance pool manager automatically triggers isolation and destruction and rebuilds the instance from the backup image pool.

[0007] Furthermore, the adaptive dynamic call routing engine uses an exponentially weighted moving average combined with a first-order Markov state transition model to predict the load probability of the next cycle, and determines the adaptive calibration threshold used for routing decisions based on offline calibration of the cluster centers of historical call distributions and online fine-tuning during runtime.

[0008] Furthermore, the dynamic call routing engine also performs a context atomic migration mechanism: before switching execution environments, a snapshot containing the transaction isolation level, current transaction identifier, cursor state, and SPI call stack frame is created through the PostgreSQL memory context manager; after the new instance takes over, the snapshot is atomically restored through a two-phase commit protocol; if the restoration fails, a rollback compensation is triggered and the instance is downgraded to the previous stable instance.

[0009] Furthermore, in the collaborative GC scheduling step, the logic for identifying low-activity time windows is as follows: the time difference between the end time of a database transaction and the start time of the last query is greater than a preset threshold; when a low-activity time window is determined to exist, ZGC or Shenandoah concurrent mark-and-sweep is triggered; when cross-language JNI reference chain leakage or memory fragmentation rate exceeds a preset threshold is detected, the system automatically downgrades to the G1 generational garbage collection strategy and generates a heap dump snapshot.

[0010] Furthermore, in the zero-copy data exchange channel, after the PostgreSQL backend process completes the data writing, it refreshes the cache consistency through a memory barrier. After the JVM polls the metadata area status flag to confirm that it is ready, it obtains the data through the memory barrier, thereby bypassing the JNI field-level copy path.

[0011] According to another aspect of the present invention, a PostgreSQL PL Java optimization system based on adaptive JVM warm-up and zero-copy data exchange is provided; This PostgreSQL PL Java optimization system based on adaptive JVM warm-up and zero-copy data exchange includes: an instance pool management module configured to maintain the JVM process lifecycle according to a three-level state machine of cold state, warm state, and hot state; The shared memory bridging module is configured to allocate PostgreSQL dynamic shared memory segments and perform cross-language memory layout alignment and address mapping. The dynamic routing decision module is configured to collect call profiles in real time, run load prediction algorithms, and perform instance switching based on adaptive thresholds. The collaborative GC scheduling module is configured to register transaction callback hooks, identify low-activity time windows, and trigger concurrent garbage collection. The PL / Java function execution proxy module is configured to receive routing instructions and bind zero-copy channels to read and write function parameters and results. The monitoring and diagnostic module is configured to collect performance metrics and output optimization suggestions. Each module interacts with shared memory through the PostgreSQL internal message bus and is dynamically loaded into the database process space as a shared library.

[0012] Furthermore, the instance pool management module is also used to perform health checks, load statistics, and fault reconstruction; when an instance experiences continuous timeouts or memory overflows, it automatically triggers isolation and destruction and is rebuilt from the backup image pool; the shared memory bridging module implements a zero-copy read / write interface by defining a memory layout alignment protocol that includes a metadata area and a data payload area, and establishing a synchronization mechanism based on memory barriers.

[0013] Furthermore, the dynamic routing decision module uses an exponentially weighted moving average combined with a first-order Markov state transition model to predict the load probability, and determines the adaptive calibration threshold based on offline calibration of the cluster centers of historical call distribution and online fine-tuning during runtime; when performing instance switching, the database ACID characteristics are ensured by saving and atomically restoring SPI context snapshots.

[0014] Furthermore, the collaborative GC scheduling module identifies low-activity time windows based on the time difference between the end of a database transaction and the start of the last query being greater than a preset threshold. When a low-activity time window is identified, ZGC or Shenandoah concurrent mark-and-sweep is triggered. When a cross-language JNI reference chain leak is detected or the memory fragmentation rate exceeds a preset threshold, the module automatically downgrades to the G1 generational garbage collection strategy and generates a heap dump snapshot.

[0015] The beneficial effects of this invention are as follows: By constructing a three-tiered JVM instance pool (cold, warm, and hot) and implementing adaptive routing, the initial call latency is significantly reduced and memory resources can be elastically scaled on demand; by establishing a zero-copy data exchange channel based on shared memory and off-heap memory mapping and a memory barrier synchronization mechanism, cross-language data interaction bypasses JNI serialization overhead, resulting in a significant increase in throughput; through transaction boundary-aware collaborative GC scheduling, concurrent garbage collection is performed using a low-activity window in the database, keeping garbage collection pause time below milliseconds, thereby completely eliminating the technical defects of high cold start latency, large data copy overhead, low resource utilization, and GC blocking database transactions in traditional PL / Java. Attached Figure Description

[0016] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 This is the overall flowchart of the PostgreSQL PL Java optimization method based on adaptive JVM warm-up and zero-copy data exchange described in this invention; Figure 2 This is a schematic diagram of the hierarchical JVM instance pool and dynamic routing state machine of the present invention; Figure 3 This is a schematic diagram illustrating the alignment of the PG DSM and JVM DirectMemory zero-copy memory layout in this invention. Figure 4 This is a timing diagram of the transaction-aware collaborative GC scheduling of the present invention; Figure 5 This is a diagram showing the system module architecture and data flow relationship of the PostgreSQL PL Java optimization system based on adaptive JVM warm-up and zero-copy data exchange as described in this invention. Detailed Implementation

[0018] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," and "counterclockwise," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting this invention.

[0020] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified. Furthermore, the terms "installed," "connected," and "linked" should be interpreted broadly; for example, they may refer to a fixed connection, a detachable connection, or an integral connection; they may refer to a mechanical connection or an electrical connection; they may refer to a direct connection or an indirect connection through an intermediate medium; and they may refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0021] like Figures 1 to 4 As shown, the PostgreSQL PLJava optimization method based on adaptive JVM warm-up and zero-copy data exchange according to an embodiment of the present invention includes the following steps: Construct a load-aware, tiered JVM instance pool: Based on function call frequency characteristics and warm-up status, maintain a three-level JVM instance pool containing cold instances, warm instances, and hot instances. The cold instances run based on pre-compiled native executables, the warm instances run in a lightweight JVM with pre-loaded core class libraries, and the hot instances run in a standard JVM with just-in-time compilation optimization enabled. Establish a zero-copy data exchange channel: allocate a PostgreSQL dynamic shared memory segment, map the shared memory segment to the JVM off-heap memory, define a cross-language memory layout alignment protocol including the metadata area and the data payload area, and establish a synchronization mechanism based on memory barriers, so that the PostgreSQL backend process and the JVM can directly read and write the shared memory segment through the physical address space to exchange function parameters and execution results; Deploy an adaptive dynamic call routing engine: collect the call profile of the target Java function in real time, predict the load probability of the next cycle based on the call profile, and dynamically route the current call to the corresponding instance in the hierarchical JVM instance pool for execution according to the adaptive calibration threshold and prediction results; Implement coordinated GC scheduling: Register PostgreSQL transaction manager callback hooks, listen for database transaction boundary events and background worker idle signals, identify low-activity time windows, and trigger the current JVM instance to perform concurrent garbage collection within the time window.

[0022] According to an embodiment of the present invention, the PostgreSQLPLJava optimization method based on adaptive JVM warm-up and zero-copy data exchange is described in a specific implementation. In one specific implementation, the hierarchical JVM instance pool is maintained by the instance pool manager, which maintains the health status, load count, and circuit breaker policy of each instance. When an instance experiences continuous timeouts or memory overflows, the instance pool manager automatically triggers isolation and destruction and rebuilds the instance from the backup image pool.

[0023] According to an embodiment of the present invention, the PostgreSQLPLJava optimization method based on adaptive JVM warm-up and zero-copy data exchange, in a specific implementation, the adaptive dynamic call routing engine uses an exponentially weighted moving average combined with a first-order Markov state transition model to predict the load probability of the next cycle, and determines the adaptive calibration threshold used for routing decisions based on offline calibration of the cluster centers of historical call distribution and online fine-tuning during runtime.

[0024] According to an embodiment of the present invention, the PostgreSQLPLJava optimization method based on adaptive JVM warm-up and zero-copy data exchange, in a specific implementation, the dynamic call routing engine also performs a context atomic migration mechanism: before switching execution environments, a snapshot containing transaction isolation level, current transaction identifier, cursor state, and SPI call stack frame is created through the PostgreSQL memory context manager; after the new instance takes over, the snapshot is atomically restored through a two-phase commit protocol; if the restoration fails, a rollback compensation is triggered and the instance is downgraded to the previous stable instance.

[0025] According to an embodiment of the present invention, the PostgreSQLPLJava optimization method based on adaptive JVM warm-up and zero-copy data exchange is described in a specific implementation. In the collaborative GC scheduling step, the judgment logic for identifying low-activity time windows is as follows: the time difference between the end time of a database transaction and the start time of the last query is greater than a preset threshold; when a low-activity time window is determined to exist, ZGC or Shenandoah concurrent mark-and-sweep is triggered; when cross-language JNI reference chain leakage or memory fragmentation rate exceeds a preset threshold is detected, the system automatically downgrades to the G1 generational garbage collection strategy and generates a heap dump snapshot.

[0026] According to an embodiment of the present invention, in a specific implementation of the PostgreSQLPLJava optimization method based on adaptive JVM warm-up and zero-copy data exchange, the PostgreSQL backend process refreshes cache consistency through a memory barrier after completing data writing, and the JVM side polls the metadata area status flag to confirm readiness before obtaining data through the memory barrier, thereby bypassing the JNI field-level copy path.

[0027] Secondly, such as Figure 5 As shown, the PostgreSQL PLJava optimization system based on adaptive JVM warm-up and zero-copy data exchange according to an embodiment of the present invention includes: an instance pool management module, configured to maintain the JVM process lifecycle according to a three-level state machine of cold state, warm state, and hot state; The shared memory bridging module is configured to allocate PostgreSQL dynamic shared memory segments and perform cross-language memory layout alignment and address mapping. The dynamic routing decision module is configured to collect call profiles in real time, run load prediction algorithms, and perform instance switching based on adaptive thresholds. The collaborative GC scheduling module is configured to register transaction callback hooks, identify low-activity time windows, and trigger concurrent garbage collection. The PL / Java function execution proxy module is configured to receive routing instructions and bind zero-copy channels to read and write function parameters and results. The monitoring and diagnostic module is configured to collect performance metrics and output optimization suggestions. Each module interacts with shared memory through the PostgreSQL internal message bus and is dynamically loaded into the database process space as a shared library.

[0028] In a specific embodiment of the PostgreSQLPLJava optimization system based on adaptive JVM warm-up and zero-copy data exchange according to an embodiment of the present invention, the instance pool management module is further used to perform health checks, load statistics, and fault reconstruction; when an instance experiences continuous timeouts or memory overflows, it automatically triggers isolation and destruction and is rebuilt from the backup image pool; the shared memory bridging module implements a zero-copy read / write interface by defining a memory layout alignment protocol that includes a metadata area and a data payload area, and establishing a synchronization mechanism based on memory barriers.

[0029] According to an embodiment of the present invention, the PostgreSQLPLJava optimization system based on adaptive JVM warm-up and zero-copy data exchange, in a specific implementation, the dynamic routing decision module uses an exponentially weighted moving average combined with a first-order Markov state transition model to predict the load probability, and determines the adaptive calibration threshold based on offline calibration of the cluster centers of historical call distribution and online fine-tuning during runtime; when performing instance switching, the database ACID characteristics are ensured by saving and atomically restoring SPI context snapshots.

[0030] According to an embodiment of the present invention, the PostgreSQLPLJava optimization system based on adaptive JVM warm-up and zero-copy data exchange, in a specific implementation, the collaborative GC scheduling module identifies low-activity time windows based on the time difference between the end of a database transaction and the start of the last query being greater than a preset threshold. When a low-activity time window is identified, ZGC or Shenandoah concurrent mark-and-sweep is triggered. When a cross-language JNI reference chain leak or memory fragmentation rate exceeds a preset threshold, the system automatically downgrades to the G1 generational garbage collection strategy and generates a heap dump snapshot.

[0031] To facilitate understanding of the above technical solutions of the present invention, the following detailed description of the above technical solutions of the present invention is provided through specific embodiments and working principles.

[0032] Example 1: System Deployment and Basic Function Verification This embodiment is used to verify the basic functions of the method and system of the present invention, including the automatic initialization of the hierarchical JVM instance pool, the establishment of zero-copy channels, and the basic effects of dynamic routing.

[0033] (a) Deployment environment and initialization process On the PostgreSQL database server, the optimized extension of this invention is installed by executing the command `CREATE EXTENSION pljava_perf`. This extension is dynamically loaded into the database process space as a shared library and automatically starts a background worker process to manage the JVM instance pool.

[0034] During the initialization phase, the instance pool manager creates corresponding JVM processes based on the pre-configured number of cold, warm, and hot instances. Specifically: Cold instances are created based on pre-compiled native executables from GraalVM Native Image, stripping away dynamic class loading and JIT compilation logic, retaining only the minimum runtime required for PL / Java SPI calls. Its working principle is to convert Java bytecode into native machine code through pre-compilation, eliminating the overhead of JVM startup and class loading during the first call, making it suitable for long-tail functions with extremely low call frequency.

[0035] A warm-state instance starts a lightweight JVM process, preloads the java.lang, java.util, and org.postgresql.pljava core libraries, sets a low heap memory limit, and disables tiered compilation and escape analysis. Its working principle is to complete the loading and initialization of core classes at startup, so that class loading is not required on the first call, without consuming excessive memory, making it suitable for functions with medium-frequency calls.

[0036] A hot instance starts a standard JVM process, enabling C1 / C2 tiered compilation, loop optimization, and vectorized instruction sets. The heap memory can be dynamically expanded as needed. Its working principle is: through just-in-time compilation, hot-topic code is compiled into highly optimized native code, combined with aggressive runtime optimizations, providing optimal execution performance for frequently used core functions.

[0037] The instance pool manager maintains a Unix Domain Socket or TCP heartbeat connection for each JVM instance, monitors its health status in real time, and records the current load count and historical success rate for each instance.

[0038] (II) The process of establishing a zero-copy channel When a PostgreSQL backend process needs to call a Java function, the dynamic routing decision module first obtains the function call profile. Before the first call, the shared memory bridging module allocates a PostgreSQL dynamic shared memory segment and obtains a globally visible handle, dsm_segment_handle.

[0039] On the JVM side, the physical pages of the shared memory segment are mapped to the JVM's DirectMemory area using the methods "java.lang.foreign.MemorySegment.mapFromAddress()" or "sun.misc.Unsafe.allocateMemory()".

[0040] The shared memory segment is divided into two logical regions: Metadata area: Used to write the base address of the function call information structure, the length of the parameter array, the type identifier of each parameter, the alignment offset, and the current transaction ID. This area also contains a status flag for the synchronization handshake between the PG and JVM.

[0041] Data payload area: Scalar values, byte arrays, or binary streams of complex types are written directly to 8-byte aligned boundaries.

[0042] The data exchange process is as follows: After completing the data write, the PG (Generic Processing) refreshes cache consistency through an atomic memory barrier (atomic_thread_fence(memory_order_release)) to ensure that all write operations are visible to the JVM; then, it sets the status flag of the metadata area to "ready". The JVM polls this status flag, and once the ready state is detected, it immediately executes the atomic memory barrier (atomic_thread_fence(memory_order_acquire)), and then directly reads the data in the shared memory physical address space through the offset pointer. Throughout this process, it completely bypasses the field-level copy APIs of JNI such as GetObjectField, NewObjectArray, and SetDoubleField, achieving zero-copy direct interaction between the PG backend process memory space and the JVM off-heap memory.

[0043] (III) Dynamic routing and adaptive threshold calibration, input data volume, and historical latency quantiles. An exponentially weighted moving average combined with a first-order Markov state transition model is used to predict the load probability for the next time period. The prediction formula is as follows: P(F_{t+1})=α·F_t+(1-α)·P(F_t); Where α is the smoothing coefficient, which is dynamically adjusted online according to the load fluctuation variance.

[0044] Routing decisions are made based on adaptive calibration thresholds: when the predicted load probability is higher than the hot threshold and the data volume is large, the route is to the hot instance; when the predicted load probability is between the warm and hot thresholds, the route is to the warm instance; otherwise, the route is to the cold instance.

[0045] The initial values ​​for the hot and warm thresholds were obtained through offline calibration: a large number of historical call samples with varying call frequencies, average data volumes, and average latency were collected, and the samples were clustered into three classes using the K-Means clustering algorithm. The boundaries of the three cluster centers were used as the initial thresholds. During operation, the system performs online fine-tuning of the thresholds, for example, every 5 minutes, based on the actual load distribution variance within the sliding window, to adapt to changes in load fluctuations.

[0046] (iv) Context Atom Transfer Mechanism When the dynamic routing decision module determines that a function call needs to be switched from one instance to another, for example, from a cold instance to a warm instance, the system executes an atomic context migration process: Snapshot creation: On the PG side, switch to a dedicated memory context via MemoryContextSwitchTo, call GetTopTransactionId to obtain the current transaction ID, and save the current transaction isolation level, cursor state, and SPI call stack frame information. Package this information into a snapshot structure and store it in shared memory.

[0047] Instance switching: The instance pool manager allocates an idle instance from the target state instance pool and passes the snapshot handle to that instance.

[0048] Atomic recovery: The target JVM instance restores the snapshot content through a two-phase commit protocol—the first phase verifies the integrity of the snapshot and the validity of the transaction ID, and the second phase rebinds the transaction context, cursor position and SPI stack frame to the current execution environment.

[0049] Degradation compensation: If an anomaly occurs during the recovery process, such as corrupted snapshot data or insufficient resources on the target instance, the system will automatically trigger rollback compensation: release the allocated resources, and call downgrade to the previous stable instance to continue execution, while recording fault logs for asynchronous diagnosis.

[0050] This mechanism ensures that the ACID properties of database transactions are not violated during dynamic route switching, and prevents issues such as transaction loss, isolation level changes, or cursor misalignment.

[0051] Example 2: Transaction Boundary Awareness Cooperative GC Scheduling This embodiment illustrates how the present invention utilizes database transaction gaps to perform garbage collection, thereby eliminating the impact of GC pauses on the query chain in traditional PL / Java.

[0052] (a) Registering Transaction Callback Hooks During system initialization, this invention registers a callback hook function (RegisterXactCallback) with the PostgreSQL transaction manager. This hook is invoked upon each transaction commit or rollback, while simultaneously listening for idle signals from background worker processes.

[0053] (ii) Identification of low-activity time windows The collaborative GC scheduling module maintains the last query start timestamp t_last_query_start for each database backend process. When a transaction commit event occurs, it records the current timestamp t_commit_end and calculates the difference: Δt_idle=t_commit_end-t_last_query_start; When Δt_idle exceeds a preset threshold, the system is determined to be in a low-activity window. This window indicates that there are currently no heavy queries being executed in the database, and transactions have been committed. In this case, triggering garbage collection will not block user requests.

[0054] (III) Triggering and Degradation Strategies for Concurrent Reclamation Upon identifying a low-activity window, the coordinated GC scheduling module sends a trigger signal to the current JVM instance, instructing it to perform garbage collection. For hot and warm instances configured with ZGC or Shenandoah GC, these collectors support concurrent marking and concurrent cleanup, and their collection operations run concurrently with Java application threads for most of the time, requiring only a very short global pause. Because the triggering time is chosen during database transaction intervals, even if a brief pause occurs, it will not be detected by database queries.

[0055] Meanwhile, the monitoring and diagnostic module continuously collects the trend of GlobalRef count changes in cross-language JNI reference chains and the heap memory fragmentation rate. If the GlobalRef count is found to be continuously increasing and not being released, it indicates that there may be a reference leak, or the memory fragmentation rate exceeds a preset threshold. The system automatically downgrades the GC strategy from concurrent collection to G1 generational collection. G1 can reduce fragmentation through partitioning and generate a heap dump snapshot, which is then analyzed offline by the asynchronous diagnostic thread. The main query thread is unaware of any GC strategy adjustments or fault handling behavior throughout the entire process.

[0056] Example 3: Overall System Module Interaction Flow This embodiment describes the collaborative working process between the modules of the present invention.

[0057] The optimization system of this invention includes six core modules: instance pool management module, shared memory bridging module, dynamic routing decision module, collaborative GC scheduling module, PL / Java function execution proxy module, and monitoring and diagnosis module.

[0058] When a user executes SELECT my_java_func(...), the execution flow is as follows: The PL / Java function execution proxy module intercepts the call and obtains the function signature and parameter list.

[0059] The dynamic routing decision module queries the historical call profile of the function, runs the EWMA-Markov prediction model, and outputs the type of the routing target instance, such as: cold / warm / hot, based on the current data volume and adaptive threshold.

[0060] The instance pool management module allocates a healthy JVM instance from the instance pool in the corresponding state and returns the connection handle of that instance.

[0061] The shared memory bridging module allocates or reuses existing dynamic shared memory segments, writes parameter data to the data payload area according to the alignment protocol, and sets the status flag in the metadata area to "ready".

[0062] The JVM reads parameters directly through a zero-copy channel, executes the corresponding Java method, and writes the return result back to the shared memory data payload area.

[0063] The PG reads the result and returns it to the client.

[0064] After this call is completed, the instance pool management module releases the instance connection and updates the load count; the monitoring and diagnostic module records the latency, data volume and GC events of this call.

[0065] If a transaction is committed and enters an idle window, the GC scheduling module will trigger the JVM to perform concurrent garbage collection.

[0066] Through the interaction of the above modules, this invention achieves a low-latency, high-throughput, resource-adaptive, and transaction-safe PL / Java execution environment.

[0067] In summary, by utilizing the technical solutions described above in this invention, and by constructing a three-tiered JVM instance pool (cold, warm, and hot states) and implementing adaptive routing, the initial call latency is significantly reduced, and memory resources can be elastically scaled on demand. Furthermore, by establishing a zero-copy data exchange channel based on shared memory and off-heap memory mapping, along with a memory barrier synchronization mechanism, cross-language data interaction bypasses JNI serialization overhead, resulting in a significant increase in throughput. Finally, through transaction boundary-aware collaborative GC scheduling, concurrent garbage collection is performed using a low-activity window in the database, keeping garbage collection pause times below milliseconds. This completely eliminates the technical defects of traditional PL / Java, such as high cold-start latency, large data copy overhead, low resource utilization, and GC blocking database transactions.

[0068] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A PostgreSQL PLJava optimization method based on adaptive JVM warm-up and zero-copy data exchange, characterized in that, Includes the following steps: Construct a load-aware, tiered JVM instance pool: Based on function call frequency characteristics and warm-up status, maintain a three-level JVM instance pool containing cold instances, warm instances, and hot instances. The cold instances run based on pre-compiled native executables, the warm instances run in a lightweight JVM with pre-loaded core class libraries, and the hot instances run in a standard JVM with just-in-time compilation optimization enabled. Establish a zero-copy data exchange channel: allocate a PostgreSQL dynamic shared memory segment, map the shared memory segment to the JVM off-heap memory, define a cross-language memory layout alignment protocol including the metadata area and the data payload area, and establish a synchronization mechanism based on memory barriers, so that the PostgreSQL backend process and the JVM can directly read and write the shared memory segment through the physical address space to exchange function parameters and execution results; Deploy an adaptive dynamic call routing engine: collect the call profile of the target Java function in real time, predict the load probability of the next cycle based on the call profile, and dynamically route the current call to the corresponding instance in the hierarchical JVM instance pool for execution according to the adaptive calibration threshold and prediction results; Implement coordinated GC scheduling: Register PostgreSQL transaction manager callback hooks, listen for database transaction boundary events and background worker idle signals, identify low-activity time windows, and trigger the current JVM instance to perform concurrent garbage collection within the time window.

2. The PostgreSQL PLJava optimization method based on adaptive JVM warm-up and zero-copy data exchange according to claim 1, characterized in that, The tiered JVM instance pool is maintained by the instance pool manager, which manages the health status, load count, and circuit breaker policy of each instance. When an instance experiences consecutive timeouts or memory overflows, the instance pool manager automatically triggers isolation and destruction and rebuilds the instance from the backup image pool.

3. The PostgreSQL PLJava optimization method based on adaptive JVM warm-up and zero-copy data exchange according to claim 1, characterized in that, The adaptive dynamic call routing engine uses an exponentially weighted moving average combined with a first-order Markov state transition model to predict the load probability of the next cycle, and determines the adaptive calibration threshold used for routing decisions based on offline calibration of the cluster centers of historical call distribution and online fine-tuning during runtime.

4. The PostgreSQL PLJava optimization method based on adaptive JVM warm-up and zero-copy data exchange according to claim 1, characterized in that, The dynamic call routing engine also performs a context atomic migration mechanism: before switching execution environments, a snapshot containing the transaction isolation level, current transaction identifier, cursor state, and SPI call stack frame is created through the PostgreSQL memory context manager; After the new instance takes over, the snapshot is atomically restored via a two-phase commit protocol; if the restoration fails, a rollback compensation is triggered and the instance is downgraded to the previous stable instance.

5. The PostgreSQL PLJava optimization method based on adaptive JVM warm-up and zero-copy data exchange according to claim 1, characterized in that, In the collaborative GC scheduling step, the logic for identifying low-activity time windows is as follows: the time difference between the end time of a database transaction and the start time of the last query is greater than a preset threshold; when a low-activity time window is determined to exist, ZGC or Shenandoah concurrent mark-and-sweep is triggered; when cross-language JNI reference chain leakage or memory fragmentation rate exceeds a preset threshold is detected, the system automatically downgrades to the G1 generational garbage collection strategy and generates a heap dump snapshot.

6. The PostgreSQL PLJava optimization method based on adaptive JVM warm-up and zero-copy data exchange according to claim 1, characterized in that, In the zero-copy data exchange channel, after the PostgreSQL backend process completes the data writing, it refreshes the cache consistency through a memory barrier. The JVM side polls the metadata area status flags to confirm that it is ready and then obtains the data through the memory barrier, thereby bypassing the JNI field-level copy path.

7. A PostgreSQL PL Java optimization system based on adaptive JVM warm-up and zero-copy data exchange, characterized in that, include: The instance pool management module is configured to maintain the JVM process lifecycle using a three-level state machine: cold, warm, and hot. The shared memory bridging module is configured to allocate PostgreSQL dynamic shared memory segments and perform cross-language memory layout alignment and address mapping. The dynamic routing decision module is configured to collect call profiles in real time, run load prediction algorithms, and perform instance switching based on adaptive thresholds. The collaborative GC scheduling module is configured to register transaction callback hooks, identify low-activity time windows, and trigger concurrent garbage collection. The PL / Java function execution proxy module is configured to receive routing instructions and bind zero-copy channels to read and write function parameters and results. The monitoring and diagnostic module is configured to collect performance metrics and output optimization suggestions. Each module interacts with shared memory through the PostgreSQL internal message bus and is dynamically loaded into the database process space as a shared library.

8. The PostgreSQL PLJava optimization system based on adaptive JVM warm-up and zero-copy data exchange according to claim 7, characterized in that, The instance pool management module is also used to perform health checks, load statistics, and fault reconstruction; when an instance times out or experiences memory overflow, it is automatically triggered to be isolated and destroyed and rebuilt from the backup image pool; the shared memory bridging module implements a zero-copy read / write interface by defining a memory layout alignment protocol that includes a metadata area and a data payload area, and establishing a synchronization mechanism based on memory barriers.

9. The PostgreSQL PLJava optimization system based on adaptive JVM warm-up and zero-copy data exchange according to claim 7, characterized in that, The dynamic routing decision module uses an exponentially weighted moving average combined with a first-order Markov state transition model to predict load probability, and determines the adaptive calibration threshold based on offline calibration of the cluster centers of historical call distribution and online fine-tuning during runtime; when performing instance switching, it ensures the ACID properties of the database by saving and atomically restoring SPI context snapshots.

10. The PostgreSQL PLJava optimization system based on adaptive JVM warm-up and zero-copy data exchange according to claim 7, characterized in that, The collaborative GC scheduling module identifies low-activity time windows based on the time difference between the end of a database transaction and the start of the last query being greater than a preset threshold. When a low-activity time window is identified, ZGC or Shenandoah concurrent mark-and-sweep is triggered. When cross-language JNI reference chain leakage or memory fragmentation rate exceeds a preset threshold, the module automatically downgrades to the G1 generational garbage collection strategy and generates a heap dump snapshot.

Images