Memory database starting method, system and equipment and medium

Abstract

The invention provides a memory database starting method, system and device and a medium, and belongs to the technical field of computers. The method comprises the following steps: detecting a current project scale and a performance index of an original server, and determining whether to trigger a memory processing mechanism; if a memory processing mechanism is triggered, selecting a memory server according to high-performance computing and expansibility requirements, accessing an original server by utilizing a hot plug technology, configuring a memory database and initializing a memory computing cluster; configuring a to-be-synchronized database table by utilizing a metadata management function, loading original data to a memory database through a full-amount synchronization and incremental updating mechanism, and starting real-time data synchronization; starting a memory computing cluster, automatically detecting a memory processing state on an original server when a service task is triggered, and routing a large quantity of data tasks meeting conditions to a memory server to execute parallel computing. According to the method, the memory can be dynamically started for data processing, the operation efficiency is improved, and the user experience is improved.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and more particularly relates to a method, system, device and medium for enabling a memory database. Background Art

[0002] As enterprises deepen their digital transformation, the data scale and computing demands of business systems continue to grow. Traditional disk-based database architectures are gradually showing performance shortcomings when handling large-scale data tasks. For example, in scenarios involving massive amounts of data, such as budget management and cost accounting, traditional databases often suffer from issues such as limited storage media read and write speeds and irrational allocation of computing resources, leading to inefficient task execution. This can even lead to excessive server load and significantly increased response latency, making it difficult to meet real-time business processing requirements.

[0003] In existing technologies, although in-memory database technology can improve computing efficiency by loading data into memory, it still has the following technical limitations in practical applications: 1. Inadequate adaptability of trigger conditions to business scenarios: Existing solutions often rely on single-dimensional metrics to determine whether to enable in-memory databases. This simplistic trigger logic can lead to a mismatch between resource allocation and business needs, impacting overall system resource utilization.

[0004] 2. Low automation in the deployment and configuration process: Because hardware selection, cluster initialization, and database connection configuration for in-memory servers rely on manual operations, there is a lack of standardized, automated deployment toolchain support. Manual configuration is not only time-consuming and labor-intensive, but can also lead to inconsistent cluster parameters and abnormal service startup due to operational errors.

[0005] 3. The timeliness and reliability of data synchronization need to be improved: Traditional data synchronization solutions often use a periodic full-data synchronization mode, which cannot capture changes to source data in real time. This results in a time difference between the data in the in-memory database and the original data, which may cause deviations in calculation results in high-frequency business scenarios.

[0006] 4. Task scheduling strategies lack dynamic flexibility: Existing systems often use static rules to route business tasks, without dynamically adjusting tasks based on data volume, computational complexity, and current server load. Computationally intensive tasks are still handled by traditional servers, leading to an imbalance in computing resource allocation and difficulty optimizing overall processing efficiency.

[0007] In summary, existing technologies have yet to form a complete technical system that integrates dynamic triggering, automated deployment, real-time data synchronization, and intelligent task scheduling in the process of enabling in-memory databases. This makes it difficult for the system to balance performance optimization and resource utilization when dealing with large data volumes and high-concurrency business scenarios. Summary of the Invention

[0008] In view of the above problems, the purpose of the present invention is to provide a method, system, device and medium for enabling a memory database, which intelligently triggers a memory acceleration mechanism by dynamically monitoring the project and server status, and realizes exponential improvement of computing performance, dynamic unloading of server load and millisecond-level optimization of business response latency through elastic hardware deployment, full-link data consistency guarantee and intelligent task shunting technology, comprehensively enhancing the high-concurrency processing ability and data reliability of the system.

[0009] To achieve the above object, the present invention is realized through the following technical solutions: In a first aspect, an embodiment of the present application provides a method for enabling a memory database, including: Detect the current project scale and the performance metrics of the original server, and determine whether to trigger the memory processing mechanism; If the memory processing mechanism is triggered, select a memory server according to the high-performance computing and scalability requirements, connect it to the original server using hot-plug technology, configure the memory database and initialize the memory computing cluster; Configure the database tables to be synchronized using the metadata management function, load the original data into the memory database through full synchronization and incremental update mechanisms, and enable real-time data synchronization; Start the memory computing cluster. When a business task is triggered, automatically detect the memory processing status on the original server, and route eligible large-volume data tasks to the memory server for parallel computing.

[0010] In an optional embodiment, the detecting the current project scale and the performance metrics of the original server includes: Extract the number of project organizations and the number of rows in the budget form from the business system metadata; Use system and application monitoring tools to obtain the current CPU utilization rate, disk I / O wait queue length, and task response time of the original server.

[0011] In an optional embodiment, the determining whether to trigger the memory processing mechanism includes: If the number of project organizations, the number of rows in the budget form, the CPU utilization rate, the disk I / O wait queue length, and the task response time are all higher than the corresponding preset thresholds, then trigger the memory processing mechanism.

[0012] In an optional embodiment, the selecting a memory server according to the high-performance computing and scalability requirements includes: According to the requirements of multi-core parallel computing, large memory storage and high-speed I / O transmission, select a server with qualified hardware configuration and support for expansion as the memory server; The hardware configuration requirements of the memory server include: The number of cores is greater than or equal to 28, the memory capacity is greater than or equal to 512G, memory interleaving technology is adopted, a solid-state drive is deployed as the cache layer, and a 10Gbps network card is configured.

[0013] In an optional implementation, configuring the in-memory database and initializing the in-memory computing cluster includes: Installing the in-memory database; Through the built-in in-memory computing module GUI, inputting the connection string, authentication information of the original database, and the IP of the cluster nodes of the in-memory database; Using scripts to batch deploy the server operating system and in-memory database software, and initialize the in-memory computing cluster.

[0014] In an optional implementation, using the metadata management function to configure the database tables to be synchronized, loading the original data into the in-memory database through the full synchronization and incremental update mechanism, and enabling real-time data synchronization, includes: According to the database tables to be synchronized, defining the table structures to be migrated in the data synchronization module, specifying the primary key, index, and synchronization direction; filtering the table names by supporting regular expressions, and batch-selecting related tables for synchronization; Batch loading the original table data into the in-memory database through an ETL tool; Based on database triggers or change data capture technology, real-time monitoring of the change events of the original table; if a change occurs, pushing the incremental data to the in-memory database through a message queue; Enabling real-time data synchronization, and recording transaction logs during the synchronization process; after the synchronization is completed, performing a verification task to ensure the consistency of data synchronization by comparing the hash values of the original table data and the in-memory data.

[0015] In an optional implementation, starting the in-memory computing cluster, when a business task is triggered, automatically detecting the memory processing status on the original server, and routing eligible large-volume data tasks to the memory server for parallel computing, includes: Remotely starting the database service and parallel computing process on the memory server through an interactive interface; When a business task is initiated at the task layer, detecting whether the in-memory computing cluster has been enabled; If it has been enabled, detecting the type of the business task; If the business task is a large-volume data task, encapsulating the business task as a query statement and sending it to the memory server; The memory server adopts a massively parallel processing architecture, splitting the business task into subtasks and distributing them to different CPU cores, and merging the calculation results after independent calculation by each CPU core; Return the calculation result to the original server through the network interface.

[0016] In a second aspect, an embodiment of the present application further provides a system for enabling an in-memory database, including: A performance detection module, configured to detect the current project scale and the performance metrics of the original server, and determine whether to trigger the in-memory processing mechanism; An in-memory server initialization module, configured to, if the in-memory processing mechanism is triggered, select an in-memory server according to the high-performance computing and scalability requirements, connect it to the original server using the hot-plug technology, configure the in-memory database, and initialize the in-memory computing cluster; A data synchronization and in-memory computing preparation module, configured to configure the database tables to be synchronized using the metadata management function, load the original data into the in-memory database through the full synchronization and incremental update mechanisms, and enable real-time data synchronization; An in-memory computing execution module, configured to start the in-memory computing cluster, automatically detect the in-memory processing status on the original server when a business task is triggered, and route eligible large-volume data tasks to the in-memory server for parallel computing.

[0017] In a third aspect, an embodiment of the present application further provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the steps of the method for enabling an in-memory database as described in any one of the above.

[0018] In a fourth aspect, an embodiment of the present application further provides a storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of the method for enabling an in-memory database as described in any one of the above.

[0019] From the above technical solutions, it can be seen that the present invention has the following advantages: In the method for enabling an in-memory database provided by the present application, the in-memory processing mechanism is intelligently triggered by detecting the project scale and the performance metrics of the original server, the in-memory server is accurately selected and connected according to the high-performance computing and scalability requirements, the data is efficiently loaded and real-time synchronization is ensured by means of metadata management and full-incremental synchronization mechanisms, the in-memory processing status is automatically detected when a business task is triggered, and large-volume data tasks are routed to the in-memory server for parallel computing. This can effectively improve the data processing efficiency, reduce the load on the original server, enhance the system response speed and concurrent processing ability, and at the same time ensure the consistency of data synchronization, providing a more efficient and stable operating environment for the business system.

[0020] The present application dynamically determines whether to enable the in-memory processing mechanism by detecting the project scale and the performance metrics of the original server, and selects a suitable in-memory server according to the high-performance computing and scalability requirements, realizing the rational utilization of resources and avoiding resource waste.

[0021] This application uses the metadata management function to accurately configure the database tables to be synchronized, and combines the full - volume synchronization and incremental update mechanisms to quickly and accurately load the original data into the in - memory database, and starts real - time synchronization to ensure data timeliness and consistency.

[0022] This application starts the in - memory computing cluster, routes a large number of data tasks to the in - memory server for parallel computing, and with the help of the large - scale parallel processing architecture, gives full play to the multi - core parallel computing ability, significantly improving the data processing speed and efficiency.

[0023] This application transfers a large number of data tasks to the in - memory server for processing, effectively reducing the computing and storage pressure on the original server, avoiding performance degradation or system failures caused by high loads, and ensuring the stable operation of the original server.

[0024] This application records transaction logs during the data synchronization process, and performs verification tasks after synchronization is completed. By comparing hash values, it ensures data synchronization consistency and provides accurate and reliable data support for the business system. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the present invention, the drawings required for description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0026] Figure 1 It is a schematic flowchart of the method for enabling an in - memory database provided by this application.

[0027] Figure 2 It is a schematic structural diagram of the system for enabling an in - memory database provided by this application.

[0028] Figure 3 It is a schematic structural diagram of the electronic device provided by this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0029] In the following, the specific steps of the method for enabling an in - memory database will be described in detail, and various embodiments of the present disclosure will be described more comprehensively. The present disclosure can have various embodiments, and adjustments and changes can be made therein. However, it should be understood that there is no intention to limit the various embodiments of the present disclosure to the specific embodiments disclosed herein, but the present disclosure should be understood to cover all adjustments, equivalents, and / or alternative solutions that fall within the spirit and scope of the various embodiments of the present disclosure.

[0030] In the following, the term "comprising" or "may comprise" that can be used in various embodiments of the present disclosure indicates the presence of the disclosed functions, operations, or elements, and does not limit the addition of one or more functions, operations, or elements. Further, as used in various embodiments of the present disclosure, the terms "comprising", "having" and their cognates are only intended to indicate a specific feature, number, step, operation, element, component, or combination of the foregoing items, and should not be construed as precluding the existence or possibility of addition of one or more other features, numbers, steps, operations, elements, components, or combinations of the foregoing items first.

[0031] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0032] Please refer to Figure 1 The following is a flowchart of a method for enabling an in-memory database in a specific embodiment. The method includes: S1: Detect the current project scale and the performance metrics of the original server, and determine whether to trigger the in-memory processing mechanism.

[0033] The purpose of this step is to check the project volume and determine whether to enable in-memory processing: Judging based on the project performance. If the project organization volume exceeds ten thousand, the number of rows in the budget form organization exceeds ten thousand, and the current calculation response time exceeds 10 minutes, and at this time, through the performance detection tool, it is detected that the server CPU is in a high-load state or there are a large number of disk waiting queues frequently in disk I / O reads and writes, it means that there is a bottleneck in processing data of this magnitude with the current server, and it is necessary to enable in-memory processing means to ensure the stable and efficient operation of the system.

[0034] Based on the above purpose, in the specific implementation, first extract the project organization quantity and the number of rows in the budget form from the business system metadata; and use the system and application program monitoring tool to obtain the current CPU utilization rate, disk I / O waiting queue length, and task response time of the original server.

[0035] If the project organization quantity, the number of rows in the budget form, the CPU utilization rate, the disk I / O waiting queue length, and the task response time are all higher than the corresponding preset thresholds, then trigger the in-memory processing mechanism, otherwise do not trigger.

[0036] Exemplarily, by monitoring the project scale (organization size, number of form rows) and server performance metrics (CPU load, disk I / O wait queue, response time), using a performance detection tool to automatically determine whether to trigger the memory processing mechanism, and adopting threshold matching and dynamic analysis techniques to ensure the accuracy of decision-making.

[0037] Specifically, extract the project organization quantity by querying the number of records in the organization_table from the business system metadata through the database, and parse the number of rows in the Excel / database table budget_form_table to obtain the number of budget form rows; among them, the thresholds for these two types of data are organization size > 10,000 and number of form rows > 10,000.

[0038] On the other hand, use Prometheus + Grafana or Zabbix to monitor the CPU utilization rate of the original server (threshold > 80%), the length of the disk I / O wait queue (threshold > 5), and the task response time (threshold > 10 minutes), and collect data every 5 seconds.

[0039] When each piece of data exceeds the corresponding threshold, it indicates that the project scale exceeds the threshold and the server performance metrics reach the bottleneck, generating an "enable memory processing" instruction and notifying the memory calculation module through the Kafka message queue.

[0040] S2: If the memory processing mechanism is triggered, select a memory server according to the high-performance computing and scalability requirements, connect it to the original server using hot-swapping technology, configure the memory database, and initialize the memory calculation cluster.

[0041] In the specific implementation, if the memory processing mechanism is triggered, first select a server with qualified hardware configuration and support for expansion as the memory server according to the requirements of multi-core parallel computing, large memory storage, and high-speed I / O transmission.

[0042] In this step, the selection of the memory server should follow the following principles: 1. In terms of performance: Processing speed: It is necessary to select a server with a large number of CPU cores and a high main frequency. A multi-core CPU can better support multi-node and multi-task parallel computing, enabling each task to run on different cores simultaneously, giving full play to the advantages of parallel computing, and thus accelerating the overall processing speed.

[0043] Memory capacity: The memory database stores data in memory, which requires the server to have a sufficiently large memory capacity to ensure that a large amount of data can be stored, avoiding frequent memory swapping due to insufficient memory and affecting system performance.

[0044] I / O Performance: Although using an in-memory database reduces the I / O consumption of data access, the I / O performance of the server is still crucial. High-performance I / O devices can accelerate the data transfer speed inside and outside the server, ensuring that data interaction between nodes can proceed quickly. Therefore, it is recommended to select a server equipped with high-speed solid-state drives (SSDs) and high-speed network interfaces.

[0045] 2. In terms of scalability: Hardware Expansion: As the business develops, the amount of data and the number of parallel tasks may continue to increase. Therefore, the server needs to have good hardware scalability and be able to easily add hardware resources such as CPUs, memory, and hard disks.

[0046] Software Expansion: The server's operating system and software platform should have good scalability and be able to support the upgrade and expansion of in-memory databases and multi-node parallel computing technologies. At the same time, consider the server's support for clustering technology so that multiple servers can be formed into a cluster when needed to further enhance the system's processing power.

[0047] Based on the above principles, the hardware configuration requirements for the in-memory server include: CPU: Select the AMD EPYC 7003 series (≥32 cores) or the Intel Xeon Platinum 8300 series (≥28 cores), which support hyper-threading technology to meet multi-node parallel computing.

[0048] Memory: Configure DDR4-3200 RDIMM memory (≥512GB) and use Memory Interleaving technology to increase bandwidth.

[0049] Storage: Deploy NVMe SSDs (single-disk capacity ≥4TB, sequential read / write ≥7000MB / s) as the cache layer, and pair with 10Gbps network cards (such as Intel X550-T2) to reduce network latency.

[0050] Then, use hot-swap technology to connect the in-memory server to the original server, configure the in-memory database for the in-memory server, and initialize the in-memory computing cluster. The specific process is as follows: Install the in-memory database; Through the in-memory computing module GUI built into the product, input the connection string, authentication information of the original database (such as Oracle), and the cluster node IP / port of the in-memory database (such as SAP HANA); Use Terraform scripts to batch deploy the server operating system (such as CentOS 8), in-memory database software, and initialize the in-memory computing cluster (master node + slave node architecture, supporting automatic failover).

[0051] In this step, by introducing an in-memory database in the server, the I / O consumption during data access is significantly reduced. An multi-node multi-task parallel computing mechanism is enabled on the in-memory server side, and the overall time consumption is greatly reduced by means of parallel processing.

[0052] S3: Configure the database tables to be synchronized by using the metadata management function, load the original data into the in-memory database through the full synchronization and incremental update mechanisms, and enable real-time data synchronization.

[0053] The purpose of this step is the metadata management function of the product. Only the information of the database tables to be synchronized needs to be configured, and then with one key operation, data synchronization can be achieved. When the data synchronization is successfully completed and the in-memory computing switch is turned on, the system will quickly start an efficient computing process. If the in-memory processing function has been enabled, when there are addition, deletion, or modification operations on the basic data, the system will automatically push the updated data to the in-memory database to ensure data real-time and consistency.

[0054] For the above purposes, in the specific implementation, first, according to the database tables to be synchronized, define the table structures to be migrated in the data synchronization module, specify the primary key, index, and synchronization direction; support filtering table names through regular expressions, and batch select associated tables for synchronization.

[0055] Then, batch load the original table data into the in-memory database through an ETL tool.

[0056] At this time, based on database triggers or change data capture technology, real-time monitor the change events of the original table; if there is a change, push the incremental data to the in-memory database through a message queue; Finally, enable real-time data synchronization, and record transaction logs during the synchronization process; after synchronization is completed, execute a verification task to ensure the consistency of data synchronization by comparing the hash values of the original table data and the in-memory data.

[0057] For example, first define the tables to be migrated in the data synchronization module (such as budget_data, organization_hierarchy), specify the primary key (such as id) and index fields, and support filtering associated tables through regular expressions (such as budget_*.).

[0058] For the first synchronization, use Apache NiFi to perform full-scale ETL, read the original database in parallel through multiple threads, write to the in-memory database in batches (batch size = 100,000 records per time), and record the synchronization timestamp last_full_sync_time.

[0059] When performing incremental synchronization, CDC is implemented based on Debezium + Kafka. The INSERT / UPDATE / DELETE events of the original table are listened for, serialized in Avro format, and then pushed to the in-memory database. The timestamp last_incremental_sync_time is updated.

[0060] Transaction logs are recorded during the synchronization process. After synchronization is completed, the COUNT(*) and CHECKSUM values of the original table and the in-memory table are compared. If they are inconsistent, a retry is triggered (up to 3 times), and an alert is sent to the operation and maintenance personnel in case of failure.

[0061] After the in-memory computing switch is turned on, the system automatically creates in-memory table partitions (such as hash partitioning by organization_id), and preloads frequently queried data into the CPU cache.

[0062] S4: Start the in-memory computing cluster. When a business task is triggered, automatically detect the in-memory processing status on the original server, and route eligible large-volume data tasks to the in-memory server for parallel computing.

[0063] In the specific implementation, first remotely start the database service and parallel computing process on the in-memory server through the interactive interface; when a business task is initiated at the task layer, detect whether the in-memory computing cluster has been enabled; if it has been enabled, detect the type of the business task.

[0064] If the business task is a large-volume data task, encapsulate the business task as a query statement and send it to the in-memory server; in the in-memory server, adopt a massively parallel processing architecture, split the business task into subtasks and allocate them to different CPU cores, and merge the calculation results after independent calculation by each CPU core.

[0065] Finally, return the calculation result to the original server through the network interface.

[0066] Exemplarily, when a business task is started, the system automatically detects the in-memory processing status, routes eligible large-volume data tasks to the in-memory server for parallel computing, and lightweight tasks are retained for processing on the original server, adopting dynamic data source and load balancing technologies to achieve efficient resource allocation.

[0067] Specifically: First, remotely start the database service and parallel computing process on the in-memory server through SSH or API interface. Dynamically allocate computing nodes on the in-memory server (such as dividing task partitions by the number of CPU cores), and enable load balancing algorithms (such as round-robin, least connections) to allocate tasks.

[0068] Obtain the current in-memory processing status (enabled / disabled) through the performance detection module.

[0069] If enabled, determine the task type: If the task type is a large-volume data task, such as budget summary calculation or cost batch verification, it is encapsulated as SQL / NoSQL statements and sent to the in-memory database for execution.

[0070] If the task type is a lightweight task, such as single-data query, continue to process on the original server.

[0071] The task result is returned to the application layer through a high-speed network, and an asynchronous callback mechanism is supported to handle long-time-consuming tasks.

[0072] In addition, in this step, Spring Boot Actuator is used to monitor the status of the in-memory computing cluster, and task routing is implemented through the dynamic data source of MyBatis.

[0073] In this embodiment, by dynamically detecting the business load and server performance metrics, the in-memory processing mechanism is intelligently triggered. Combining high-performance hardware configuration and hot-plug technology, an elastic computing cluster is constructed to achieve real-time synchronization and consistency verification of full and incremental data. And when task routing, large-volume calculations are automatically diverted to the in-memory server for parallel processing, thereby significantly improving the data processing efficiency in high-load scenarios, reducing business latency. At the same time, the system elasticity and reliability are enhanced through modular expansion and automated operation and maintenance, effectively balancing resource costs and performance requirements.

[0074] Such as Figure 2 As shown, the following is an embodiment of the in-memory database enabling system provided by the present disclosure. This system and the in-memory database enabling methods of the above embodiments belong to the same inventive concept. For the details not described in detail in the embodiment of the in-memory database enabling system, reference can be made to the embodiments of the in-memory database enabling methods above.

[0075] An in-memory database enabling system includes: a performance detection module, an in-memory server initialization module, a data synchronization and in-memory calculation preparation module, and an in-memory calculation execution module.

[0076] The performance detection module is used to detect the current project scale and the performance metrics of the original server, and determine whether to trigger the in-memory processing mechanism.

[0077] The in-memory server initialization module is used to, if the in-memory processing mechanism is triggered, select an in-memory server according to the high-performance computing and scalability requirements, connect it to the original server using hot-plug technology, configure the in-memory database, and initialize the in-memory computing cluster.

[0078] A data synchronization and in-memory computing preparation module is used to configure the database tables to be synchronized by using the metadata management function, load the original data into the in-memory database through the full synchronization and incremental update mechanisms, and enable real-time data synchronization.

[0079] An in-memory computing execution module is used to start the in-memory computing cluster. When a business task is triggered, it automatically detects the in-memory processing status on the original server and routes eligible large-scale data tasks to the in-memory server for parallel computing.

[0080] The in-memory database enabling system provided in this embodiment intelligently triggers the in-memory processing mechanism by detecting the project scale and the performance metrics of the original server, selects a suitable in-memory server according to the high-performance computing and scalability requirements and connects it to the original server, efficiently loads and synchronizes data to the in-memory database by using the metadata management, full synchronization and incremental update mechanisms, and finally automatically routes large-scale data tasks to the in-memory server for parallel computing when a business task is triggered, effectively improving the data processing efficiency and system performance, ensuring the accuracy and consistency of data synchronization, enhancing the system's response speed and computing power, and meeting the requirements for high-performance computing in complex business scenarios.

[0081] Figure 3 A schematic diagram of the hardware structure of an electronic device for implementing various embodiments of the present invention.

[0082] The method for enabling an in-memory database provided in the embodiments of the present application can be applied to an electronic device. Those skilled in the art can understand that the structure of the electronic device involved in the embodiments of the present invention does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine some components, or have different component arrangements. In the embodiments of the present invention, the electronic device includes, but is not limited to, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the embodiments of the present application described herein and / or claimed.

[0083] The electronic device may include a processor, an external memory interface, an internal memory, a universal serial bus (USB) interface, a charging management module, a power management module, a battery, a wireless communication module, an audio module, a speaker, a microphone, a sensor module, a key, a camera, a display screen, and a SIM card interface, etc.

[0084] The processor may include one or more processing units. For example, the processor may include a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0085] Among them, the processor may be the nerve center and command center of the electronic device. The controller may generate operation control signals according to the instruction operation code and timing signals to complete the control of fetching and executing instructions.

[0086] A memory may also be provided in the processor for storing instructions and data. In some embodiments, the memory in the processor is a cache memory. This memory can store the instructions or data that the processor has just used or recycled. If the processor needs to use the instruction or data again, it can directly call it from this memory. This avoids repeated accesses, reduces the waiting time of the processor, and thus improves the system efficiency.

[0087] The external memory interface can be used to connect to an external memory card, such as a MicroSD card, to expand the storage capacity of the electronic device. The external memory card communicates with the processor through the external memory interface to achieve the data storage function. For example, files such as music and videos are saved in the external memory card.

[0088] The internal memory can be used to store computer-executable program code, and the computer-executable program code includes instructions. The processor executes various functional applications and data processing of the electronic device by running the instructions stored in the internal memory. The internal memory may include a program storage area and a data storage area. The internal memory may include a high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0089] The wireless communication function of the electronic device can be implemented through an antenna, a wireless communication module, a modem processor, a baseband processor, etc.

[0090] The wireless communication module can provide solutions for wireless communications applied to electronic devices, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite systems (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), etc.

[0091] The electronic device can implement audio functions through an audio module, speaker, receiver, microphone, headphone jack, application processor, etc.

[0092] The electronic device can implement a shooting function through an ISP, camera, video codec, GPU, display screen, application processor, etc.

[0093] The electronic device can implement a display function through a GPU, display screen, application processor, etc.

[0094] The GPU is a microprocessor for image processing, connecting the display screen and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor may include one or more GPUs that execute program instructions to generate or change display information.

[0095] The display screen is used to display images, videos, etc. The display screen includes a display panel.

[0096] The above-mentioned electronic device realizes the enabling method of the in-memory database of the present application by detecting the project scale and the performance indicators of the original server to trigger the in-memory processing mechanism, selecting and accessing the in-memory server on demand, configuring the in-memory database and initializing the computing cluster, realizing efficient data loading and real-time synchronization through the metadata management and synchronization mechanism, and automatically routing a large amount of data to the in-memory server for parallel computing when a business task is triggered, achieving the beneficial effects of improving data processing efficiency, enhancing system response speed and computing power, and ensuring data synchronization consistency.

[0097] In the storage medium provided by the present application, there is a program product capable of realizing the enabling method of the in-memory database.

[0098] The enabling method of the in-memory database includes: Detecting the current project scale and the performance indicators of the original server to determine whether to trigger the in-memory processing mechanism; If the memory processing mechanism is triggered, select a memory server according to the high-performance computing and scalability requirements, connect it to the original server using hot-swapping technology, configure a memory database, and initialize the memory computing cluster; Configure the database tables to be synchronized using the metadata management function, load the original data into the memory database through the full synchronization and incremental update mechanisms, and enable real-time data synchronization; Start the memory computing cluster. When a business task is triggered, automatically detect the memory processing status on the original server, and route eligible large-volume data tasks to the memory server for parallel computing. In some possible implementation manners, the method for enabling the memory database of the present disclosure may be implemented in the form of a program product, which includes program code. When the program product runs on a terminal device, the program code is used to cause the terminal device to execute the steps according to various exemplary embodiments of the present disclosure described in the above "Exemplary Method" section of this specification.

[0099] The storage medium of the present disclosure may adopt any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the readable storage medium include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0100] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for enabling an in-memory database, characterized in that Including: Detect the current project scale and the performance metrics of the original server, and determine whether to trigger the memory processing mechanism; If the memory processing mechanism is triggered, select a memory server according to the high-performance computing and scalability requirements, connect it to the original server using hot-swap technology, configure the memory database, and initialize the memory computing cluster; Configure the database tables to be synchronized using the metadata management function, load the original data into the memory database through full synchronization and incremental update mechanisms, and enable real-time data synchronization; Start the memory computing cluster. When a business task is triggered, automatically detect the memory processing status on the original server, and route eligible large-volume data tasks to the memory server for parallel computing.

2. The enabling method of the in-memory database according to claim 1, wherein The detection of the current project scale and the performance metrics of the original server includes: Extract the number of project organizations and the number of rows in the budget form from the business system metadata; Use system and application monitoring tools to obtain the current CPU utilization, disk I / O wait queue length, and task response time of the original server.

3. The enabling method of the in-memory database according to claim 2, wherein The determination of whether to trigger the memory processing mechanism includes: If the number of project organizations, the number of rows in the budget form, the CPU utilization, the disk I / O wait queue length, and the task response time are all higher than the corresponding preset thresholds, trigger the memory processing mechanism.

4. The enabling method of the in-memory database according to claim 3, characterized in that, The selection of the memory server according to the high-performance computing and scalability requirements includes: Select a server with compliant hardware configuration and support for expansion as the memory server according to the requirements of multi-core parallel computing, large memory storage, and high-speed I / O transmission; The hardware configuration requirements of the memory server include: The number of cores is greater than or equal to 28, the memory capacity is greater than or equal to 512G and the memory interleaving technology is adopted, a solid-state drive is deployed as the cache layer, and a 10Gbps network card is configured.

5. The method for enabling a memory database according to claim 4, wherein The configuration of the memory database and the initialization of the memory computing cluster include: Install the memory database; Through the built-in memory computing module GUI, input the connection string, authentication information of the original database, and the cluster node IP of the memory database; Adopt scripts to batch deploy the server operating system and memory database software, and initialize the memory computing cluster.

6. The enabling method of the in-memory database according to claim 5, wherein The use of the metadata management function to configure the database tables to be synchronized, load the original data into the memory database through full synchronization and incremental update mechanisms, and enable real-time data synchronization includes: According to the database tables to be synchronized, define the table structure to be migrated in the data synchronization module, specify the primary key, index, and synchronization direction; support regular expression filtering of table names, and batch select associated tables for synchronization; Batch load the original table data into the memory database through ETL tools; Based on database triggers or change data capture technology, monitor the original table change events in real time; if a change occurs, push the incremental data to the memory database through a message queue; Enable real-time data synchronization, and record transaction logs during the synchronization process; after synchronization is completed, execute a verification task to ensure data synchronization consistency by comparing the hash values of the original table data and the memory data.

7. The enabling method of the in-memory database according to claim 6, characterized in that, When the in-memory computing cluster is started, when a business task is triggered, the memory processing status is automatically detected on the original server, and a large number of eligible data tasks are routed to the memory server for parallel computing, including: Remotely start the database service and parallel computing process on the memory server through the interaction interface; When a business task is initiated at the task layer, detect whether the in-memory computing cluster has been enabled; If it has been enabled, detect the type of the business task; If the business task is a large number of data tasks, encapsulate the business task as a query statement and send it to the memory server; The memory server adopts a massively parallel processing architecture, splits the business task into subtasks and distributes them to different CPU cores, and combines the calculation results after independent calculation by each CPU core; Return the calculation result to the original server through the network interface.

8. An enabling system for an in-memory database, characterized in that, The system adopts the method for enabling the in-memory database according to any one of claims 1 to 7; The system includes: A performance detection module for detecting the current project scale and the performance metrics of the original server to determine whether to trigger the memory processing mechanism; An in-memory server initialization module for, if the memory processing mechanism is triggered, selecting an in-memory server according to the high-performance computing and scalability requirements, accessing the original server using the hot-plug technology, configuring the in-memory database, and initializing the in-memory computing cluster; A data synchronization and in-memory computing preparation module for configuring the database tables to be synchronized using the metadata management function, loading the original data into the in-memory database through the full synchronization and incremental update mechanisms, and enabling real-time data synchronization; An in-memory computing execution module for starting the in-memory computing cluster, automatically detecting the memory processing status on the original server when a business task is triggered, and routing eligible large-scale data tasks to the in-memory server for parallel computing.

9. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method for enabling the in-memory database according to any one of claims 1 to 7.

10. A storage medium, on which a computer program is stored, characterized in that, When the computer program is executed by the processor, it implements the steps of the method for enabling the in-memory database according to any one of claims 1 to 7.

Images