Public cloud elastic capacity expansion method and device based on freight rate search cluster

Abstract

The invention discloses a public cloud elastic capacity expansion method and device based on a freight rate search cluster, and relates to the technical field of freight rate search or other related fields, and the method comprises the following steps: a data summarization node receives a data file, verifies a file format, distributes the data file with the correct format to a data preprocessing node, and stores the data file in the data preprocessing node; the data preprocessing node updates the memory mapping database, distributes the updated part of the database to the freight rate search engine node, obtains the average CPU and the memory utilization rate of the freight rate search engine node, records the duration and the growth rate of the average CPU and the memory utilization rate when the two are greater than respective threshold values, and sends the freight rate search engine node to the data preprocessing node when the duration is greater than the target interval duration. And calculating a capacity expansion target number of the freight rate search engine nodes, determining a target capacity expansion strategy based on the capacity expansion target number and a capacity expansion threshold value, and carrying out capacity expansion processing on the public cloud. According to the invention, the technical problem of unstable resource utilization rate caused by large change of system access volume in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the field of freight rate search technology or other related fields, and in particular to a method and device for elastically expanding a public cloud based on a freight rate search cluster. Background Art

[0002] In the field of fare search technology, with the rapid development of domestic civil aviation and e-commerce, in order to better meet the personalized travel needs of passengers and consolidate and expand the existing market, online air ticket purchases have experienced explosive growth. As airlines and agencies are increasingly involved in major promotions such as "Double Eleven" and "618" in ticket sales, the concurrency of the fare search system has increased dramatically during the promotion period, while in the off-season for ticket sales, the concurrency of the fare search system is relatively low and stable. The large fluctuations in the concurrency of the fare search system have led to unstable utilization of the hardware and network resources of the fare search system, showing a state of low resource utilization in the off-season and basically full-load operation of resources during the promotion period.

[0003] In the related art, the deployment mode of the freight rate search system is usually to purchase hardware resources and deploy them in a private data center, which will result in the inability to release resources in time when resource utilization is low, resulting in increased costs. When the system is overloaded, the speed of expanding resources will also be affected by the quality of hardware resources, and it is impossible to achieve rapid expansion. Therefore, when the concurrency increases or decreases, the use of resources is very inflexible. At the same time, in a market where costs are becoming increasingly refined, since hardware maintenance costs, labor costs, electricity costs, depreciation costs and other expenses cannot be specifically calculated, the traditional data center application deployment strategy cannot calculate the system operation costs, that is, the freight rate search system deployment strategy of purchasing hardware resources and deploying them in a data center can no longer meet market demand.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiment of the present invention provides a method and device for elastically expanding the capacity of a public cloud based on a freight rate search cluster, so as to at least solve the technical problem in the related art that the system access volume varies greatly, resulting in unstable resource utilization.

[0006] According to one aspect of an embodiment of the present invention, a public cloud elastic expansion method based on a freight search cluster is provided. The freight search cluster is deployed on a public cloud. The freight search cluster includes at least a data aggregation node, a data preprocessing node, and a freight search engine node. The method includes: responding to a freight search request, receiving a business data file by the data aggregation node, verifying the format of the business data file, and distributing the business data file with a correct format to the data preprocessing node; updating a memory mapping database according to a preset update frequency by the data preprocessing node, and distributing the updated memory mapping database to the freight search engine node; obtaining an average C of the freight search engine node; and PU usage rate, average memory usage rate. When the average CPU usage rate is greater than the preset CPU usage rate threshold or the average memory usage rate is greater than the preset memory usage rate threshold, record the duration and the growth rate of the average CPU usage rate and the average memory usage rate. When the duration is greater than the target interval, calculate the target number of expansion of the freight search engine nodes according to the growth rate. Determine the target expansion strategy based on the target number of expansion of the freight search engine nodes and the expansion threshold, and expand the public cloud through the target expansion strategy, where the target expansion strategy is one of the following: cluster expansion method and node expansion method.

[0007] Optionally, the step of verifying the format of the business data file and distributing the business data file with the correct format to the data preprocessing node includes: performing regular expression matching verification on the file name of the business data file; verifying the timestamp of the business data file; archiving the business data file according to the file name and timestamp when both the file name and timestamp of the business data file are verified to be correct; and distributing the archived business data file to the data preprocessing node using a file transfer tool.

[0008] Optionally, the step of distributing the updated part of the memory-mapped database to the freight rate search engine node includes: sending a first heartbeat message by the data preprocessing node to the freight rate search engine node, the first heartbeat message including the timestamp and modification data of the updated memory-mapped database; upon receiving the first heartbeat message, receiving a second heartbeat message sent by the freight rate search engine node by the data preprocessing node, the second heartbeat message including the timestamp and modification data of the local database of the freight rate search engine node; and distributing the updated part of the memory-mapped database to the freight rate search engine node when there is a difference between the first heartbeat message and the second heartbeat message and the timestamp of the local database of the data preprocessing node is updated.

[0009] Optionally, after distributing the updated part of the memory-mapped database to the freight rate search engine node, the method includes: loading the memory-mapped database into a disk of the freight rate search engine node; and performing a search operation using different memory-mapped databases according to a freight rate search request.

[0010] Optionally, when obtaining the capacity expansion threshold of the freight rate search engine node, it includes: obtaining the resource configuration of all freight rate search engine nodes in the freight rate search cluster, wherein the resource configuration includes at least one of the following: the number of CPU cores, memory capacity, disk capacity, network bandwidth, and the maximum transmission rate of the memory mapping database; based on the resource configuration of all freight rate search engine nodes and a preset resource configuration threshold, the capacity expansion threshold of the freight rate search cluster is calculated.

[0011] Optionally, the step of determining the target expansion strategy includes: when the sum of the number of freight rate search engine nodes and the expansion target number is greater than the expansion threshold, selecting the target expansion strategy as the cluster expansion mode; when the sum of the number of freight rate search engine nodes and the expansion target number is less than or equal to the expansion threshold, selecting the target expansion strategy as the node expansion mode.

[0012] Optionally, after the public cloud is expanded, the process includes: obtaining the average CPU usage and average memory usage of all freight rate search engine nodes; recording the duration when the average CPU usage is less than a preset CPU usage threshold and the average memory usage is less than a preset memory usage threshold; and releasing the newly added freight rate search engine node when the duration is less than or equal to the target interval.

[0013] According to another aspect of an embodiment of the present invention, a public cloud elastic expansion device based on a freight search cluster is also provided. The freight search cluster is deployed on a public cloud. The freight search cluster includes at least a data aggregation node, a data preprocessing node and a freight search engine node, including: a data aggregation unit, which is used to respond to freight search requests, receive business data files by the data aggregation node, verify the format of the business data files, and distribute the business data files with correct formats to the data preprocessing node; a data preprocessing unit, which is used to update the memory mapping database according to a preset update frequency by the data preprocessing node, and distribute the updated part of the memory mapping database to the freight search engine node; a recording unit, which is used to obtain the freight search index; The average CPU usage and average memory usage of the engine node are recorded. When the average CPU usage is greater than the preset CPU usage threshold or the average memory usage is greater than the preset memory usage threshold, the duration is recorded, and the growth rate of the average CPU usage and the average memory usage is recorded; a calculation unit is used to calculate the expansion target number of freight search engine nodes according to the growth rate when the duration is greater than the target interval; an expansion unit determines the target expansion strategy based on the expansion target number and the expansion threshold of the freight search engine nodes, and expands the public cloud through the target expansion strategy, wherein the target expansion strategy is one of the following: cluster expansion method and node expansion method.

[0014] Optionally, the data aggregation unit includes: performing regular expression matching verification on the file name of the business data file; verifying the timestamp of the business data file; archiving the business data file according to the file name and timestamp when both the file name and timestamp of the business data file are verified correctly; and using a file transfer tool to distribute the archived business data file to a data preprocessing node.

[0015] Optionally, the data preprocessing unit includes: sending a first heartbeat message by the data preprocessing node to the freight rate search engine node, the first heartbeat message including the timestamp and modification data of the updated memory mapping database; upon receiving the first heartbeat message, receiving a second heartbeat message sent by the freight rate search engine node by the data preprocessing node, the second heartbeat message including the timestamp and modification data of the local database of the freight rate search engine node; when there is a difference between the first heartbeat message and the second heartbeat message and the timestamp of the local database of the data preprocessing node is updated, distributing the updated part of the memory mapping database to the freight rate search engine node.

[0016] Optionally, the data preprocessing unit further includes: loading the memory mapping database into a disk of the freight rate search engine node; and performing a search operation using different memory mapping databases according to the freight rate search request.

[0017] Optionally, the capacity expansion unit includes: obtaining resource configuration of all freight search engine nodes in the freight search cluster, wherein the resource configuration includes at least one of the following: number of CPU cores, memory capacity, disk capacity, network bandwidth, and maximum transmission rate of a memory-mapped database; and calculating an expansion threshold of the freight search cluster based on the resource configuration of all freight search engine nodes and a preset resource configuration threshold.

[0018] Optionally, the expansion unit also includes: when the sum of the number of freight search engine nodes and the expansion target number is greater than the expansion threshold, selecting the target expansion strategy as the cluster expansion method; when the sum of the number of freight search engine nodes and the expansion target number is less than or equal to the expansion threshold, selecting the target expansion strategy as the node expansion method.

[0019] Optionally, the expansion unit also includes: obtaining the average CPU usage and average memory usage of all freight rate search engine nodes; when the average CPU usage is less than a preset CPU usage threshold and the average memory usage is less than a preset memory usage threshold, recording the duration; when the duration is less than or equal to the target interval duration, releasing the newly added freight rate search engine node.

[0020] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is further provided, the computer-readable storage medium including a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned methods for elastically expanding the capacity of a public cloud based on a freight rate search cluster.

[0021] According to another aspect of an embodiment of the present invention, there is also provided an electronic device, comprising one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors implement any one of the above-mentioned methods for elastically expanding the capacity of a public cloud based on a freight rate search cluster.

[0022] According to another aspect of an embodiment of the present invention, a computer program product is further provided, including a computer program, which, when executed by a processor, implements the steps of any one of the above-mentioned methods for elastically expanding the capacity of a public cloud based on a freight rate search cluster.

[0023] In the present disclosure, a freight search cluster is deployed on a public cloud, a data aggregation node receives a business data file, verifies the format of the business data file, distributes the business data file with the correct format to a data preprocessing node, and then the data preprocessing node updates a memory mapping database according to a preset update frequency, and distributes the updated memory mapping database to a freight search engine node. By obtaining an average CPU usage rate and an average memory usage rate of the freight search engine node, when the average CPU usage rate is greater than a preset CPU usage rate threshold or the average memory usage rate is greater than a preset memory usage rate threshold, the duration is recorded, and the growth rate of the average CPU usage rate and the average memory usage rate is recorded. Finally, when the duration is greater than a target interval, the expansion target number of the freight search engine node is calculated according to the growth rate, and based on the expansion target number of the freight search engine node and the expansion threshold, a target expansion strategy is determined, and the public cloud is expanded through the target expansion strategy. The present disclosure obtains the average CPU usage and average memory usage of the freight search engine nodes, calculates the expansion target number of the freight search engine nodes, and then determines whether it is necessary to expand the public cloud based on the expansion target number and the expansion threshold. At the same time, the expansion strategy is determined when expansion is required, that is, by adjusting the number of resources according to the increase or decrease in the actual concurrency of the freight search system, resource waste or resource shortage is avoided, the purpose of reducing costs and increasing efficiency is achieved, and the technical problem of large changes in system access volume and unstable resource utilization caused by related technologies is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0025] Figure 1 is a flow chart of an optional public cloud elastic expansion method based on a freight rate search cluster according to an embodiment of the present invention;

[0026] Figure 2 is a schematic diagram of an optional public cloud elastic expansion method based on a freight rate search cluster according to an embodiment of the present invention;

[0027] Figure 3 It is a flow chart of a freight rate search cluster of an optional public cloud elastic expansion method based on a freight rate search cluster according to an embodiment of the present invention;

[0028] Figure 4 It is a capacity expansion flow chart of an optional public cloud elastic capacity expansion method based on a freight rate search cluster according to an embodiment of the present invention;

[0029] Figure 5 It is an overall structural diagram of an optional public cloud elastic expansion method based on a freight rate search cluster according to an embodiment of the present invention;

[0030] Figure 6 is a schematic diagram of an optional public cloud elastic capacity expansion device based on a freight rate search cluster according to an embodiment of the present invention;

[0031] Figure 7 The present invention is a hardware structure block diagram of an electronic device (or mobile device) according to a method for elastically expanding capacity of a public cloud based on a freight rate search cluster according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0034] To facilitate those skilled in the art to understand the present invention, some terms or nouns involved in the embodiments of the present invention are explained below:

[0035] Freight rate search: A search strategy that uses a specific search engine or tool through channels such as the Internet or applications to query and compare the prices and rules of different transportation services.

[0036] Public cloud: A shared resource such as computing, storage, and network provided by a third-party provider.

[0037] It should be noted that the method and device for elastically expanding the public cloud based on the freight rate search cluster in the present disclosure can be used in the field of freight rate search technology. When the public cloud is elastically expanded based on the freight rate search cluster, it can also be used in any field except the field of freight rate search technology. When the public cloud is elastically expanded based on the freight rate search cluster, the present disclosure does not limit the application field of the method and device for elastically expanding the public cloud based on the freight rate search cluster.

[0038] It should be noted that the information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) collected by this disclosure are information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data are in compliance with the relevant laws, regulations and standards of the relevant regions, necessary confidentiality measures are taken, and public order and good customs are not violated, and corresponding operation portals are provided for users to choose to authorize or refuse. For example, an interface is set up between this system and relevant users or organizations. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or organization through the interface, and obtain relevant information after receiving the consent information fed back by the aforementioned user or organization.

[0039] It should be noted that in the present disclosure, when collecting and analyzing customer information, corresponding operation entrances are provided for users to choose to agree or reject the automated decision results; if the user chooses to reject, the expert decision-making process is entered.

[0040] The following embodiments of the present invention can be applied to various systems / applications / devices for elastic expansion of public clouds based on freight search clusters. The present invention obtains the average CPU usage and average memory usage of freight search engine nodes, calculates the expansion target number of freight search engine nodes, and then determines whether the public cloud needs to be expanded based on the expansion target number and the expansion threshold. At the same time, the expansion strategy is determined when expansion is required, that is, by adjusting the number of resources according to the increase or decrease in the actual concurrency of the freight search system, the phenomenon of resource waste or resource shortage is avoided, and the purpose of reducing costs and increasing efficiency is achieved, thereby solving the technical problem of large changes in system access volume and unstable resource utilization in related technologies.

[0041] The present invention is described in detail below in conjunction with various embodiments.

[0042] Embodiment 1

[0043] According to an embodiment of the present invention, an embodiment of a method for elastically expanding the capacity of a public cloud based on a freight rate search cluster is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0044] Figure 1 is a flowchart of an optional public cloud elastic expansion method based on a freight rate search cluster according to an embodiment of the present invention, such as Figure 1 As shown, the method comprises the following steps:

[0045] Step S101, in response to a freight rate search request, a data aggregation node receives a business data file, verifies the format of the business data file, and distributes the business data file with a correct format to a data preprocessing node.

[0046] Step S102: The data preprocessing node updates the memory mapping database according to a preset update frequency, and distributes the updated memory mapping database to the freight rate search engine node.

[0047] Step S103, obtaining the average CPU usage and average memory usage of the freight rate search engine node, and when the average CPU usage is greater than a preset CPU usage threshold or the average memory usage is greater than a preset memory usage threshold, recording the duration and the growth rate of the average CPU usage and the average memory usage.

[0048] Step S104: when the duration is longer than the target interval, the target number of expansion of the freight rate search engine nodes is calculated according to the growth rate.

[0049] Step S105, based on the expansion target number and expansion threshold of the freight rate search engine node, determine the target expansion strategy, and expand the public cloud through the target expansion strategy, wherein the target expansion strategy is one of the following: cluster expansion mode, node expansion mode.

[0050] The steps provided by the above-mentioned public cloud elastic expansion method based on the freight search cluster can be performed by deploying the freight search cluster on the public cloud, receiving the business data file by the data aggregation node, verifying the format of the business data file, distributing the business data file with the correct format to the data preprocessing node, and then updating the memory mapping database by the data preprocessing node according to the preset update frequency, distributing the updated part of the memory mapping database to the freight search engine node, obtaining the average CPU usage and average memory usage of the freight search engine node, and recording the duration when the average CPU usage is greater than the preset CPU usage threshold or the average memory usage is greater than the preset memory usage threshold, and recording the growth rate of the average CPU usage and the average memory usage. Finally, when the duration is greater than the target interval, the expansion target number of the freight search engine node is calculated according to the growth rate, and based on the expansion target number and the expansion threshold of the freight search engine node, the target expansion strategy is determined, and the public cloud is expanded through the target expansion strategy. This embodiment obtains the average CPU usage and average memory usage of the freight search engine node, calculates the expansion target number of the freight search engine node, and then determines whether the public cloud needs to be expanded based on the expansion target number and the expansion threshold. At the same time, the expansion strategy is determined when expansion is needed, that is, by adjusting the number of resources according to the increase or decrease in the actual concurrency of the freight search system, resource waste or resource shortage is avoided, the purpose of reducing costs and increasing efficiency is achieved, and the technical problem of large changes in system access volume and unstable resource utilization caused by related technologies is solved.

[0051] The embodiment of the present invention is described in detail below in combination with the above steps.

[0052] It should be noted that the freight search cluster mentioned in this embodiment is deployed on the public cloud. By renting the cloud server provided by the public cloud, data aggregation nodes, data preprocessing nodes and freight search engine nodes are deployed. This embodiment adopts a node-based distributed system architecture to deploy each node separately on different cloud servers, which brings many benefits to the freight search system, such as: better replaceability: if a node needs to be replaced, it can be operated without affecting the entire system, greatly reducing the risk and cost of upgrading or replacing nodes; better scalability: as demand changes, the freight search system may need to add additional functions, and the distributed system architecture allows new nodes to be added without destroying the existing system architecture; faster development speed: different teams or individuals can develop different nodes at the same time, shortening the development cycle; better flexibility and adaptability: because the nodes are independent, the freight search system can adapt to changes in the external environment more flexibly, and different technology stacks or frameworks can be selected to implement specific nodes. In this embodiment, the distributed system architecture allows performance optimization for specific nodes without worrying about affecting other parts of the entire freight search system.

[0053] Figure 2 is a schematic diagram of an optional public cloud elastic expansion method based on a freight rate search cluster according to an embodiment of the present invention. Figure 2 In the present invention, the freight rate search cluster includes multiple data aggregation nodes, multiple data preprocessing nodes and multiple freight rate search engine nodes, wherein the data aggregation node distributes the processed business data files to the data preprocessing node, and the data preprocessing node distributes the updated part of the memory mapping database to the freight rate search node.

[0054] Step S101, in response to a freight rate search request, a data aggregation node receives a business data file, verifies the format of the business data file, and distributes the business data file with a correct format to a data preprocessing node.

[0055] Here, the business of the fare search system is very complex, and the business data file contains a lot of information, such as: information related to the price of the ticket, information related to the applicable conditions of the ticket, information related to the applicable routes of the ticket, information related to taxes and fees in countries around the world, and information related to seat selection and upgrades. The embodiment of the present invention summarizes the aforementioned information into N types of business data files, and sends the business data files to the data aggregation node of the fare search cluster.

[0056] Figure 3 is a flow chart of a freight rate search cluster of an optional public cloud elastic expansion method based on a freight rate search cluster according to an embodiment of the present invention. Figure 3In the data aggregation node, by responding to the freight rate search request, the embodiment of the present invention receives the business data file (step 1.1) by the data aggregation node, then verifies the format of the business data file (step 1.2), and finally distributes the business data file with the correct format to the data preprocessing node (step 1.3), and the data preprocessing node processes the business data file into a memory-mapped database file.

[0057] Optionally, the step of verifying the format of the business data file and distributing the business data file with the correct format to the data preprocessing node includes: performing regular expression matching verification on the file name of the business data file; verifying the timestamp of the business data file; archiving the business data file according to the file name and timestamp when both the file name and timestamp of the business data file are verified to be correct; and distributing the archived business data file to the data preprocessing node using a file transfer tool.

[0058] In an embodiment of the present invention, when format verification is performed on a business data file, it is necessary to perform file name verification and file timestamp verification. The embodiment of the present invention first maintains a system configuration file, which contains a regular expression. When the file name of the business data file is verified, the file name of the received business data file is matched with the regular expression in the configuration file. If the match is successful, it is determined that the file name verification is correct. If the match is unsuccessful, it is determined that the file name verification has failed. When the file timestamp of the business data file is verified, it is verified whether the number of digits of the timestamp meets the regulations and whether the date of the timestamp is the date of the freight rate search system. If the number of digits and the date of the timestamp meet the requirements, it is determined that the file timestamp verification of the business data file is correct. If any of the number of digits and the date of the timestamp does not meet the requirements, it is determined that the file timestamp verification of the business data file has failed.

[0059] When the file name and file timestamp of the business data file are verified correctly, the embodiment of the present invention will archive the business data file according to the file name and timestamp, and then use the file transfer tool to distribute the archived business data file to the data preprocessing node, and the data preprocessing node will process the business data file into a memory-mapped database file.

[0060] Step S102: The data preprocessing node updates the memory mapping database according to a preset update frequency, and distributes the updated memory mapping database to the freight rate search engine node.

[0061] In an embodiment of the present invention, the data preprocessing node uses a memory-mapped database to store data. A memory-mapped database is a database that reads data from a disk into a memory and then accesses and operates the data in the memory. The memory-mapped database has many advantages, including: fast access speed: the memory-mapped database stores data in the memory and does not need to read from the disk frequently, so it has a very fast access speed, which is very useful for applications that require high-speed reading and writing; scalability: the memory-mapped database can handle very large data sets, because the memory-mapped database can use the virtual memory management mechanism of the operating system to manage memory, so that data that exceeds the physical memory capacity can be accessed; persistence: the memory-mapped database stores data in the disk, so the data can still be retained after the process is closed, which makes the memory-mapped database very suitable for applications that need to retain data for a long time; easy to manage: the memory-mapped database is usually easier to manage than a traditional disk-based database, because the memory-mapped database does not require the application to perform cache management for read / write operations; easy to share: the memory-mapped database can be shared by multiple processes, which makes the memory-mapped database very suitable for multi-process applications. In a shared manner, multiple processes can access the same database file at the same time without copying and synchronizing data.

[0062] The embodiment of the present invention greatly improves the search performance of the freight rate search system by adopting a memory mapping database. Figure 3 In the data preprocessing node, the embodiment of the present invention receives the business data file by the data preprocessing node (step 2.1), then processes the business data file into a memory-mapped database file, then updates the memory-mapped database (step 2.2), and then distributes the updated part of the memory-mapped database to the freight search engine node (step 2.3), and the freight search engine node provides a freight search service.

[0063] Optionally, the step of distributing the updated part of the memory-mapped database to the freight rate search engine node includes: sending a first heartbeat message by the data preprocessing node to the freight rate search engine node, the first heartbeat message including the timestamp and modification data of the updated memory-mapped database; upon receiving the first heartbeat message, receiving a second heartbeat message sent by the freight rate search engine node by the data preprocessing node, the second heartbeat message including the timestamp and modification data of the local database of the freight rate search engine node; and distributing the updated part of the memory-mapped database to the freight rate search engine node when there is a difference between the first heartbeat message and the second heartbeat message and the timestamp of the local database of the data preprocessing node is updated.

[0064] In an embodiment of the present invention, the data preprocessing node will immediately update the memory mapping database after receiving a new business data file according to the preset update frequency, ensuring that the memory mapping database is always kept up to date. After the memory mapping database is updated, the embodiment of the present invention will store the database file to the local disk for persistent storage of the data file.

[0065] After the memory mapping database is updated, the embodiment of the present invention will distribute the updated part of the memory mapping database to all freight search engine nodes by unicast. In order to determine when to distribute the updated part of the memory mapping database to the freight search engine node, the embodiment of the present invention sends a first heartbeat message to the freight search engine node through the data preprocessing node. The content of the first heartbeat message includes the timestamp and modification data of the updated memory mapping database. After receiving the first heartbeat message, the freight search engine node is sent a second heartbeat message by the data preprocessing node. The content of the second heartbeat message includes the timestamp and modification data of the local database of the freight search engine node. The data preprocessing node will compare the contents of the first heartbeat message and the second heartbeat message, respectively compare the timestamps and modification data of the first heartbeat message and the second heartbeat message. If there is a difference between the first heartbeat message and the second heartbeat message, and the timestamp of the local database of the data preprocessing node is updated, the updated part of the memory mapping database will be distributed to the freight search engine node.

[0066] Optionally, after distributing the updated part of the memory-mapped database to the freight rate search engine node, the method includes: loading the memory-mapped database into a disk of the freight rate search engine node; and performing a search operation using different memory-mapped databases according to a freight rate search request.

[0067] In the embodiment of the present invention, after distributing the updated memory mapping database to the freight search engine node, the freight search engine node will load the memory mapping database to the freight search engine node's disk, and use different memory mapping databases to perform search operations according to the freight search request. Figure 3 In the freight rate search engine node, the embodiment of the present invention first receives the memory mapping database distributed by the data preprocessing node (step 3.1), then loads the new memory mapping database into the memory (step 3.2), and finally uses different memory mapping databases to provide freight rate search services according to the freight rate search request (step 3.3).

[0068] Step S103, obtaining the average CPU usage and average memory usage of the freight rate search engine node, and when the average CPU usage is greater than a preset CPU usage threshold or the average memory usage is greater than a preset memory usage threshold, recording the duration and the growth rate of the average CPU usage and the average memory usage.

[0069] In an embodiment of the present invention, the elastic expansion strategy of the public cloud is implemented based on the elastic expansion application node. The elastic expansion application node is independently deployed on the public cloud, and the CPU usage and memory usage of all freight search engine nodes are obtained in real time, and the average CPU usage and average memory usage of the freight search engine node are obtained by using a weighted average calculation strategy, and compared with the preset CPU usage threshold and the preset memory usage threshold. When the average CPU usage is greater than the preset CPU usage threshold or the average memory usage is greater than the preset memory usage threshold, the duration is recorded, and the growth rate of the average CPU usage and the average memory usage is recorded. The recorded duration and growth rate will be used in the subsequent steps to calculate the target number of freight search engine node expansion.

[0070] Step S104: when the duration is longer than the target interval, the target number of expansion of the freight rate search engine nodes is calculated according to the growth rate.

[0071] In the embodiment of the present invention, the duration and the growth rate are obtained in step S103. When the duration is greater than the target interval, the expansion target number of the freight search engine node is calculated according to the growth rate. The target interval is set to five minutes in the embodiment of the present invention. That is, when the average CPU usage of the freight search engine node is greater than the preset CPU usage threshold or the average memory usage is greater than the preset memory usage threshold for five minutes, the expansion target number of the freight search engine node is calculated by continuously recording the growth rate of the average CPU usage and the average memory usage.

[0072] Figure 4 is an expansion flow chart of an optional public cloud elastic expansion method based on a freight rate search cluster according to an embodiment of the present invention. Figure 4 In the process, the elastic expansion application node obtains the usage status from the freight search engine node of the freight search cluster. The usage status includes the average CPU usage and the average memory usage. The elastic expansion application node determines whether the public cloud needs to be expanded based on the usage status of the freight search engine node. That is, when the average CPU usage of the freight search engine node is greater than the preset CPU usage threshold or the average memory usage is greater than the preset memory usage threshold, and the duration is greater than the target interval, it is determined that the public cloud needs to be expanded.

[0073] Step S105, based on the expansion target number and expansion threshold of the freight rate search engine node, determine the target expansion strategy, and expand the public cloud through the target expansion strategy, wherein the target expansion strategy is one of the following: cluster expansion mode, node expansion mode.

[0074] In the embodiment of the present invention, the expansion target number of the freight rate search engine node is obtained in step S104, and the target expansion strategy is determined by combining the expansion threshold, and the public cloud is expanded by the target expansion strategy. Figure 4 In the embodiment of the present invention, after determining that the public cloud needs to be expanded, the target expansion strategy is determined, and then an expansion notification is sent to the public cloud elastic policy interface. After receiving the expansion notification, the public cloud elastic policy interface will notify the public cloud resource pool to open / release resources.

[0075] Optionally, when obtaining the capacity expansion threshold of the freight rate search engine node, it includes: obtaining the resource configuration of all freight rate search engine nodes in the freight rate search cluster, wherein the resource configuration includes at least one of the following: the number of CPU cores, memory capacity, disk capacity, network bandwidth, and the maximum transmission rate of the memory mapping database; based on the resource configuration of all freight rate search engine nodes and a preset resource configuration threshold, the capacity expansion threshold of the freight rate search cluster is calculated.

[0076] In an embodiment of the present invention, the step of obtaining the expansion threshold of the freight search engine node includes first obtaining the resource configuration of all freight search engine nodes in the freight search cluster, the resource configuration including at least one of the following: number of CPU cores, memory capacity, disk capacity, network bandwidth, and maximum transmission rate of the memory mapping database, and then calculating the expansion threshold of the freight search cluster based on the resource configuration of all freight search engine nodes and the preset resource configuration threshold, that is, calculating the maximum number of freight search engine nodes that the freight search cluster can be expanded to, wherein the preset resource configuration threshold is set according to resource quality and maintenance experience. Figure 4 In the elastic expansion application node, the resource configuration of all freight search engine nodes is obtained from the freight search cluster, the expansion threshold is obtained according to the resource configuration, and finally the expansion strategy is determined according to the expansion threshold, that is, whether to perform cluster expansion or node expansion.

[0077] Optionally, the step of determining the target expansion strategy includes: when the sum of the number of freight rate search engine nodes and the expansion target number is greater than the expansion threshold, selecting the target expansion strategy as the cluster expansion mode; when the sum of the number of freight rate search engine nodes and the expansion target number is less than or equal to the expansion threshold, selecting the target expansion strategy as the node expansion mode.

[0078] In the embodiment of the present invention, when determining the target expansion strategy, when the sum of the number of freight search engine nodes and the expansion target number is greater than the expansion threshold, the cluster expansion method is selected; when the sum of the number of freight search engine nodes and the expansion target number is less than or equal to the expansion threshold, the node expansion method is selected. Figure 4When the target expansion method is determined to be cluster expansion, the public cloud elastic policy interface will notify the public cloud resource pool to start a new freight search cluster. The new freight search cluster includes complete data aggregation nodes, data preprocessing nodes, and freight search engine nodes. When the target expansion method is determined to be node expansion, the public cloud elastic policy interface will notify the public cloud resource pool to start one or more freight search engine nodes.

[0079] Optionally, after the public cloud is expanded, the process includes: obtaining the average CPU usage and average memory usage of all freight rate search engine nodes; recording the duration when the average CPU usage is less than a preset CPU usage threshold and the average memory usage is less than a preset memory usage threshold; and releasing the newly added freight rate search engine node when the duration is less than or equal to the target interval.

[0080] In an embodiment of the present invention, after the public cloud is expanded, the concurrency of the freight search system will be continuously monitored. When the concurrency decreases and persists, the newly opened cluster or new node will be released. The specific steps are: first, the average CPU usage and average memory usage of all freight search engine nodes are obtained. When the average CPU usage is less than the preset CPU usage threshold and the average memory usage is less than the preset memory usage threshold, the duration is recorded. When the duration is less than or equal to the target interval duration, the newly added freight search engine node or the newly added freight search cluster is released, wherein the target interval duration is set by oneself. For example, the target interval duration is set to fifteen minutes in an embodiment of the present invention.

[0081] In an embodiment of the present invention, the freight rate search system is deployed on a public cloud. It does not use public cloud native databases, middleware and other infrastructure, but only uses public cloud servers to deploy the system. The advantage of this deployment strategy is that if a public cloud has a large-scale failure, the same code can be quickly deployed to other public clouds or private cloud environments, thereby reducing the time required for fault recovery. At the same time, since the freight rate search system is not bound to the public cloud, it is more flexible in the design of disaster recovery solutions. By deploying the freight rate search system on multiple public clouds at the same time, unified management of multiple freight rate search systems can be achieved.

[0082] Figure 5 is an overall structural diagram of an optional public cloud elastic expansion method based on a freight rate search cluster according to an embodiment of the present invention. Figure 5In the figure, freight search cluster 1 is the original freight search cluster, which includes data aggregation nodes, data preprocessing nodes, and freight search engine nodes. The elastic expansion application node determines whether the public cloud needs to be elastically expanded by obtaining the node usage status of the freight search engine nodes, that is, the average CPU usage and average memory usage of the freight search engine nodes. In the case where it is determined that the public cloud needs to be elastically expanded, the expansion target number of freight search engine nodes is calculated, and the target expansion strategy is determined based on the expansion threshold, and the public cloud elastic strategy interface is notified to perform corresponding expansion processing. If the target expansion strategy is determined to be a cluster expansion method, a new freight search cluster is popped up, that is, Figure 5 If the target expansion method is determined to be node expansion, one or more freight search engine nodes will be added. After the expansion process, the elastic expansion application node will manage the newly added freight search cluster or the newly added freight search engine node.

[0083] The embodiment of the present invention obtains the average CPU usage and average memory usage of the freight search engine node, calculates the expansion target number of the freight search engine node, and then determines whether it is necessary to expand the public cloud based on the expansion target number and the expansion threshold. At the same time, the expansion strategy is determined when expansion is required, that is, by adjusting the number of resources according to the increase or decrease of the actual concurrency of the freight search system, the phenomenon of resource waste or resource shortage is avoided, and the purpose of reducing costs and increasing efficiency is achieved.

[0084] The following is a detailed description in conjunction with another embodiment.

[0085] Embodiment 2

[0086] A public cloud elastic capacity expansion device based on a freight rate search cluster provided in this embodiment includes multiple implementation units, each of which corresponds to each implementation step in the above-mentioned embodiment 1.

[0087] Figure 6 is a schematic diagram of an optional public cloud elastic expansion device based on a freight rate search cluster according to an embodiment of the present invention, such as Figure 6 As shown, the public cloud elastic capacity expansion device based on the freight rate search cluster may include: a data aggregation unit 60, a data preprocessing unit 61, a recording unit 62, a calculation unit 63, and a capacity expansion unit 64.

[0088] The data aggregation unit 60 is used to respond to the freight rate search request, receive the business data file from the data aggregation node, verify the format of the business data file, and distribute the business data file with the correct format to the data pre-processing node.

[0089] The data preprocessing unit 61 is used to update the memory mapping database according to a preset update frequency by the data preprocessing node, and distribute the updated part of the memory mapping database to the freight rate search engine node.

[0090] The recording unit 62 is used to obtain the average CPU usage and average memory usage of the freight rate search engine node, and when the average CPU usage is greater than a preset CPU usage threshold or the average memory usage is greater than a preset memory usage threshold, record the duration and the growth rate of the average CPU usage and the average memory usage.

[0091] The calculation unit 63 is used to calculate the expansion target number of the freight rate search engine nodes according to the growth rate when the duration is greater than the target interval.

[0092] The expansion unit 64 determines a target expansion strategy based on the expansion target number and expansion threshold of the freight rate search engine node, and expands the public cloud through the target expansion strategy, wherein the target expansion strategy is one of the following: cluster expansion mode and node expansion mode.

[0093] The above-mentioned public cloud elastic expansion device based on the freight search cluster can be implemented by deploying the freight search cluster on the public cloud, receiving the business data file by the data aggregation node through the data aggregation unit 60, verifying the format of the business data file, distributing the business data file with the correct format to the data preprocessing node, and then updating the memory mapping database by the data preprocessing node according to the preset update frequency through the data preprocessing unit 61, distributing the updated part of the memory mapping database to the freight search engine node, obtaining the average CPU usage rate and the average memory usage rate of the freight search engine node through the recording unit 62, and recording the duration when the average CPU usage rate is greater than the preset CPU usage rate threshold or the average memory usage rate is greater than the preset memory usage rate threshold, and recording the growth rate of the average CPU usage rate and the average memory usage rate, and finally calculating the expansion target number of the freight search engine node according to the growth rate when the duration is greater than the target interval time through the calculation unit 63, and determining the target expansion strategy based on the expansion target number of the freight search engine node and the expansion threshold through the expansion unit 64, and performing expansion processing on the public cloud through the target expansion strategy. This embodiment obtains the average CPU usage and average memory usage of the freight search engine node, calculates the expansion target number of the freight search engine node, and then determines whether the public cloud needs to be expanded based on the expansion target number and the expansion threshold. At the same time, the expansion strategy is determined when expansion is needed, that is, by adjusting the number of resources according to the increase or decrease in the actual concurrency of the freight search system, resource waste or resource shortage is avoided, the purpose of reducing costs and increasing efficiency is achieved, and the technical problem of large changes in system access volume and unstable resource utilization caused by related technologies is solved.

[0094] Optionally, the data aggregation unit 60 includes: performing regular expression matching verification on the file name of the business data file; verifying the timestamp of the business data file; archiving the business data file according to the file name and timestamp when both the file name and timestamp of the business data file are verified correctly; and using a file transfer tool to distribute the archived business data file to a data preprocessing node.

[0095] Optionally, the data preprocessing unit 61 includes: sending a first heartbeat message by the data preprocessing node to the freight rate search engine node, the first heartbeat message including the timestamp and modification data of the updated memory mapping database; receiving a second heartbeat message sent by the freight rate search engine node by the data preprocessing node upon receiving the first heartbeat message, the second heartbeat message including the timestamp and modification data of the local database of the freight rate search engine node; when there is a difference between the first heartbeat message and the second heartbeat message and the timestamp of the local database of the data preprocessing node is updated, distributing the updated part of the memory mapping database to the freight rate search engine node.

[0096] Optionally, the data preprocessing unit 61 further includes: loading the memory mapping database into a disk of the freight rate search engine node; and using different memory mapping databases to perform search operations according to the freight rate search request.

[0097] Optionally, the capacity expansion unit 64 includes: obtaining resource configuration of all freight search engine nodes in the freight search cluster, wherein the resource configuration includes at least one of the following: number of CPU cores, memory capacity, disk capacity, network bandwidth, and maximum transmission rate of the memory mapping database; based on the resource configuration of all freight search engine nodes and a preset resource configuration threshold, calculating the capacity expansion threshold of the freight search cluster.

[0098] Optionally, the expansion unit 64 further includes: when the sum of the number of freight search engine nodes and the expansion target number is greater than the expansion threshold, selecting the target expansion strategy as the cluster expansion mode; when the sum of the number of freight search engine nodes and the expansion target number is less than or equal to the expansion threshold, selecting the target expansion strategy as the node expansion mode.

[0099] Optionally, the expansion unit 64 also includes: obtaining the average CPU usage and average memory usage of all freight rate search engine nodes; when the average CPU usage is less than a preset CPU usage threshold and the average memory usage is less than a preset memory usage threshold, recording the duration; when the duration is less than or equal to the target interval duration, releasing the newly added freight rate search engine node.

[0100] The above-mentioned public cloud elastic capacity expansion device based on freight rate search cluster can also include a processor and a memory. The above-mentioned data aggregation unit 60, data preprocessing unit 61, recording unit 62, calculation unit 63, capacity expansion unit 64, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions.

[0101] The above processor includes a kernel, which calls the corresponding program unit from the memory. One or more kernels can be set, and the public cloud can be elastically expanded by adjusting the kernel parameters to make full use of resources.

[0102] The above-mentioned memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash memory (fl ash RAM), and the memory includes at least one storage chip.

[0103] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is further provided, the computer-readable storage medium including a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the public cloud elastic expansion method based on the freight rate search cluster according to any one of the above-mentioned embodiments 1.

[0104] According to another aspect of an embodiment of the present invention, there is further provided an electronic device, comprising one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors implement the public cloud elastic expansion method based on the freight rate search cluster according to any one of the above-mentioned embodiments one.

[0105] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the public cloud elastic expansion method based on the freight rate search cluster described in each embodiment of the present application.

[0106] The present application also provides a computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the public cloud elastic expansion method based on the freight rate search cluster described in each embodiment of the present application are implemented.

[0107] Figure 7 1 is a hardware structure block diagram of an electronic device (or mobile device) according to an embodiment of the present invention, a method for elastically expanding capacity of a public cloud based on a freight search cluster. Figure 7 As shown, the electronic device may include one or more ( Figure 7 , 702a, 702b, ..., 702n are used to illustrate) a processor (the processor may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 704 for storing data. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a keyboard, a power supply and / or a camera. A person skilled in the art can understand that Figure 7 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 7 More or fewer components as shown, or with Figure 7 Different configurations are shown.

[0108] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0109] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0110] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0111] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0112] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0113] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0114] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for elastic expansion of a public cloud based on a freight rate search cluster, characterized in that: The freight rate search cluster is deployed on the public cloud. The freight rate search cluster includes at least a data aggregation node, a data preprocessing node and a freight rate search engine node, including: In response to the freight rate search request, the data aggregation node receives the business data file, verifies the format of the business data file, and distributes the business data file with a correct format to the data preprocessing node; The data preprocessing node updates the memory mapping database according to a preset update frequency, and distributes the updated part of the memory mapping database to the freight rate search engine node; Obtaining the average CPU usage and average memory usage of the freight rate search engine node, and when the average CPU usage is greater than a preset CPU usage threshold or the average memory usage is greater than a preset memory usage threshold, recording the duration and the growth rate of the average CPU usage and the average memory usage; When the duration is longer than the target interval, the target number of expansion of the freight rate search engine node is calculated according to the growth rate; Based on the expansion target number and expansion threshold of the freight rate search engine node, a target expansion strategy is determined, and the public cloud is expanded using the target expansion strategy, wherein the target expansion strategy is one of the following: cluster expansion mode and node expansion mode.

2. The public cloud elastic expansion method according to claim 1, characterized in that: The step of verifying the format of the business data file and distributing the business data file with a correct format to the data preprocessing node comprises: Performing regular expression matching verification on the file name of the business data file; Verifying the timestamp of the business data file; When the file name and the timestamp of the business data file are verified to be correct, archiving the business data file according to the file name and the timestamp; The archived business data files are distributed to the data preprocessing nodes using a file transfer tool.

3. The public cloud elastic expansion method according to claim 1, characterized in that: The step of distributing the updated portion of the memory mapping database to the freight rate search engine node comprises: The data preprocessing node sends a first heartbeat message to the freight rate search engine node, wherein the first heartbeat message includes a timestamp and modification data of the updated memory mapping database; In the case of receiving the first heartbeat message, the data preprocessing node receives a second heartbeat message sent by the freight rate search engine node, wherein the second heartbeat message includes a timestamp and modification data of a local database of the freight rate search engine node; When there is a difference between the first heartbeat message and the second heartbeat message and the timestamp of the local database of the data preprocessing node is updated, the updated part of the memory mapping database is distributed to the freight rate search engine node.

4. The public cloud elastic expansion method according to claim 1, characterized in that: After distributing the updated part of the memory mapping database to the freight rate search engine node, the method includes: Loading the memory mapping database into the disk of the freight rate search engine node; According to the freight rate search request, different memory mapping databases are used to perform search operations.

5. The public cloud elastic expansion method according to claim 1, characterized in that: When obtaining the expansion threshold of the freight rate search engine node, it includes: Obtaining resource configurations of all the freight search engine nodes in the freight search cluster, wherein the resource configurations include at least one of the following: number of CPU cores, memory capacity, disk capacity, network bandwidth, and maximum transmission rate of the memory mapping database; Based on the resource configuration of all the freight rate search engine nodes and a preset resource configuration threshold, a capacity expansion threshold of the freight rate search cluster is calculated.

6. The public cloud elastic expansion method according to claim 1, characterized in that: The steps to determine the target expansion strategy include: When the sum of the number of the freight rate search engine nodes and the expansion target number is greater than the expansion threshold, selecting the target expansion strategy as the cluster expansion mode; When the sum of the number of the freight rate search engine nodes and the expansion target number is less than or equal to the expansion threshold, the target expansion strategy is selected as the node expansion mode.

7. The public cloud elastic expansion method according to claim 1, characterized in that: After the public cloud is expanded, the following steps are included: Obtain the average CPU usage and the average memory usage of all the freight rate search engine nodes; When the average CPU usage is less than a preset CPU usage threshold and the average memory usage is less than a preset memory usage threshold, record the duration; When the duration is less than or equal to the target interval duration, the newly added freight rate search engine node is released.

8. A public cloud elastic expansion device based on freight rate search cluster, characterized in that: The freight rate search cluster is deployed on the public cloud. The freight rate search cluster includes at least a data aggregation node, a data preprocessing node and a freight rate search engine node, including: A data aggregation unit, used to respond to the freight rate search request, receive the business data file from the data aggregation node, verify the format of the business data file, and distribute the business data file with the correct format to the data pre-processing node; A data preprocessing unit, configured to update the memory mapping database according to a preset update frequency by the data preprocessing node, and distribute the updated part of the memory mapping database to the freight rate search engine node; A recording unit, used to obtain an average CPU usage rate and an average memory usage rate of the freight rate search engine node, and when the average CPU usage rate is greater than a preset CPU usage rate threshold or the average memory usage rate is greater than a preset memory usage rate threshold, record a duration, and record a growth rate of the average CPU usage rate and the average memory usage rate; A calculation unit, configured to calculate the target number of expansion of the freight rate search engine node according to the growth rate when the duration is greater than the target interval; The expansion unit determines a target expansion strategy based on the expansion target number and expansion threshold of the freight rate search engine node, and expands the public cloud through the target expansion strategy, wherein the target expansion strategy is one of the following: cluster expansion mode and node expansion mode.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the public cloud elastic expansion method based on the freight rate search cluster according to any one of claims 1 to 7.

10. An electronic device, characterized in that: It includes one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the public cloud elastic expansion method based on the freight rate search cluster as described in any one of claims 1 to 7.

Images