Overtime self-checking and self-adjusting dynamic response method and device of freight rate searching system

Abstract

The invention belongs to the technical field of networks, and discloses a timeout self-checking and self-adjusting dynamic response method and device for a freight rate search system, and the method comprises the steps: obtaining a user query request, dispersing the user query request to a plurality of servers through a load balancing server, and carrying out the matching of a working mode; obtaining a search result of the user query request through pre-verification setting, quick cache search and dynamic resource allocation; during the operation period of the freight rate searching system, overtime events are adjusted and analyzed through heartbeat monitoring and overtime alarm. According to the invention, the system performance and the user request processing condition can be monitored in real time, the system resource allocation and request processing strategy can be automatically adjusted, the resource utilization rate is optimized, and the system operation cost is reduced.

Description

Technical Field

[0001] The invention belongs to the field of network technology, and in particular relates to a timeout self-checking and self-adjusting dynamic response method and device for a freight rate search system. Background Art

[0002] In the context of a globalized economy, the international civil aviation passenger transport industry is particularly important. As an important bridge between passenger service providers and customers, the performance of the fare search system directly affects the user experience and ticketing efficiency. Currently, a major challenge facing the international fare search system is processing a large number of data requests, high-dimensional query combinations, and complex pricing rules. These factors together lead to a sharp increase in the computing power required to process requests, especially during peak hours. System timeouts have become a key factor affecting user experience.

[0003] The biggest shortcoming of the existing technology is that it mostly takes post-processing remedial measures to deal with timeout problems, rather than preventing timeouts from happening fundamentally. In most cases, the measures taken after the system times out include request retry, increasing server resources, etc. These measures not only increase costs, but also fail to fundamentally solve the problem, especially when the system design does not take into account the dynamic response adjustment mechanism, and its effect is even more limited. Summary of the invention

[0004] In view of the deficiencies of the prior art, the present invention proposes a timeout self-checking and self-adjusting dynamic response method and device for a freight rate search system.

[0005] The present invention is achieved through the following technical solutions:

[0006] In a first aspect, a method for dynamic response of overtime quality inspection self-adjustment of a freight rate search system includes:

[0007] Obtain user query requests, distribute them to multiple servers through the load balancing server, and match the working mode;

[0008] Obtaining search results for the user query request through pre-checking settings, fast cache search, and dynamic resource allocation;

[0009] During the operation of the freight rate search system, timeout events are adjusted and analyzed through heartbeat monitoring and timeout alarms.

[0010] In some embodiments, when the heartbeat monitoring detects that the heartbeat stops, the service is automatically restarted through a script;

[0011] The timeout alarm provides relevant log materials and timeout information records for timeout time analysis. By conducting in-depth analysis of timeout events, timed requests are found, and the maximum response time is automatically added to similar requests to limit concurrent calls of similar requests.

[0012] In some embodiments, the user query request includes the number of passenger types, the origin, the destination and the departure time, and when a round trip is involved, also includes return trip information.

[0013] In some embodiments, the user query request is distributed to multiple servers through a load balancing server, and the load balancing server adopts a microservice architecture.

[0014] In some embodiments, the pre-verification settings include fare pre-verification and flight pre-verification.

[0015] In some embodiments, the fast cache search returns pre-retained search results to the user;

[0016] If there is no pre-reserved search result, cloud resources are called through the dynamic resource allocation.

[0017] In some embodiments, data preprocessing is also included, and the data includes preprocessable information extracted from multiple real-time data such as position data, freight rate data, flight data, and location exchange rate.

[0018] In a second aspect, a timeout self-checking and self-adjusting dynamic response device of a freight rate search system is applied to an electronic device, specifically comprising:

[0019] Pre-event preparation module, in-event processing module and post-event adjustment module;

[0020] The pre-preparation module includes dynamic user-centric services and load balancing, which are used to allocate servers and match working modes for user query requests before searching;

[0021] The in-process processing module includes pre-checking, fast cache search and dynamic resource allocation, which is used to provide search results for user query requests;

[0022] The post-adjustment module includes a timeout event analysis, which is used to analyze and adjust the user's timeout request;

[0023] In some embodiments, the dynamic user-centric service is used to confirm the definition and commitment of response time in the service level agreement and enable the working mode configuration for the user;

[0024] The load balancing server adopts a microservice architecture to improve the scalability and independence of modules of the system by distributing query requests to multiple servers through the use of a load balancing server;

[0025] The pre-verification is used to pre-verify the freight rate rules and pre-verify the route map, process the freight rate rule relationship and the route path in advance, and save real-time search time;

[0026] The fast cache search is used to poll user requests when the application node of the international freight rate search system is idle, cache the results in advance and set the update time, and wait for the user's real-time query to trigger the cache to directly return the search results;

[0027] The dynamic resource allocation is used to dynamically adjust cloud resources according to real-time traffic and system load;

[0028] The timeout event analysis is used to conduct an in-depth analysis of the timeout event, find out the timeout request, and automatically add a maximum response time to similar requests to limit concurrent calls of similar requests.

[0029] Compared with the prior art, the present invention has the following advantages:

[0030] 1. It can monitor system performance and user request processing in real time, and automatically adjust system resource allocation and request processing strategies;

[0031] 2. Reduce the equipment cost for solving system timeout problems and optimize resource utilization;

[0032] 3. Through the dynamic response device, the system can automatically perform performance tuning and resource reallocation when facing high concurrent requests, fundamentally reducing the probability of timeouts.

[0033] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or be understood by implementing the present use information. The purpose and other advantages of the present invention can be realized and obtained by the structures pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0035] Figure 1 It is a schematic diagram of a method for dynamic response of a freight rate search system to self-check and adjust itself overtime according to an embodiment of the present application;

[0036] Figure 2 This is a flow chart of a method for dynamic response of a freight rate search system to self-check and adjust itself overtime according to an embodiment of the present application;

[0037] Figure 3 It is a schematic diagram of a timeout self-checking and self-adjusting dynamic response device of a freight rate search system in an embodiment of the present application. DETAILED DESCRIPTION

[0038] The drawings are only for illustrative purposes and cannot be construed as limiting the present invention. To better illustrate the present embodiment, some parts of the drawings may be omitted, enlarged, or reduced, and do not represent the size of the actual product. For those skilled in the art, it is understandable that some well-known structures and their descriptions may be omitted in the drawings. The positional relationships described in the drawings are only for illustrative purposes and cannot be construed as limiting the present invention.

[0039] like Figure 1 The method for dynamic response of a timeout self-checking and self-adjusting of a freight rate search system shown in the figure comprises the following steps:

[0040] S1. The international freight rate search system background obtains user query requests, distributes user query requests to multiple servers through the load balancing server and matches the working mode.

[0041] Specifically, the user query request content includes the number of passenger types, origin, destination and departure time. If a round trip is involved, it also includes return information. The working mode includes Session mode and non-Session mode. The user transmits the information to be queried to the background through the international freight rate search system interface API, and the query request enters the load balancing server to determine whether the background has idle resources to calculate the itinerary and fare results in real time.

[0042] Furthermore, the load balancing server adopts a microservice architecture to improve the scalability of the international freight rate search system and the independence between modules, and distributes the query requests to multiple servers through the load balancing server, thereby avoiding overloading of a single international freight rate search application node and ensuring even distribution of query requests.

[0043] S2. Obtain search results of the user query request through pre-checking settings, fast cache search and dynamic resource allocation.

[0044] Specifically, Figure 2 In the timeout self-checking and self-adjusting dynamic response process of the freight rate search system shown, the load balancing server determines whether there are idle resources in the background, and the idle resources are the application nodes of the international freight rate search system. If there are idle resources, the international freight rate search system will push the query request to the international freight rate search application node for search, and at the same time, the freight rate rules and route maps will be processed in advance through the pre-check setting and matched and attached to the corresponding search results and returned to the user to save search time; when there are no idle resources and the queue is full, the query request will be pushed to the fast cache search, and it will be checked whether there are pre-reserved search results. If so, the cached common historical results will be directly returned to the user. If there are no relevant records, the cloud cluster will be called through dynamic resource allocation to trigger the international freight rate search system application node on the public cloud to return the search results to the user.

[0045] Furthermore, the process of obtaining the search results of the user query request also includes that before the user sends a real-time request to call the international freight rate search system, the background has already received position data, freight rate data, flight data, location exchange rate and other pre-processable information extracted from the real-time data from the external system and performed relevant pre-processing operations. When the user requests to call the international freight rate search system, the background system will not need to recalculate the association relationship of the search data. The system can directly extract the information corresponding to its address pointer from the memory and attach the matching information to the itinerary result and return it to the user.

[0046] S3. During the operation of the freight rate search system, timeout events are adjusted and analyzed through heartbeat monitoring and timeout alarms.

[0047] Specifically, heartbeat monitoring and timeout alarms detect the status of the system in real time. For example, if the heartbeat stops, the freight rate search process may stop, and a script is needed to automatically restart the service. The timeout alarm provides relevant log materials and timeout information records for timeout event analysis. By conducting in-depth analysis of timeout events, timed requests are found, and the maximum response time is automatically added to similar requests to limit concurrent calls of similar requests.

[0048] like Figure 3 The timeout self-checking and self-adjusting dynamic response device of the freight rate search system shown can be understood as the freight rate search system software, a computer, server or cloud platform with data computing and information processing capabilities, including: a pre-preparation module, an in-process processing module and a post-adjustment module.

[0049] The pre-preparation module includes a dynamic user center service and a load balancing server, which is used to allocate servers and match working modes for user query requests before searching;

[0050] The in-process processing module includes pre-checking, fast cache search and dynamic resource allocation, which is used to provide search results for user query requests;

[0051] The post-adjustment module includes a timeout event analysis for analyzing and adjusting a user's timeout request; further, the dynamic user center service is used to confirm the definition and commitment of the response time in the service level agreement and enable the working mode configuration for the user.

[0052] Specifically, when the user's query request enters the background engine, the working mode matching is first performed. The configuration information used for matching is maintained by the administrator through the central console UI. When matching the working mode, first find the corresponding matching item from the database based on the information contained in the json code of the query request, such as the query party, query area, etc., and switch to the corresponding mode. The main modes are divided into session and non-session modes. The session mode means that the user's client can call the system twice. After the first call, the system will quickly return a response containing a small number of results, and at the same time carry a session id to the client. When the client requests the background with the session id for the second time, the central console returns all the results. Therefore, this mode can reduce the waiting time for users to obtain the first batch of data, but this method requires the client, that is, the airline or agent, to implement two calls, which involves the modification of the airline or agent's client code. The non-session mode can achieve seamless docking with the existing client code and improve the response speed without modifying the client code. The principle is that the background engine concurrently calls the international freight rate search system application node, thereby changing the query pressure from relying entirely on one international freight rate search system application node to multiple international freight rate search system application nodes completing part of the query. In the batch mode of the session, the central console uses distributed cache Redis to cache intermediate results to improve the overall speed, and the engine sends the internal processing process to the Kafka message bus in the form of messages for subsequent auxiliary systems to process.

[0053] The load balancing server adopts a microservice architecture to improve the scalability of the system and the independence between modules. By using the load balancing server to distribute query requests to multiple servers, overload of a single node is avoided and even distribution of requests is ensured.

[0054] Specifically, the load balancing technology ensures the high performance and stability of the system through the dual strategies of the application layer and the data layer. At the application layer, the system adopts a multi-node multi-process cluster based on load balancing to achieve dynamic and linear expansion of business processing capabilities. At the same time, it ensures that single point failure will not cause business interruption through multi-layer parallel architecture and hardware-level load balancing. At the data layer, the full memory cache mechanism greatly improves the data access speed, and the incremental data distribution ensures the scalability of the system and the high availability of data update steps. In addition, the data update node adopts HA high availability backup (High Availability Classer) to enhance the flexibility of data recovery, so that the dynamic response device of the present invention can effectively respond to high concurrent requests, significantly improve the system's processing capacity and user's search experience, and provide strong support for the international freight rate search system.

[0055] The pre-verification is used to pre-verify the freight rate rules and pre-verify the route map, process the rule relationship and the route path in advance, and save real-time search time.

[0056] Specifically, whether a fare can be used for sale depends on a series of fare rules. The original matching logic stipulated in the industry starts with the fare, matches the corresponding fare record according to the starting point, travel date and other information determined by the user, and then matches the unique basic record 1. Then, the classification record 2 is matched according to the fare record and the information of the basic record 1. Tens of thousands of fare rules exist in each detailed record 3. The fare is available only after each fare rule is verified. However, there are hundreds of millions of fare rules. Freight usually reuses rules, and the corresponding rule data is only a few million. The data volume of the basic record 1 is much smaller than the number of fare. Therefore, based on performance considerations, based on user requests, basic record 1 is used as the entry, and all available fare rules are filtered to achieve pre-verification of the fare.

[0057] Furthermore, when users search for air tickets, there are no direct flights between some departure points and destinations, so flights need to be set up, and flight setting can be processed in advance. The construction process is roughly as follows: take a city as a point, get a set of flights with the city as the departure point and destination, and then expand the list to the left or right to form a flight setting database. In the construction process, it is necessary to consider factors such as detour rate, mileage, and flight carrying capacity of the travel code route year. The available flight combinations are pre-processed in advance to prepare for the purpose of quickly mounting flights when users query the connection results, thereby completing flight pre-verification.

[0058] The fast cache search is used to poll user requests when the application node of the international freight rate search system is idle, cache the results in advance and set the update time, and wait for the user's real-time query to trigger the cache to directly return the results.

[0059] Specifically, the most used user request template among the dynamic user request templates generated by the main process is used as the analysis basis. After analyzing the popular origins, destinations, and popular dates, a request that needs to be polled is generated. The highest priority of the polling request template is the template configured by the user, followed by the most used template in the main process. Then, according to the configuration, the second most used template, the third most used template, and so on are added. In addition, check the cache timestamp. For the cache of the same origin, destination, popular date, and template, if the validity period exceeds the maximum polling time limit configured by the user, the request will not be polled. If there is no such item in the cache, or the cache validity period is within the maximum polling time limit configured by the user, the international freight rate search system will be queried in real time to update the cache and the cache timestamp.

[0060] The dynamic resource allocation is used to dynamically adjust cloud resources according to real-time traffic and system load, such as automatically expanding the number of servers to cope with high requests and elastically expanding the application nodes of the international search system.

[0061] Specifically, in some embodiments, the database file in the cloud deployment has been optimized to about 40G, which can be fully loaded into the memory (128G). Even during the switching between the old and new database files, the new and old data can be fully loaded into the memory. Every time a new database file is mmap'ed into the memory, the entire virtual memory space needs to be traversed sequentially to ensure that all data is read into the memory. Since it is a sequential read, the random access performance index TPS of the hard disk can be ignored, and the hard disk throughput index is mainly considered. The following table takes 40G data as an example to illustrate the impact of different hard disk performance on network transmission and data reading. The international freight rate search system does not have high requirements for data delay (1 hour). Selecting a hard disk / cloud disk with a throughput of more than 400MB / S can meet system requirements.

[0062] The timeout event analysis is used to conduct an in-depth analysis of the timeout event, find out the timeout request, and automatically add a maximum response time to similar requests to limit concurrent calls of similar requests.

[0063] Specifically, according to the current status of user usage, the system will dynamically formulate the maximum TPS and maximum response time of users. The system collects user activity information through user behavior data, log management system and user sales promotion statistics tools. The maximum TPS and response time data of users are collected on an annual basis, and then data analysis is performed on the collected data to calculate the saturated call concurrency and the maximum tolerable response time value of users. At the same time, the user center is open to the outside world so that users can adjust the restrictions in real time. However, the expansion of application resources and services cannot exceed 4 times the original value. If there are special circumstances, it is necessary to set higher permissions in the background to override the maximum TPS and response time values ​​of users. The default initial value priority order is that the background configuration permission value is the highest, followed by the user request value, and finally the dynamic value of data analysis. When a timeout request is detected by the system, an integer second will be determined by the 2-point approximation method, and the background configuration permission will be automatically adjusted directly, and then the reduced value will be released after investigation. Under normal circumstances, the background configuration permission value is empty. Once a timeout occurs, or the application resource business expansion reaches more than 4 times the original value, the value will be set to a maximum integer second without timeout or 4 times the original value.

[0064] Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent substitutions for some of the technical features therein; and these modifications or substitutions do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A timeout self-checking and self-adjusting dynamic response method for a freight rate search system, characterized in that: include: Obtain user query requests, distribute them to multiple servers, and match the working modes; Obtaining search results for the user query request through pre-checking settings, fast cache search, and dynamic resource allocation; During the operation of the freight rate search system, timeout events are adjusted and analyzed through heartbeat monitoring and timeout alarms; When the heartbeat monitoring detects that the heartbeat stops, the service is automatically restarted through a script; The timeout alarm analyzes the timeout event, finds the timeout request, and automatically adds the maximum response time to similar requests to limit the concurrent calls of similar requests.

2. The method according to claim 1, characterized in that The user query request includes the number of passenger types, origin, destination and departure time.

3. The method according to claim 1, characterized in that The user query request is distributed to multiple servers through a load balancing server, and the load balancing server adopts a microservice architecture.

4. The method according to claim 1, characterized in that The pre-verification settings include freight rate pre-verification and flight pre-verification.

5. The method according to claim 1, characterized in that The fast cache search returns the pre-reserved search results to the user; If there is no pre-reserved search result, cloud resources are called through the dynamic resource allocation.

6. The method according to claim 1, characterized in that Obtaining the search results of the user query request also includes data preprocessing, where the data includes preprocessable information extracted from real-time data.

7. A timeout self-checking and self-adjusting dynamic response device for a freight rate search system, applied to electronic equipment, characterized in that: include: Pre-event preparation module, in-event processing module and post-event adjustment module; The pre-preparation module includes a dynamic user center service and a load balancing server, which is used to allocate servers and match working modes for user query requests before searching; The in-process processing module includes pre-checking, fast cache search and dynamic resource allocation, which is used to provide search results for user query requests; The post-adjustment module includes a timeout event analysis module, which is used to analyze and adjust the user's timeout request.

8. The device according to claim 7, characterized in that The dynamic user center service is used to confirm the definition and commitment of response time in the service level agreement and enable the working mode configuration for the user; The load balancing server adopts a microservice architecture to improve the scalability and inter-module independence of the system, and distributes user query requests to multiple servers by using a load balancing server; The pre-verification is used to pre-verify the freight rate rules and pre-verify the route map, process the freight rate rule relationship and the route path in advance, and save real-time search time; The fast cache search is used to poll user query requests when the application node of the international freight rate search system is idle, cache the query results in advance and set the update time, and wait for the user's real-time query to trigger the cache to directly return the search results; The dynamic resource allocation is used to dynamically adjust cloud resources according to real-time traffic and system load; The timeout event analysis is used to conduct an in-depth analysis of the timeout event, find out the timeout request, and automatically add a maximum response time to similar requests to limit concurrent calls of similar requests.

9. The device according to claim 8, characterized in that The working mode configuration includes session mode and non-session mode.

10. The device according to claim 8, characterized in that The timeout event analysis module performs analysis based on the user's maximum TPS and maximum response time data.

Images