Freight rate search data analysis method, system and equipment and storage medium

Abstract

The invention belongs to the technical field of passenger ticket query service, and particularly relates to a freight rate search data analysis method, system and device and a storage medium. The method comprises the steps of obtaining a flight query request; generating flight query result data according to the flight query request; verifying a flight query result data format; constructing a corresponding path expression based on the verification result; analyzing key attributes in the flight query result data by utilizing the path expression to obtain a first analysis result, and storing the first analysis result through a cache mechanism; and presenting the data in pages based on the first analysis result. The method has the capability of efficiently analyzing the JSON and XML data structures, and the core technology is that the quick matching and analysis of the data structures are realized by constructing the JSONPath and XMLPath path expressions and combining a path cache mechanism.

Description

TECHNICAL FIELD

[0001] The present disclosure belongs to the technical field of ticket query services, and particularly relates to a fare search data analysis method, system, device and storage medium. BACKGROUND

[0002] Under the background of the vigorous development of global civil aviation industry and the continuous expansion of economic globalization, air travel has become the first choice for most people on long trips, which means more routes and corresponding fare data, and more user groups, which leads to the rapid growth of the cluster size of the fare search system. After several versions of iteration, the fare search result uses JSON and XML as the response result of the query data. However, with the increase of flights, the data volume of JSON and XML also expands dramatically, and the JSON and XML data size of popular routes can easily exceed 100,000 lines. On the one hand, the complex data is difficult to locate to the core information, and the front-end display page loading time is very long.

[0003] Therefore, there is a need for a visualization method that can quickly display JSON and XML. On the one hand, it can be compatible with JSON and XML formats and display the same page structure. On the other hand, it is more convenient and efficient, provides more rich functions and smoother user experience, and users do not need to install any software, but only need to use a browser to visualize JSON and XML data anytime and anywhere. The present disclosure analyzes the JSON and XML data format fed back by the fare search engine, uses path expression to accurately locate the result, and processes the front-end and back-end in steps in parallel. By using path caching tools in combination with front-end virtual DOM or traditional AJAX, a smooth user experience is provided. SUMMARY

[0004] To solve the above problems, the present disclosure provides a fare search data analysis method, which comprises: obtaining a flight query request; generating flight query result data according to the flight query request; verifying the format of the flight query result data; constructing a corresponding path expression based on the verification result; analyzing the key attributes in the flight query result data by using the path expression to obtain a first analysis result, and storing the first analysis result through a caching mechanism; presenting the data in pages based on the first analysis result.

[0005] According to some embodiments of the present disclosure, the verification of the format of the flight query result data comprises: verifying the format of the flight query result data by using a feature recognition algorithm and processing abnormal data.

[0006] According to some embodiments of this disclosure, the data format includes JSON format and XML format; The path expressions include JSONPath expressions and XPath expressions.

[0007] According to some embodiments of this disclosure, a corresponding path expression is constructed based on the verification results, including: Based on the verification results, and combined with a predefined rule base, the corresponding path expression is dynamically constructed.

[0008] According to some embodiments of this disclosure, the caching mechanism caches the path expression and the first parsing result in key-value pairs.

[0009] According to some embodiments of this disclosure, data is presented in pages based on the first parsing result, including: Get the number of items to display as set on the front-end page; Multithreading is used to process and parse the first parsing result to obtain the second parsing result; Based on the results of the second analysis, the system is paginated according to the number of routes and the number of routes displayed, and the results of the second analysis are visualized.

[0010] This disclosure also provides a fare search data parsing system, the system comprising: The acquisition module is used to obtain flight query requests; The generation module is used to generate flight query result data based on the flight query request; The verification module is used to verify the data format of flight query results; The building module is used to construct the corresponding path expression based on the verification results; The caching module is used to parse key attributes in flight query results data using path expressions, obtain the first parsing result, and store the first parsing result through a caching mechanism. The presentation module is used to present data in pages based on the first parsing result.

[0011] According to some embodiments of this disclosure, a verification module is used to verify the data format of flight query results, including: The verification module is used to verify the data format of flight query results through feature recognition algorithms and to handle abnormal data.

[0012] According to some embodiments of this disclosure, a construction module is used to construct a corresponding path expression based on the verification result, including: The building module is used to dynamically construct the corresponding path expression based on the verification results and a predefined rule base.

[0013] According to some embodiments of this disclosure, a presentation module is used to present data in pages based on a first parsing result, including: The presentation module is used to obtain the number of items to be displayed as set on the front-end page; Multithreading is used to process and parse the first parsing result to obtain the second parsing result; Based on the results of the second analysis, the system is paginated according to the number of routes and the number of routes displayed, and the results of the second analysis are visualized.

[0014] This disclosure also provides an electronic device, including: Processor and memory; The processor invokes the computer program stored in the memory to execute the fare search data parsing method.

[0015] This disclosure also provides a computer-readable storage medium, The computer-readable storage medium stores a computer program that, when executed by a processor, enables the processor to perform the freight search data parsing method.

[0016] This disclosure has the following beneficial effects: This disclosure involves obtaining flight query requests; generating flight query result data based on the flight query requests; verifying the format of the flight query result data; constructing a corresponding path expression based on the verification result; parsing key attributes in the flight query result data using the path expression to obtain a first parsing result, and storing the first parsing result through a caching mechanism; and presenting the data in pages based on the first parsing result. The biggest feature of this technical solution is that it improves user query efficiency and user experience by processing data through caching the parsing process. At the same time, it can maximize concurrent data processing according to hardware limitations, without needing to process data sequentially or in a streaming manner. Furthermore, the existence of the caching structure can minimize the pressure on the fare search engine.

[0017] Other features and advantages of this disclosure will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the disclosure. The objects and other advantages of this disclosure may be realized and obtained by means of the structures pointed out in the description and the accompanying drawings. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This diagram illustrates a method for parsing fare search data in an embodiment of this disclosure. Figure 2 This diagram shows a detailed flowchart of the fare search data parsing method in an embodiment of this disclosure; Figure 3 This diagram illustrates a flowchart of flight cache data in an embodiment of this disclosure. Figure 4 This diagram illustrates a fare search data parsing system according to an embodiment of the present disclosure. Figure 5 A diagram of an electronic device according to an embodiment of this disclosure is shown. Detailed Implementation

[0020] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this disclosure more comprehensive and complete, and to fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a full understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more of the specific details omitted, or other methods, components, apparatus, steps, etc., can be employed. In other instances, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0021] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware units or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0022] The flowchart shown in the attached diagram is merely an illustrative example and does not necessarily include all steps. For example, some steps may be broken down, while others may be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0023] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein.

[0024] Furthermore, the terms “comprising” and “having”, and any variations thereof, are intended to cover non-exclusive inclusion, such that a process, method, system, product, or device that includes a series of steps or sub-modules is not necessarily limited to those steps or sub-modules that are explicitly listed, but may include other steps or sub-modules that are not explicitly listed or that are inherent to such process, method, product, or device.

[0025] This disclosure aims to address the problems of low efficiency and poor user experience associated with traditional data parsing methods. It possesses the ability to efficiently parse JSON and XML data structures. Its core technology lies in constructing JSONPath and XMLPath path expressions, combined with a path caching mechanism, to achieve rapid matching and parsing of data structures. In practical applications, the results data generated by the fare search system needs to be parsed and converted into a flight list format. This disclosure employs a strategy of pre-refreshing page data in the background, strictly adhering to visual pagination logic, and processing and loading data in advance. This optimization significantly reduces the time users spend waiting for data loading, effectively improving system response speed and greatly enhancing the user experience and operational smoothness when using the fare search system.

[0026] This publicly available parsing process aims to efficiently handle user flight query requests. Based on in-depth analysis of user behavior, over 90% of query requests contain only three core parameters: origin, destination, and travel date. To achieve rapid response, the system pre-caches the correspondence between origin / destination (OD) combinations, dates, and route expressions, building a foundation for fast data parsing.

[0027] like Figure 1 As shown, this disclosure provides a method for parsing fare search data, the method comprising: Get flight query request; Generate flight query results data based on the flight query request; Verify the data format of flight query results; Construct the corresponding path expression based on the verification results; The key attributes in the flight query results data are parsed using path expressions to obtain the first parsing result, and the first parsing result is stored through a caching mechanism; Data is presented in pages based on the first parsing result.

[0028] Specifically, the data format for verifying flight query results includes: The data format of flight query results is verified using a feature recognition algorithm, and abnormal data is processed.

[0029] Specifically, the data formats include JSON and XML formats; The path expressions include JSONPath expressions and XPath expressions.

[0030] Specifically, the corresponding path expression is constructed based on the verification results, including: Based on the verification results, and combined with a predefined rule base, the corresponding path expression is dynamically constructed.

[0031] Specifically, the caching mechanism caches the path expression and the first parsing result in key-value pairs.

[0032] Specifically, the data is presented in pages based on the first parsing result, including: Get the number of items to display as set on the front-end page; Multithreading is used to process and parse the first parsing result to obtain the second parsing result; Based on the results of the second analysis, the system is paginated according to the number of routes and the number of routes displayed, and the results of the second analysis are visualized.

[0033] like Figure 2 As shown, the detailed steps of this disclosure are as follows: When a user initiates a flight query request for a specific date, the fare search engine returns detailed flight data containing key fields such as airline information, ticket price, and departure and arrival times. The entire data parsing process is collaboratively completed by the data analysis module, attribute caching module, and flight caching module. Data Analysis Module: Responsible for processing the received fare search engine data, first parsing the data to determine the data type, and handling abnormal data returns.

[0034] The attribute caching module is responsible for constructing path expressions and caching all path expressions based on the parsed number of flights. The caching implementation can use a key-value data structure, such as a HashMap in Java.

[0035] Flight caching module: Responsible for parsing flight data based on path expressions. First, it parses and returns the page number data. Subsequently, it can process all data concurrently based on the page number. The caching implementation can use a key-value data structure, such as HashMap in Java.

[0036] Data format recognition and path expression construction: After receiving the returned data, the primary task of the data analysis module is to perform data format verification, accurately determining whether the data is in JSON or XML format using a feature recognition algorithm. Based on the format determination result, and combined with a predefined rule base, a path expression suitable for this data structure is dynamically constructed, laying the foundation for subsequent data extraction. Attribute caching and data parsing: The attribute caching module uses the path expressions generated by the data analysis module to perform deep parsing of flight data. During the parsing process, an LRU (Least Recently Used) caching strategy is adopted to persistently store frequently accessed parsing results. In subsequent similar queries, cached data can be directly retrieved, significantly reducing the performance loss caused by repeated parsing. Concurrent processing and pagination: The flight caching module utilizes multi-threading technology to achieve concurrent data parsing in the background based on the number of flights displayed as set by the front-end page, effectively improving data processing efficiency. After parsing, the system performs pagination according to the number of flight routes and returns the data to the front-end page for visualization through standardized interfaces, ensuring a smooth query experience for users.

[0037] The preheated data parser can provide some high-traffic flight data without further querying the fare engine, allowing for faster result display. Based on the fare search engine's caching time, the data parser's data will remain consistent with the fare search engine's returned results.

[0038] The data analysis module receives the results data returned by the fare search engine and first determines the data type. Based on the JSON or XML format, it processes the data to determine if the return is normal and handles exceptions. If the exception is that no data is found for a flight segment, the module skips the attribute caching module and directly proceeds to the flight caching module. Using OD with the date as the index, the returned result is stored as a value in the caching module.

[0039] If the request result is returned normally, it will be in JSON format. Construct a JSONPath expression. The most highlighted attributes displayed on the user's front end are flight number, airline, time, and price. Based on the JSON structure, let's assume the JSONPath expression is: , , Pagination can be achieved by looping through the Flight array. If it's in XML format, the XPATH expression is: , , The data is parsed using path expressions to obtain attribute results. The attribute cache index is JSON / XML + airline / time / price, and the value is the path expression constructed in the code. Attribute caching improves the parsing speed of JSONPath or XPath. Only about 100 attributes need to be displayed, and the module can cache all expression data. Because the result data contains a maximum of 999 flight records, the cached data can contain a maximum of [number missing]. One data entry. It only occupies about 200MB of memory.

[0040] like Figure 3As shown, in the flight caching module, path expressions are used to parse the data displayed on the front end. Following the list, the first page number and the total number of items in the list are parsed first (if it's JSON format, the JSONPath is...). The system then returns the result data to the front-end page. The front-end program determines the page number and displays the first page of data based on the data and length. The flight caching module continues to parse the result data based on the length. (Following OD...) The date is indexed, and the resulting data is cached in the module. When a user sends a pagination request, the backend only needs to query the cache to perform data slicing.

[0041] Traditional JSON and XML parsing primarily focuses on converting JSON or XML into objects or tables, without considering the data presented on the user's page. This open-source tool focuses on user-focused data, caches the parsing tool, quickly extracts the data displayed to the user, and asynchronously parses the entire dataset, significantly improving the user experience.

[0042] Example Using an actual international flight request and a domestic flight request, the specific implementation method is demonstrated step by step.

[0043] 1. User requests, the fare search system returns data, and the data is analyzed. A user initiates a request, the fare search engine returns the results data, and the data analysis module obtains the data. International flights are generally in JSON format, while domestic flights are in XML format. Assuming the user's query is for a flight from Beijing Daxing to Shanghai Pudong on October 1st, the OD (Original Data Type) is a concatenation of three-letter codes PKXPVG. Since it's a domestic flight, the data structure is XML. The data is then passed to the attribute caching module.

[0044] 2. Attribute caching module construction path expression Domestic flight data is in XML format, and path expressions begin with " / ". The system checks the attribute cache structure to see if a completed path expression already exists. Since path expressions are unaffected by data in the code implementation, caching the code object first improves query efficiency. First, the flight count is parsed; the path expression index is XMLPKXPVG1001-LENGTH. If not found, based on the XML result format of domestic flights, the XPath expression is... Insert the data into the cache data structure according to the key-value format. The flight number follows the same principle, with the index being... If no flight number data is found, the XPath expression is: You can create all expressions based on the number of flights, but since the amount of cached data is not large, you generally cache all the data directly, i.e., keep creating until... The same applies to the other attributes. The attributes of the index are all those displayed on the front-end page.

[0045] 3. Flight caching module: processes expression parsing data and caches the entire structure. The XML data is parsed using the retrieved XPath expressions. Initially, based on the user's query display interface limitations, only 8 flight records are retrieved, including airline, price, and time. Since all XPaths are independent, all queries can be performed concurrently as long as memory is not limited. After the query is complete, the data is returned to the front-end page according to the displayed structure. Simultaneously, the back-end continues parsing the flight data until all flights are parsed. The index structure for this flight is: LPKXPVG1001. Subsequent page-turning queries by the user all return results through caching, eliminating the need to query the fare search engine again.

[0046] like Figure 4 As shown, this disclosure also provides a fare search data parsing system, the system comprising: The acquisition module is used to obtain flight query requests; The generation module is used to generate flight query result data based on the flight query request; The verification module is used to verify the data format of flight query results; The building module is used to construct the corresponding path expression based on the verification results; The caching module is used to parse key attributes in flight query results data using path expressions, obtain the first parsing result, and store the first parsing result through a caching mechanism. The presentation module is used to present data in pages based on the first parsing result.

[0047] Specifically, the verification module is used to verify the data format of flight query results, including: The verification module is used to verify the data format of flight query results through feature recognition algorithms and to handle abnormal data.

[0048] Specifically, the construction module is used to construct the corresponding path expression based on the verification results, including: The building module is used to dynamically construct the corresponding path expression based on the verification results and a predefined rule base.

[0049] Specifically, the presentation module, used to present data in pages based on the first parsing result, includes: The presentation module is used to obtain the number of items to be displayed as set on the front-end page; Multithreading is used to process and parse the first parsing result to obtain the second parsing result; Based on the results of the second analysis, the system is paginated according to the number of routes and the number of routes displayed, and the results of the second analysis are visualized.

[0050] like Figure 5 As shown, corresponding to the fare search data parsing method provided above, this disclosure also provides an electronic device. Since the embodiment of this device is similar to the above method embodiment, the description is relatively simple; relevant details can be found in the description of the above method embodiment section. The device described below is merely illustrative. This device may include: a processor 1, a memory 2, a communication bus (i.e., the aforementioned device bus), and a search engine. The processor 1 and memory 2 communicate with each other via the communication bus and communicate with external systems via a communication interface. The processor 1 can call logical instructions in the memory 2 to execute the fare search data parsing method.

[0051] Furthermore, the logical instructions in the aforementioned memory 2 can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as memory chips, USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0052] On the other hand, this disclosure also provides a processor-readable storage medium storing a computer program 3, which, when executed by a processor 1, is implemented to perform the fare search data parsing method provided in the above embodiments.

[0053] The processor-readable storage medium can be any available medium or data storage device that the processor 1 can access, including but not limited to magnetic memory (e.g., floppy disk, hard disk, magnetic tape, magneto-optical disk (MO)), optical memory (e.g., CD, DVD, BD, HVD), and semiconductor memory (e.g., ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)).

[0054] Embodiments of this disclosure also provide a computer program product comprising a computer program that includes computer program code means stored on a computer-readable medium or carrier wave, the computer program code means being configured to cause a computer or processor to control the execution of steps of a method according to any embodiment of this disclosure.

[0055] Those skilled in the art should understand that, despite the detailed description of this disclosure with reference to the foregoing embodiments, modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure.

Claims

1. A rate search data parsing method characterized by comprising: The method comprises: acquiring a flight query request; generating flight query result data according to the flight query request; verifying the format of the flight query result data; constructing a corresponding path expression based on the verification result; parsing key attributes in the flight query result data using the path expression to obtain a first parsing result, and storing the first parsing result through a caching mechanism; presenting data based on the first parsing result.

2. The fare search data parsing method according to claim 1, wherein the verification of the format of the flight query result data comprises: verifying the format of the flight query result data through a feature recognition algorithm, and processing abnormal data.

3. The fare search data parsing method according to claim 2, wherein the data format comprises a JSON format and an XML format; the path expression comprises a JSONPath expression and an XPath expression.

4. The fare search data parsing method according to claim 3, wherein constructing the corresponding path expression based on the verification result comprises: based on the verification result, dynamically constructing the corresponding path expression in combination with a predefined rule library.

5. The fare search data parsing method according to claim 1, wherein the caching mechanism caches the path expression and the first parsing result in the form of a key-value pair.

6. The fare search data parsing method according to claim 1, wherein presenting data based on the first parsing result comprises: acquiring a number of display items set on a front-end page; processing the first parsing result using a multi-threading technology to obtain a second parsing result; based on the second parsing result, performing page processing according to the number of flight routes and the number of display items, and visually displaying the second parsing result.

7. A fare search data parsing system characterized by comprising: The system comprises: an acquisition module configured to acquire a flight query request; a generation module configured to generate flight query result data according to the flight query request; a verification module configured to verify the format of the flight query result data; a construction module configured to construct a corresponding path expression based on the verification result; a caching module configured to parse key attributes in the flight query result data using the path expression to obtain a first parsing result, and store the first parsing result through a caching mechanism; a presentation module configured to present data based on the first parsing result.

8. The fare search data parsing system according to claim 7, wherein the verification module is configured to verify the format of the flight query result data, comprising: the verification module is configured to verify the format of the flight query result data through a feature recognition algorithm, and process abnormal data.

9. The fare search data parsing system according to claim 7, wherein the construction module is configured to construct the corresponding path expression based on the verification result, comprising: the construction module is configured to dynamically construct the corresponding path expression in combination with a predefined rule library based on the verification result.

10. The fare search data parsing system according to claim 7, wherein the presentation module is configured to present data based on the first parsing result, comprising: the presentation module is configured to acquire a number of display items set on a front-end page. The first analysis result is processed by using a multi-thread technology to obtain a second analysis result. Based on the second analysis result, the second analysis result is visualized and displayed by page processing according to the number of air routes and the number of display lines.

11. An electronic device, comprising: It comprises: A processor and a memory; The processor calls the computer program stored in the memory to execute the airfare search data analysis method in any one of claims 1-6. 12.A computer readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor can execute the airfare search data analysis method in any one of claims 1-6.

Images