Fare search data collection and statistical method and system in domestic hybrid cloud environment

Abstract

The application discloses a kind of domestic mixed cloud environment under the search data collection and statistical method and system of freight rate, belong to ticket query service field, including: in public cloud gateway layer, receive multi-source search data of freight rate, and the multi-source search data of freight rate is divided into core data and non-core data;In cross-cloud collaboration layer, the core data and the non-core data are respectively transmitted to private cloud business layer using differentiated transmission strategy by function component;In private cloud business layer, the transmitted data is decrypted, and the internal running data of private cloud side is collected by function component, the decrypted transmission data and the internal running data of private cloud side are converged, the statistical processing is carried out to all data after convergence, and the statistical index is stored to domestic memory database, and the detail data generated by business operation is forwarded to domestic time series database for aggregated storage.Five major pain points of insufficient domestic freight rate system, slow transmission, insecurity, rough statistics and high cost are solved.

Description

Technical Field

[0001] This application belongs to the field of passenger ticket inquiry services, and specifically relates to a method and system for collecting and statistically analyzing fare search data in a domestic hybrid cloud environment. Background Technology

[0002] With the rapid development of the aviation industry, the real-time nature and accuracy of fare information are crucial for corporate decision-making and user experience. Fare search systems need to process massive amounts of data from multiple channels. This data is characterized by diverse sources, heterogeneous formats, and frequent updates, posing significant challenges to data collection, storage, and analysis. Currently, most fare search systems adopt traditional centralized data processing architectures or solutions based on foreign cloud platforms, which suffer from technical deficiencies such as insufficient localization adaptation, low data synchronization efficiency, poor security and compliance, and insufficient scalability. Summary of the Invention

[0003] To address the shortcomings of existing fare search systems, such as insufficient localization (reliance on foreign components), low cross-cloud transmission efficiency (full synchronization consumes significant bandwidth), imbalance between security and real-time performance (encryption reduces efficiency), and limited statistical dimensions (lack of real-time-offline collaboration), this application provides a method and system for collecting and statistically analyzing fare search data in a domestic hybrid cloud environment. Through the implementation of "security + real-time + scalability" technologies, it solves the pain points of "difficult data collection, weak security, and slow statistics" in domestic hybrid cloud environments. This not only meets the requirements of government and enterprises for "autonomy and controllability" but also provides accurate and real-time data support for fare search systems.

[0004] This application provides a method for collecting and statistically analyzing fare search data in a domestically developed hybrid cloud environment, including: At the public cloud gateway layer, multi-source fare search data is received and divided into core data and non-core data. In the cross-cloud collaboration layer, the core data and the non-core data are transmitted to the private cloud business layer using differentiated transmission strategies through functional components. At the private cloud business layer, the transmitted data is decrypted, and the internal operation data of the private cloud side is collected through functional components. The decrypted transmitted data and the internal operation data of the private cloud side are aggregated. All the aggregated data are statistically processed to generate statistical indicators and stored in a domestic in-memory database. At the same time, the detailed data generated by the business operation is forwarded to a domestic time-series database for aggregation and storage.

[0005] Optionally, the multi-source freight rate search data can be categorized into core data and non-core data based on their sensitivity, including: Data containing airline fare strategies, user payment information, and undisclosed cabin inventory will be classified as encrypted transmission-level core data. Data containing publicly available route lists, historical query statistics, and user geographic distribution is classified as non-core data for regular transmission.

[0006] Optionally, the functional components include a cloud node adaptation component, a national cryptographic encryption transmission component, a multi-source data acquisition component, and a real-time statistical processing component; The cloud node adaptation component is deployed in the cross-cloud collaboration layer and is used to select the corresponding transmission channel and processing method according to whether the data is sensitive. The national cryptographic encryption transmission component is deployed in the private cloud service layer for encryption; The multi-source data acquisition component is deployed in the private cloud service layer and is used to collect API data and IoT device data and send them. The real-time statistical processing component is deployed in the private cloud business layer and calculates statistical indicators based on a domestically developed stream computing framework.

[0007] Optionally, the differentiated transmission strategy for the core data includes: The core data is transmitted using symmetric encryption algorithm via a hybrid cloud dedicated line. The file block comparison tool is called to calculate the hash values ​​of local and cloud data blocks, and only the data blocks with inconsistent hash values ​​are transmitted; Each transmission operation log is written to a domestic blockchain service in real time to generate an immutable evidence hash.

[0008] Optionally, the differentiated transmission strategy for the non-core data includes: The non-core data is transmitted using a publish-subscribe pattern from a domestically developed message queue. Enable secure communication protocol encryption and configure cluster redundancy with at least three nodes.

[0009] Optional, The step of statistically processing all the aggregated data to generate statistical indicators and storing them in a domestic in-memory database includes: The real-time statistical processing component performs real-time statistics on all aggregated data based on a domestic stream computing framework, generating statistical indicators including freight rate volatility, API call success rate, TPS, and traffic volume. The statistical indicators are written into a domestic in-memory database in key-value pairs for real-time display of high-frequency data.

[0010] Optionally, forwarding the detailed data generated by the business operation to a domestic time-series database for aggregation and storage includes: The detailed data generated by business operations is written to a local log file, and the local log file is parsed and fields are extracted by a domestic log service. After being aggregated by route and time dimensions, the data is written to a domestic time-series database.

[0011] Optional, After receiving multi-source fare search data, and before dividing the multi-source fare search data into core data and non-core data, the process also includes: verifying the multi-source fare search data. Verify the request signature using a cryptographic hash algorithm to prevent data tampering; Limit the frequency of requests from a single IP address to no more than a preset threshold; Return an error code for a format error request.

[0012] Based on the same inventive concept, this application also provides a domestic hybrid cloud environment-based fare search data collection and statistics system, including: The public cloud gateway layer is used to receive multi-source fare search data and divide the multi-source fare search data into core data and non-core data; The cross-cloud collaboration layer is used to transmit the core data and the non-core data to the private cloud service layer by adopting differentiated transmission strategies through functional components. The private cloud business layer is used to decrypt the transmitted data and collect internal operating data of the private cloud through functional components. It aggregates the decrypted transmitted data with the internal operating data of the private cloud, performs statistical processing on all aggregated data, generates statistical indicators and stores them in a domestic in-memory database, and forwards the detailed data generated by business operations to a domestic time-series database for aggregation and storage.

[0013] Furthermore, this application also provides an electronic device, comprising: at least one processor and a memory; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, a method for collecting and statistically analyzing freight rate search data in a domestic hybrid cloud environment, as described above, is implemented.

[0014] Furthermore, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed, implements the above-described method for collecting and statistically analyzing freight rate search data in a domestic hybrid cloud environment.

[0015] Compared with the prior art, this application has the following advantages: This application provides a method and system for collecting and statistically analyzing fare search data in a domestically developed hybrid cloud environment, comprising: receiving multi-source fare search data at the public cloud gateway layer and dividing the multi-source fare search data into core data and non-core data; transmitting the core data and the non-core data to the private cloud business layer using differentiated transmission strategies through functional components at the cross-cloud collaboration layer; decrypting the transmitted data at the private cloud business layer and collecting internal operating data on the private cloud side through functional components; aggregating the decrypted transmitted data and the internal operating data on the private cloud side; performing statistical processing on all the aggregated data; generating statistical indicators and storing them in a domestically developed in-memory database; and forwarding the detailed data generated by business operations to a domestically developed time-series database for aggregated storage.

[0016] By utilizing a three-tiered domestic architecture of public cloud, cross-cloud, and private cloud, along with code dependencies for functional components and a full-process technical design model, the advantages of this application solution are not due to a single technological breakthrough, but rather to a combination of domestic component collaboration, hierarchical efficiency design, secure and compliant implementation, closed-loop statistical capabilities, and controllable cost optimization. This approach precisely addresses the five core pain points of air logistics freight rate systems: insufficient domestic production, slow transmission, insecurity, coarse statistics, and high costs.

[0017] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures pointed out in the description, claims and drawings. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This paper illustrates a flowchart of the method for collecting and statistically analyzing fare search data in a domestically developed hybrid cloud environment provided in this application. Figure 2 This application provides a flowchart of the public cloud gateway layer. Figure 3 The flowchart of the cross-cloud collaboration layer provided in this application is shown; Figure 4 The diagram illustrates the architecture of the domestic hybrid cloud environment for collecting and statistically analyzing freight rate search data provided in this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] Example 1 This application provides a method for collecting and statistically analyzing fare search data in a domestically developed hybrid cloud environment, such as... Figure 1 ,include: At the public cloud gateway layer, multi-source fare search data is received and divided into core data and non-core data. In the cross-cloud collaboration layer, the core data and the non-core data are transmitted to the private cloud business layer using differentiated transmission strategies through functional components. At the private cloud business layer, the transmitted data is decrypted, and the internal operation data of the private cloud side is collected through functional components. The decrypted transmitted data and the internal operation data of the private cloud side are aggregated. All the aggregated data are statistically processed to generate statistical indicators and stored in a domestic in-memory database. At the same time, the detailed data generated by the business operation is forwarded to a domestic time-series database for aggregation and storage.

[0022] Step S1: The flowchart of the public cloud gateway layer is as follows: Figure 2 Multi-source fare search data is accessed through the gateway in XML / JSON format. After verification, data that fails verification is blocked and returned a 400 error, while data that passes verification is entered into a sensitivity level determination process. Combined with the transmission of domestic NATS (distributed) message queues, multi-source fare search data is divided into two sensitivity levels: LEVEL1 encrypted transmission level core data and LEVEL2 conventional transmission level non-core data.

[0023] Optionally, multi-source fare search data includes airline fare interface data and logistics data source data accessed through domestic API gateways.

[0024] Step S2: Optionally, after receiving the multi-source fare search data and before dividing the multi-source fare search data into core data and non-core data, the method further includes: verifying the multi-source fare search data; The request signature is verified using a cryptographic hash algorithm to prevent data tampering; the cryptographic hash algorithm is the SM3 algorithm. The frequency of requests from a single IP address is limited to a preset threshold. In this embodiment, the preset threshold is 100 times / second to block malicious order-brushing and web crawling requests. The request returned error code 400 for a format error.

[0025] Deploy the domestic version of NATS message queue to receive data and transmit "regular transmission level" non-core data, supporting millions of TPS (transactions per second) concurrency; enable JetStream (message persistence component) to achieve message persistence, avoid data loss due to network interruption.

[0026] Optionally, the multi-source freight rate search data can be categorized into core data and non-core data based on their sensitivity, including: Data containing airline fare strategies, user payment information, and undisclosed cabin inventory will be classified as encrypted transmission-level core data. Data containing publicly available route lists, historical query statistics, and user geographic distribution is classified as non-core data for regular transmission.

[0027] Optionally, the functional components include a cloud node adaptation component, a national cryptographic encryption transmission component, a multi-source data acquisition component, and a real-time statistical processing component; The cloud node adaptation component is deployed in the cross-cloud collaboration layer and is used to select the corresponding transmission channel and processing method according to whether the data is sensitive. The national cryptographic encryption transmission component is deployed in the private cloud service layer for encryption; The multi-source data acquisition component is deployed in the private cloud service layer and is used to collect API data and IoT device data and send them. The real-time statistical processing component is deployed in the private cloud business layer and calculates statistical indicators based on a domestically developed stream computing framework.

[0028] Cross-cloud collaboration layer flowchart as follows Figure 3 After sensitivity level assessment, core data at the LEVEL1 encrypted transmission level and non-core data at the LEVEL2 regular transmission level employ differentiated transmission strategies at the cross-cloud collaboration layer: LEVEL1 data is transmitted via China Telecom Cloud hybrid cloud dedicated line, and after comparing the differences in SHA256 hash values ​​calculated by a file block comparison tool, it is transmitted using a symmetric encryption algorithm (SM4 national cryptographic algorithm); LEVEL2 data is transmitted via the domestic version of the NATS message queue, with Transport Layer Security Protocol (TLS1.3) encryption enabled, in a publish-subscribe mode; both types of data are audited and synchronized with cloud logs before being aggregated to the private cloud business layer.

[0029] Enable NATS's TLS 1.3 encryption function, with the certificate issued by Huawei Cloud SSL Certificate Management Service to ensure that data is not leaked during transmission; configure NATS cluster redundancy (3 nodes) to avoid data transmission interruption due to single point of failure.

[0030] Each transmission operation log (including transmission time, data volume, sender IP, and encryption algorithm) is written to the China Telecom Cloud Blockchain Service in real time, generating an immutable evidence hash to meet audit and traceability requirements.

[0031] Optionally, the differentiated transmission strategy for the core data includes: The core data is transmitted using symmetric encryption algorithm via a hybrid cloud dedicated line. The file block comparison tool is called to calculate the hash values ​​of local and cloud data blocks, and only the data blocks with inconsistent hash values ​​are transmitted; Each transmission operation log is written to a domestic blockchain service in real time to generate an immutable evidence hash.

[0032] Optionally, the differentiated transmission strategy for the non-core data includes: The non-core data is transmitted using a publish-subscribe pattern from a domestically developed message queue. Enable secure communication protocol encryption and configure cluster redundancy with at least three nodes.

[0033] Step S3: Optionally, the step of performing statistical processing on all aggregated data to generate statistical indicators and storing them in a domestic in-memory database includes: The real-time statistical processing component performs real-time statistics on all aggregated data based on a domestic stream computing framework, generating statistical indicators including freight rate volatility, API call success rate, TPS, and traffic volume. The statistical indicators are written into a domestic in-memory database in key-value pairs for real-time display of high-frequency data.

[0034] Optionally, forwarding the detailed data generated by the business operation to a domestic time-series database for aggregation and storage includes: The detailed data generated by business operations is written to a local log file, and the local log file is parsed and fields are extracted by a domestic log service. After being aggregated by route and time dimensions, the data is written to a domestic time-series database.

[0035] The private cloud service layer is implemented using a dual data source access + layered visualization solution. This embodiment is based entirely on Huawei Cloud ecosystem components to ensure domestic compatibility and data flow efficiency.

[0036] Real-time data is stored in Huawei Cloud GaussDB (for the domestic in-memory database Redis): The Redis SDK is called through business code to write the data in the form of "key-value pairs", which is suitable for high-frequency data that can be displayed in real time (such as current flight information, query volume in 5 minutes, and number of active users). Detailed log records are stored in local log files. Since the data visualization tool (Grafana) cannot directly read local log files, the logs need to be parsed and stored in a time-series database using Huawei Cloud Log Service LTS (a domestic tool). The LTS collection plugin is installed on the private cloud server for log parsing and field extraction. The data is then forwarded to Huawei Cloud TSDB (Time-Series Database): In LTS, "Data Forwarding" is configured to aggregate the parsed logs by "route + time" (e.g., to count the total number of queries for a certain route every hour) and write them to TSDB.

[0037] Configure Grafana with dual data sources: connect to Redis and TSDB.

[0038] Real-time metrics dashboard (connected to domestic in-memory database (Redis)): Observes TPS, requests, and real-time request information by time period.

[0039] Log statistics panel (connected to TSDB): Daily query volume statistics, percentage of abnormal requests, and other statistical information.

[0040] The following technical solution, combined with the actual business scenario of airfare search, serves as the best implementation example to verify the feasibility of the solution.

[0041] Step 1: Public cloud gateway layer.

[0042] User requests access.

[0043] A user queries the fare for the Guangzhou-Shenzhen (CAN-SZX) route and requests that the query be sent to the log-query topic (JSON format) of the domestic NATS message queue via the Huawei Cloud API: json { "app_id": "U12345", "route": "CAN-SZX", "query_time": "1695000300", IP address: 192.168.2.10 "status": "success", / / Query result: Success "price": 650 / / Display price } NATS determines that the log is a "public query record" and marks it as LEVEL2 (normal transport level), requiring no SM4 encryption and enabling only TLS1.3 transport.

[0044] Step 2: Cross-cloud collaboration layer.

[0045] LEVEL2 data is directly pushed to the private cloud service layer via the "publish-subscribe" mode of the domestic version of the NATS message queue. Subscription node: Private cloud LTS log collector; Transmission parameters: TPS=1200 (< million-level upper limit); Transmission latency: 18ms (< 20ms target); Reliability assurance: The NATS cluster has 3 redundant nodes. In the event of a primary node failure, it will switch to a backup node within 10ms, with no log loss.

[0046] Step 3: Private cloud service layer.

[0047] After parsing and aggregating the logs collected by Huawei Cloud LTS, perform the following operations: Field parsing: Extract route (CAN-SZX), query_time (1695000300→2024-09-23 10:05:00), and status (success); Aggregated by "Route + Hour": Statistics on the query volume (230 times) and success rate (100%) of the CAN-SZX route from 10:00 to 10:59. Synchronize to TSDB: Store data by day partition (20240923), data format: metric=query_total,route=CAN-SZX,hour=10 value=230.

[0048] Offline statistical visualization using Huawei Cloud Grafana, integrated with TSDB data source, generates two types of dashboards: Route Popularity Dashboard: The bar chart shows the top 5 routes in terms of search volume nationwide at 10:00 AM, with CAN-SZX ranking third; Success Rate Trend Dashboard: Line chart showing the success rate of the CAN-SZX route in the past 24 hours (all ≥99%); Query response: Complex statistics (such as "2024Q3 CAN-SZX weekend query volume") response time is 0.8 seconds (<1 second target).

[0049] Step 4: Anomaly Alarm (optional extension).

[0050] If the query success rate for a certain route (such as PEK-SHA) suddenly drops to 90% (<95% threshold), Grafana will automatically trigger an alert: Alarm method: Email + SMS push to operations personnel; Alert content: The success rate of querying the PEK-SHA route from 14:00 to 14:05 was 90%, with 12 abnormal requests. It is recommended to investigate the API interface.

[0051] With the development of civil aviation technology, the volume of civil aviation business is increasing day by day. In order to improve the convenience of business processing, airlines and airports have gradually developed online channels for various civil aviation services, such as civil aviation fare search services.

[0052] Based on the domestic three-tier architecture of public cloud-cross cloud-private cloud in the application, the core processor code dependence, and the full-process technical design, the advantages of the solution can be elaborated from five core dimensions: domestic compliance, transmission efficiency, security compliance, statistical capabilities, and cost optimization. Each advantage has specific technical support and quantitative indicators.

[0053] This application aims to address the technical problems of existing fare search systems, such as insufficient localization adaptation (reliance on foreign components), low cross-cloud transmission efficiency (full synchronization consumes bandwidth), imbalance between security and real-time performance (encryption reduces efficiency), and limited statistical dimensions (lack of real-time-offline collaboration), and to provide a full-process business data processing solution based on specified domestic components.

[0054] Multi-vendor compatibility is achieved through the AdapterCloudNode interface abstraction, avoiding dependence on a single foreign cloud service. The selectCollectNode method routes sensitive data to private cloud nodes for processing, while non-sensitive data is routed to public cloud nodes for computation, thus preventing the exposure of sensitive data. Encryption / decryption is implemented using SM4Utils to prevent plaintext leakage. By integrating data, no modification to core logic is required, reducing IT maintenance costs.

[0055] By employing secure, real-time, and scalable technologies, the system addresses the pain points of difficult, insecure, and slow data collection and statistics in domestic hybrid cloud environments. It not only meets the requirements of independent control but also provides accurate and real-time data support for the freight rate search system.

[0056] Example 2 Based on the same inventive concept, this application also provides a domestically developed system for collecting and statistically analyzing fare search data in a hybrid cloud environment, such as... Figure 4 ,include: The public cloud gateway layer is used to receive multi-source fare search data and divide the multi-source fare search data into core data and non-core data; The cross-cloud collaboration layer is used to transmit the core data and the non-core data to the private cloud service layer by adopting differentiated transmission strategies through functional components. The private cloud business layer is used to decrypt the transmitted data and collect internal operating data of the private cloud through functional components. It aggregates the decrypted transmitted data with the internal operating data of the private cloud, performs statistical processing on all aggregated data, generates statistical indicators and stores them in a domestic in-memory database, and forwards the detailed data generated by business operations to a domestic time-series database for aggregation and storage.

[0057] Optionally, the public cloud gateway layer includes Huawei Cloud API Gateway and NATS message queue; The Huawei Cloud API Gateway is used to connect to external data sources such as airlines / logistics companies, and to uniformly manage RESTful APIs; it also limits the frequency of requests executed (per IP). 100 times / second), format verification and signature verification; The NATS message queue is used to transmit regular transport-level non-core data (such as public flight route information) and supports millions of TPS concurrent connections.

[0058] Optionally, the cross-cloud collaboration layer includes China Telecom Cloud hybrid cloud dedicated line, China Telecom Cloud file block comparison tool, and China Telecom Cloud blockchain service; The China Telecom Cloud hybrid cloud dedicated line is used to connect public cloud and private cloud to transmit encrypted core data. The Tianyi Cloud file block comparison tool is used to transmit only data blocks whose hash values ​​do not match those of the local and cloud servers, reducing bandwidth usage by 80%. The Tianyi Cloud Blockchain Service is used to record cross-cloud data synchronization logs (such as "synchronization time, data volume, and operator") to ensure that the logs are tamper-proof and to meet the needs of data traceability and compliance auditing.

[0059] Optionally, differentiated transmission strategies may be adopted for the core data and the non-core data through functional components, including: For the transmission of LEVEL1 core data: Establish a China Telecom Cloud hybrid cloud dedicated line, use the SM4 national cryptographic algorithm to symmetrically encrypt LEVEL1 data, and build a private cloud-public cloud encrypted channel; The Tianyi Cloud file block comparison tool is used to calculate the SHA256 hash values ​​of local and cloud data blocks. Only the difference blocks with inconsistent hash values ​​are transmitted, reducing bandwidth usage by 80%. Each transmission operation log (including transmission time, data volume, sender IP, and encryption algorithm) is written to the China Telecom Cloud Blockchain Service in real time, generating an immutable evidence hash to meet audit and traceability requirements.

[0060] For LEVEL2 regular data transmission: LEVEL2 data is forwarded to the private cloud service layer via the "publish-subscribe" pattern of the domestic NATS message queue. Enable NATS's TLS 1.3 encryption function, with the certificate issued by Huawei Cloud SSL Certificate Management Service to ensure that data is not leaked during transmission; configure NATS cluster redundancy (3 nodes) to avoid data transmission interruption due to single point of failure.

[0061] Optionally, functional components can be used as the glue to hold the framework together. These components are imported into the project via a pom file and include four main modules: cloud node adaptation components, national cryptographic encryption transmission components, multi-source data acquisition components, and real-time statistical processing components. They are highly extensible and all use interfaces / abstract classes, supporting manual implementation of adaptations for different vendors' devices. When adding a domestic cloud vendor (such as Tencent Cloud) or data source (such as log files), only the implementation class needs to be added, without modifying the core logic.

[0062] 1. Cloud Node Adaptation Component (AdapterCloudNode): Based on an abstract cloud node interface, it adapts to the SDKs of domestic cloud vendors such as Huawei Cloud and Alibaba Cloud, enabling dynamic data source selection and adaptation between public and private cloud nodes. It loads cloud vendor configurations through vendor initialization methods and makes routing decisions based on data sensitivity identifiers through a node selection method—sensitive data is assigned to private cloud nodes for processing, and non-sensitive data is assigned to public cloud nodes for computation. Combined with a configuration center, it implements a hybrid cloud deployment strategy and supports extended access to nodes from different domestic cloud vendors. It has no dependence on foreign technologies and meets the requirements for domestic compliance.

[0063] The cloud node adaptation component defines a unified node adaptation interface, which includes at least the following methods: Initialization configuration method: Receive configuration parameters from the cloud vendor, complete the initialization of the data collection node, and load the configuration information of the corresponding cloud vendor; Node selection method (selectCollectNode): Receives a data sensitivity identifier parameter, dynamically selects and returns the corresponding collection node based on the identifier—when the data is sensitive, it is assigned to a private cloud node for processing, and when the data is non-sensitive, it is assigned to a public cloud node for computation; Different domestic cloud vendor nodes can all connect by implementing this unified interface. Each vendor node only needs to replace the client configuration and software development kit calling logic of the corresponding cloud vendor to achieve multi-vendor compatibility and dynamic routing.

[0064] 2. National Cryptographic Encryption Transmission Component: The collected data is encrypted using the SM4 national cryptographic algorithm, which complies with the "Basic Requirements for Cryptographic Applications in Information Security Technology" and is compatible with the output encryption format of all domestically produced resources and components.

[0065] 3. Multi-source data acquisition component: Supports access to multiple data sources such as domestic IoT devices, log files, and time-series databases.

[0066] 4. Real-time statistical processing component: Based on the domestic stream computing framework, it realizes real-time statistics of indicators such as freight rate volatility, API call success rate, access volume, TPS, and alerts.

[0067] Optionally, the private cloud service layer includes Huawei Cloud GaussDB (for Redis), Huawei Cloud Grafana, Huawei Cloud Log Service LTS, and Huawei Cloud TSDB; The Huawei Cloud GaussDB (for Redis) is used to store high-frequency data suitable for real-time display. Huawei Cloud Grafana is used for data visualization; The Huawei Cloud Log Service (LTS) is used for collecting detailed data. The Huawei Cloud TSDB is used to store detailed data in a time-series database.

[0068] Optionally, the system further includes a visualization module, specifically including: Real-time metrics dashboard (integrated with Redis): Observes TPS, requests, and real-time request information by time period.

[0069] Log statistics panel (connected to TSDB): Daily query volume statistics, percentage of abnormal requests, and other statistical information.

[0070] Example 3 Based on the same inventive concept, this application also provides an electronic device. The electronic device of this application includes at least one processor and at least one storage medium electrically connected to each other. The storage medium is electrically connected to the processor, wherein the storage medium stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method described above.

[0071] Example 4 Based on the same inventive concept, this application also provides a storage medium storing instructions executable by at least one processor, the instructions being executed by at least one processor to enable at least one processor to perform the method described above.

[0072] Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for collecting and statistically analyzing freight rate search data in a domestically developed hybrid cloud environment, characterized in that, include: At the public cloud gateway layer, multi-source fare search data is received and divided into core data and non-core data. In the cross-cloud collaboration layer, the core data and the non-core data are transmitted to the private cloud business layer using differentiated transmission strategies through functional components. At the private cloud business layer, the transmitted data is decrypted, and the internal operation data of the private cloud side is collected through functional components. The decrypted transmitted data and the internal operation data of the private cloud side are aggregated. All the aggregated data are statistically processed to generate statistical indicators and stored in a domestic in-memory database. At the same time, the detailed data generated by the business operation is forwarded to a domestic time-series database for aggregation and storage.

2. The method according to claim 1, characterized in that, The multi-source freight rate search data is categorized into core data and non-core data based on sensitivity, including: Data containing airline fare strategies, user payment information, and undisclosed cabin inventory will be classified as encrypted transmission-level core data. Data containing publicly available route lists, historical query statistics, and user geographic distribution is classified as non-core data for regular transmission.

3. The method according to claim 1, characterized in that, The functional components include a cloud node adaptation component, a national cryptographic encryption transmission component, a multi-source data acquisition component, and a real-time statistical processing component. The cloud node adaptation component is deployed in the cross-cloud collaboration layer and is used to select the corresponding transmission channel and processing method according to whether the data is sensitive. The national cryptographic encryption transmission component is deployed in the private cloud service layer for encryption; The multi-source data acquisition component is deployed in the private cloud service layer and is used to collect API data and IoT device data and send them. The real-time statistical processing component is deployed in the private cloud business layer and calculates statistical indicators based on a domestically developed stream computing framework.

4. The method according to claim 1, characterized in that, The differentiated transmission strategy for the core data includes: The core data is transmitted using symmetric encryption algorithm via a hybrid cloud dedicated line. The file block comparison tool is called to calculate the hash values ​​of local and cloud data blocks, and only the data blocks with inconsistent hash values ​​are transmitted; Each transmission operation log is written to a domestic blockchain service in real time to generate an immutable evidence hash.

5. The method according to claim 1, characterized in that, The differentiated transmission strategy for the non-core data includes: The non-core data is transmitted using a publish-subscribe pattern from a domestically developed message queue. Enable secure communication protocol encryption and configure cluster redundancy with at least three nodes.

6. The method according to claim 1, characterized in that, The step of statistically processing all the aggregated data to generate statistical indicators and storing them in a domestic in-memory database includes: The real-time statistical processing component performs real-time statistics on all aggregated data based on a domestic stream computing framework, generating statistical indicators including freight rate volatility, API call success rate, TPS, and traffic volume. The statistical indicators are written into a domestic in-memory database in key-value pairs for real-time display of high-frequency data.

7. The method according to claim 1, characterized in that, The step of forwarding detailed data generated by business operations to a domestic time-series database for aggregation and storage includes: The detailed data generated by business operations is written to a local log file, and the local log file is parsed and fields are extracted by a domestic log service. After being aggregated by route and time dimensions, the data is written to a domestic time-series database.

8. The method according to claim 1, characterized in that, After receiving multi-source fare search data, and before dividing the multi-source fare search data into core data and non-core data, the process also includes: verifying the multi-source fare search data. Verify the request signature using a cryptographic hash algorithm to prevent data tampering; Limit the frequency of requests from a single IP address to no more than a preset threshold; Return an error code for a format error request.

9. A domestically developed system for collecting and statistically analyzing fare search data in a hybrid cloud environment, characterized in that: include: The public cloud gateway layer is used to receive multi-source fare search data and divide the multi-source fare search data into core data and non-core data; The cross-cloud collaboration layer is used to transmit the core data and the non-core data to the private cloud service layer by adopting differentiated transmission strategies through functional components. The private cloud business layer is used to decrypt the transmitted data and collect internal operating data of the private cloud through functional components. It aggregates the decrypted transmitted data with the internal operating data of the private cloud, performs statistical processing on all aggregated data, generates statistical indicators and stores them in a domestic in-memory database, and forwards the detailed data generated by business operations to a domestic time-series database for aggregation and storage.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the fare search data collection and statistics method in a domestic hybrid cloud environment as described in any one of claims 1-8.

11. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; When a processor executes a program stored in memory, it implements the steps of the method for collecting and statistically analyzing freight rate search data in a domestic hybrid cloud environment as described in any one of claims 1-8.

Images