Method, device and equipment for realizing Hive metadata cache consistency and medium

Abstract

The invention relates to the field of big data technology and financial science and technology, and discloses a Hive metadata cache consistency realization method, device and equipment and a storage medium, the method comprises the following steps: obtaining serialization configuration information of a preset table from a database, obtaining first field information of the preset table from a preset cache according to the serialization configuration information, and carrying out validity detection on the first field information to obtain a detection result, and carrying out response object returning on the first field information and preset second field information to the client according to the detection result. According to the method, the cache consistency between the Hive metadata can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the fields of big data technology and financial technology, and in particular to a method, device, equipment and medium for implementing Hive metadata cache consistency. Background Art

[0002] Hive is a data warehouse analysis system built on Hadoop. It provides a variety of SQL query methods to analyze data stored in the Hadoop distributed file system. It can map structured data files to a database table and provide complete SQL query functions to convert SQL statements into MapReduce tasks.

[0003] Hive metadata contains metadata such as databases, tables, and fields created with Hive. Metadata is stored in a relational database. The metadata service is a service used by Hive to manage metadata. By caching metadata in the metadata service, metadata access performance can be significantly improved and the request pressure on the external relational database can be reduced. However, it also brings cache consistency issues between multiple metadata services. That is, when the client modifies the table structure through one metadata service, the old table structure is still cached in the remaining metadata services. When other clients connect to another metadata service to obtain the table structure, an exception will occur. In financial business, if the client modifies the table structure through a metadata service, other services still cache the old metadata. When users query data, they may get wrong results, affecting decision-making. Although the metadata service improves access performance through caching, in high-concurrency scenarios, consistency issues may cause the client to take more time to verify and synchronize data due to cache inconsistency, increasing response time.

[0004] The existing technology currently has the problem of low cache consistency between Hive metadata. Summary of the invention

[0005] The present invention provides a method, device, equipment and medium for achieving Hive metadata cache consistency, the main purpose of which is to solve the problem of low cache consistency between Hive metadata.

[0006] In a first aspect, to achieve the above-mentioned purpose, the present invention provides a method for implementing Hive metadata cache consistency, including:

[0007] Get the serialized configuration information of the preset table from the database;

[0008] Acquire first field information of the preset table from a preset cache according to the serialization configuration information;

[0009] Performing a validity test on the first field information to obtain a test result;

[0010] The first field information and the preset second field information are returned to the client as a response object according to the detection result.

[0011] In a second aspect, the present invention also provides a device for implementing Hive metadata cache consistency, the device comprising:

[0012] The configuration information acquisition module is used to obtain the serialized configuration information of the preset table from the database;

[0013] A field information acquisition module, used to acquire first field information of the preset table from a preset cache according to the serialization configuration information;

[0014] A validity detection module, used to perform a validity detection on the first field information to obtain a detection result;

[0015] The effective information returning module is used to return the first field information and the preset second field information to the client in a response object according to the detection result.

[0016] In a third aspect, the present invention further provides an electronic device, the electronic device comprising:

[0017] at least one processor; and,

[0018] a memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the above-mentioned method for implementing Hive metadata cache consistency.

[0020] In a fourth aspect, the present invention further provides a computer-readable storage medium, wherein at least one computer program is stored in the computer-readable storage medium, and the at least one computer program is executed by a processor in an electronic device to implement the above-mentioned method for implementing Hive metadata cache consistency.

[0021] The present invention obtains the serialization configuration information of the preset table from the database, obtains the first field information of the preset table from the preset cache according to the serialization configuration information, and uses the field name in the serialization configuration information as a key and uses the cache for fast reading. The financial system can improve the response speed while ensuring the accuracy of the data, and performs validity detection on the first field information to obtain the detection result. By judging whether the first field information is empty, it can timely identify invalid data and take measures to ensure that no erroneous or outdated information is stored in the cache. If the first field is empty, the first field is marked as invalid and the relevant cache is cleared, thereby maintaining the accuracy of the data. By obtaining and updating the second field information to the cache, it is ensured that the latest database status is reflected in the cache, which helps to build a dynamic and reliable cache mechanism, so that the Hive system can maintain high consistency and real-time performance when processing metadata. According to the detection result, the first field information and the preset second field information are returned to the client as a response object. The present invention can effectively improve the cache consistency between Hive metadata. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0023] Figure 1 It is a schematic diagram of an application environment for implementing a Hive metadata cache consistency method in one embodiment of the present invention;

[0024] Figure 2 A flow chart of a method for implementing Hive metadata cache consistency provided by an embodiment of the present invention;

[0025] Figure 3 A schematic diagram of a module for implementing a Hive metadata cache consistency device provided by an embodiment of the present invention;

[0026] Figure 4 A schematic diagram of the structure of an electronic device for implementing a method for implementing Hive metadata cache consistency provided by an embodiment of the present invention;

[0027] Figure 5 It is another structural schematic diagram of an electronic device for implementing a Hive metadata cache consistency method provided by an embodiment of the present invention.

[0028] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the technical solution of the present disclosure, and to fully understand and implement how the present disclosure applies technical means to solve technical problems and achieve the corresponding technical effects, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only embodiments of a part of the present disclosure, not all of the embodiments. The embodiments of the present disclosure and the various features in the embodiments can be combined with each other without conflict, and the technical solutions formed are all within the scope of protection of the present disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of the present disclosure.

[0030] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, device, product or equipment that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0031] The embodiment of the present application provides a method for implementing Hive metadata cache consistency, and the execution subject of the method for implementing Hive metadata cache consistency includes but is not limited to at least one of the electronic devices such as a server and a terminal that can be configured to execute the device provided by the embodiment of the present application. In other words, the method for implementing Hive metadata cache consistency can be executed by software or hardware installed on a terminal device or a server device. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms.

[0032] The embodiment of the present invention implements a Hive metadata cache consistency method, which can be applied in the following situations: Figure 1In the application environment. Among them, the client communicates with the server through the network. The server can obtain the serialized configuration information of the preset table from the database, obtain the first field information of the preset table from the preset cache according to the serialized configuration information, and use the field name in the serialized configuration information as a key and use the cache for fast reading. The financial system can ensure the accuracy of the data while improving the response speed, and perform validity detection on the first field information to obtain the detection result. By judging whether the first field information is empty, it can timely identify invalid data and take measures to ensure that no erroneous or outdated information is stored in the cache. If the first field is empty, the first field is marked as invalid and the relevant cache is cleared, thereby maintaining the accuracy of the data. By obtaining and updating the second field information to the cache, it is ensured that the latest database status is reflected in the cache. Finally, according to the detection result, the first field information and the preset second field information are returned to the client as a response object. Among them, the client can be but not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices. The server can be implemented with an independent server or a server cluster composed of multiple servers. The present invention is described in detail below through specific embodiments.

[0033] Reference Figure 2 FIG. 1 is a flow chart of a method for implementing Hive metadata cache consistency according to an embodiment of the present invention. In this embodiment, the method for implementing Hive metadata cache consistency includes:

[0034] S1. Get the serialization configuration information of the preset table from the database.

[0035] In the embodiment of the present invention, the database table refers to a structured data set stored in a database, which is composed of rows and columns. Each row represents a record or data entry, and each column represents an attribute or field, which defines the specific characteristics of the record.

[0036] The main components of a database table are: Rows: Each row represents a data record. For example, in a "User" table, each row can represent the information of a user. Columns: Each column represents a specific attribute of a record. For example, in a "User" table, there may be columns such as "User ID", "User Name", and "Email". Primary key: Used to uniquely identify each row of data in the table. The primary key ensures the uniqueness of the data and is usually a single field or a combination of multiple fields. Foreign key: Used to establish associations with other tables to help achieve data integrity and consistency.

[0037] The serialized configuration information refers to the process of converting the configuration information into a format that can be stored, transmitted or reused. This conversion can make the configuration information exist in a way that is easy to manage and share, and is usually used in scenarios such as application configuration and data exchange. In applications, configuration information often needs to be read from files. For example, configuration files in JSON, YAML or XML format are used. Through serialization, these files can be easily read and parsed.

[0038] In detail, the obtaining of serialized configuration information of a preset table from a database includes:

[0039] Get serialization rules and serialization strategies in the database;

[0040] Use the preset object-relational mapping tool to query the table structure of the preset table;

[0041] The serialization rule and the table structure are mapped according to the serialization strategy to obtain serialization configuration information.

[0042] Furthermore, mapping serialization rules to table structures ensures that subsequent data conforms to the format of subsequent transmission during serialization and deserialization, which can effectively reduce errors when data is transferred between the cache and the database and enhance data consistency. The generated serialization configuration information can standardize the data access process, help maintain metadata consistency, and ensure that different modules understand and process data in a consistent manner. It not only improves processing efficiency, but also reduces the probability of errors caused by inconsistent data, making it more flexible when facing different business needs. Clear serialization rules and serialization strategies provide a good foundation for subsequent maintenance and functional expansion, allowing the system to quickly adapt to changes without affecting data consistency, and provide reliable data support for subsequent data analysis.

[0043] In detail, mapping the serialization rule and the table structure according to the serialization strategy to obtain serialization configuration information includes:

[0044] Generate a serialization target format according to the serialization strategy;

[0045] Mapping the table to the serialization target format according to the serialization rule and the table structure to obtain a mapping relationship;

[0046] The mapping relationship is summarized as serialized configuration information.

[0047] Generating the target format according to the serialization strategy and mapping the table to the format can effectively create a clear mapping relationship. This method ensures the structuring and normalization of the data, so that the data conforms to the format of subsequent transmission when serialized, while maintaining the original table structure information. Aggregating these mapping relationships into serialization configuration information not only improves the consistency and accuracy of data processing, but also provides higher flexibility for subsequent data interaction and storage.

[0048] In detail, generating a serialization target format according to the serialization strategy includes:

[0049] Get the serialized data structure;

[0050] Iterate the serialized data structure and access each data type in the serialized data structure one by one;

[0051] The data type is filled into a preset target structure according to the serialization strategy to obtain a serialized target format.

[0052] Iterate the serialized data structure and access each data type one by one, so as to ensure the comprehensive processing of all serialized data structure related information. The data type can be flexibly filled into the preset target structure according to the serialization strategy to generate a serialized target format that meets the requirements. This not only improves the accuracy and efficiency of data conversion, but also ensures the compatibility and consistency of data between different systems.

[0053] Furthermore, an object-relational mapping (ORM) tool refers to a technology used to simplify data interaction between objects and relational databases. It allows developers to manipulate data in the database in the form of objects without having to directly write complex SQL statements, thereby improving development efficiency and code maintainability. Use the ORM tool to establish a connection with the database, and use the methods provided by the ORM to query the field information of a specific table, including field name, type, constraints, etc. The ORM tool maps tables in the database to objects in the programming language, maps table rows to object instances, and maps table columns to object properties. During the data serialization process, the data is validated and formatted according to the rules to ensure that the data returned in the end conforms to the expected format.

[0054] In a specific embodiment, in a financial system, serialization rules define the format and representation of data. For example, for transaction amounts, a two-decimal format can be set; for dates, the format of "YYYY-MM-DD" can be specified. These rules are usually stored in configuration files or databases to facilitate system maintenance and adjustment. The serialization strategy determines how to serialize objects, for example, selecting the JSON format for front-end display and back-end communication. For example, a simple serialization rule may stipulate that the amounts in all transaction records should be expressed in the format of "0.00" and the transaction date should be stored in the format of "year-month-day".

[0055] The system will use the preset object-relational mapping tool to obtain the structural information of the "Customer Transaction" table. This table may contain fields such as "Transaction ID", "Customer ID", "Transaction Amount", "Transaction Date" and "Transaction Status". Through the object-relational mapping tool, you can easily obtain the name, type and constraint information of each field to ensure that each field can be processed according to the specified serialization rules.

[0056] After obtaining the serialization rules and table structure, the next step is to map the data. Taking the "Customer Transaction" table as an example, each field will be processed according to the previously defined serialization rules. Transaction ID: As a unique identifier, it is directly serialized as an integer. Customer ID: It is also directly serialized as an integer to represent the unique identity of the customer. Transaction amount: According to the serialization rules, the transaction amount is formatted as a string in the "0.00" format, such as "100.00". Transaction date: It is processed in the "YYYY-MM-DD" format, such as "2023-10-29". Transaction status: It is directly represented in string form, such as "success" or "failure".

[0057] Obtaining serialization rules and strategies from the database and using the preset object-relational mapping tool to query the field information of the database table can effectively decouple the business logic from the data layer. By mapping serialization rules and field information, serialization configuration information can be generated, thereby simplifying the data processing process and improving the efficiency and consistency of data transmission. At the same time, clear mapping relationships help maintain and expand the system, reduce potential error risks, and improve overall development efficiency.

[0058] S2. Acquire first field information of the preset table from a preset cache according to the serialization configuration information.

[0059] In the embodiments of the present invention, cache refers to a temporary storage mechanism for quickly accessing frequently used data to improve system performance and response speed. By storing data in a storage medium (such as memory) that is faster than the original storage (such as a database or remote server), cache can significantly reduce the latency and frequency of data access, thereby improving the efficiency of the application.

[0060] In detail, the acquiring the first field information of the preset table from the preset cache according to the serialization configuration information includes:

[0061] Using the field name in the serialized configuration information as a key;

[0062] The key is read in a preset cache to obtain first field information of the preset table.

[0063] Furthermore, the field name or table name in the configuration table is used as a key to quickly locate data in the cache. Through the data stored in the cache, efficient read operations can be achieved. If the required information exists in the cache, the result is returned directly, thereby reducing the number of visits to the database and improving performance.

[0064] In a specific embodiment, in a financial system, serialization configuration information of a "customer account" table is set. Assume that the table contains the following fields: account ID, customer name, account balance, account opening date, and account status. The serialization configuration information may represent these fields in the form of key-value pairs, for example: account ID, customer name, account balance, account opening date, account status. Based on the above serialization configuration information, we use "account ID" as the key for cache reading. The cache may store information of multiple customer accounts to improve the system response speed. For example, assume that the data stored in the cache is as follows:

[0065] Key:"AccountID:12345"Value:"12345";

[0066] Key:"AccountID:67890"Value:"67890";

[0067] When the first field information of a specific account needs to be obtained, the system will use cache technology to initiate a request to the cache in the format of "AccountID:12345". The cache will quickly find the corresponding key and return the corresponding value, that is, the account ID "12345". Since the cache operation is much faster than directly accessing the database, this processing method significantly improves the efficiency of data reading.

[0068] For example, when a financial institution is performing customer identity authentication, it needs to quickly obtain the customer's basic information. Through the above cache mechanism, the customer's account ID can be quickly retrieved without accessing the database every time, which can effectively reduce the database load in high concurrency situations and improve the overall performance of the system.

[0069] By using the field name in the serialized configuration information as the key and using the cache for fast reading, the financial system can improve the response speed while ensuring data accuracy. This method has broad application prospects in multiple scenarios such as customer account management, transaction processing, and risk assessment, helping to improve user experience and business efficiency.

[0070] S3. Perform a validity check on the first field information to obtain a test result.

[0071] In an embodiment of the present invention, the validity of the first field information is checked to ensure that the table information in the cache obtained complies with the expected format and requirements, and to avoid transmitting invalid or unsafe data. By determining whether the first field information is empty, the validity of the first field information is checked to ensure the accuracy and consistency of the data.

[0072] In detail, the performing validity check on the first field information to obtain a test result includes:

[0073] Determine whether the first field information is empty;

[0074] If the first field information is empty, determining that the detection result is that the first field information is invalid;

[0075] Clear the cache where the first field information is located;

[0076] Obtaining second field information of the preset table in the database;

[0077] Updating the second field information into the cache;

[0078] If the first field information is not empty, it is determined that the detection result is that the first field information is valid.

[0079] By determining whether the first field information is empty, invalid data can be identified in time and measures can be taken to ensure that no erroneous or outdated information is stored in the cache. If the first field is empty, the first field is marked as invalid and the relevant cache is cleared to maintain data accuracy. By obtaining and updating the second field information to the cache, the latest database status is ensured to be reflected in the cache. This process effectively synchronizes data between the database and the cache, ensures metadata consistency, and helps build a dynamic and reliable cache mechanism, so that the Hive system can maintain high consistency and real-time performance when processing metadata.

[0080] In a specific embodiment, suppose a financial institution is processing a customer's loan application, which contains multiple fields of information. The "first field of information" we are concerned about here is the customer's ID number. This field is crucial for the review of the loan application because it is used to verify the customer's identity and credit record.

[0081] Financial institutions need to perform validity checks on the ID number provided by the customer. The first step in this process is to check whether the field information is empty. Specifically, the system will count the length of the ID number. If the length is zero, it means that the customer did not provide an ID number. The system will mark the test result as invalid and take appropriate measures. This may include clearing the cache of the loan application to ensure that the system no longer attempts to process invalid data.

[0082] If the ID number is empty, the system will automatically notify the relevant staff and remind them that they need to supplement the customer's ID information so that the subsequent loan approval process can continue.

[0083] If the customer provides an ID number, and the length of the ID number meets the specification, the field information will be judged to be valid. The system will update the relevant cache to ensure that the customer's ID number is correctly stored and can be used for subsequent credit review and risk assessment.

[0084] This detection method can accurately determine the validity of field information and ensure that the data that subsequent operations rely on is reliable. When the first field information is empty, clearing the cache in time can avoid using outdated or erroneous information, enhancing system stability and data consistency. At the same time, by counting the length, the validity of field information can be determined simply and clearly, making the code more readable and maintainable.

[0085] S4. Return the first field information and the preset second field information to the client as a response object according to the detection result.

[0086] In the embodiment of the present invention, returning the first field information and the preset second field information to the client as a response object according to the detection result includes:

[0087] Generate a response object according to the detection result;

[0088] If the detection result is that the first field information is invalid, then fill the second field information into the response object;

[0089] If the detection result is that the first field information is valid, filling the first field information into the response object;

[0090] Send the response object to the client.

[0091] By generating a response object and dynamically filling in the corresponding field information based on the test results, it is possible to flexibly respond to client requests and ensure that the returned data is always up-to-date and valid. If the test results show that the first field information is invalid, the system will use the second field information to fill in the response object, which ensures that even if there is invalid data, the client can still obtain valid information, reducing potential errors or data inconsistencies. If the first field information is valid, it is directly filled into the response object, further enhancing the accuracy and reliability of the data. Through this mechanism, not only can the cache content be updated dynamically, but it can also ensure that the information obtained by the client is consistent with the actual state of the database, avoiding problems caused by cache expiration or invalid data.

[0092] In detail, the response object refers to the data structure returned from the server to the client, usually containing multiple parts, such as a status code, a response header, and a response body. The response object is used to pass information and data about the request result. It can be in JSON, XML, or other formats, depending on the design of the API. The API response refers to the complete response generated by the server based on the client's request, including the response object. It contains the processing result of the request and the corresponding data for further processing by the client. The API response is usually used to confirm whether the operation is successful and return the requested information.

[0093] A response object is constructed based on the test results, including valid field information and invalid field information. Information of all valid fields is extracted from the results of the validity test and filled into the response object, which is usually sent to the client in JSON format.

[0094] In a specific embodiment, when a customer submits a loan application, the system needs to check the information in the application form to ensure the validity of all fields. For example, the application form contains multiple fields, such as name, ID number, income certificate, and loan amount. The system will first check these fields to determine whether each field conforms to the predetermined format and business logic.

[0095] Suppose the system detects that the ID number submitted by the customer is not in the correct format, but the other fields (such as name, proof of income and loan amount) are valid. At this time, the system will extract the valid field information from the detection result as the second field information (such as name, proof of income and loan amount) and prepare to return it to the client.

[0096] In the response object, the system fills in the valid second field information (such as name, income proof, and loan amount). Although the ID number is invalid, the customer's other information can still help the system understand the context of the loan application.

[0097] Finally, the system sends the constructed response object back to the client, telling the client which fields are valid and how to modify the invalid fields for resubmission. The response can include specific error prompts, such as "The ID number format is incorrect, please check and re-enter."

[0098] Through this process, the bank can not only ensure that the information submitted by the customer is valid, but also optimize the customer experience and reduce repeated operations caused by incorrect information. After the customer resubmits the revised application, the system will check again until all fields are valid.

[0099] Generating a response object based on the detection results ensures the accuracy and pertinence of the response. By judging the validity of the information in the first field, you can flexibly select appropriate data to fill in the response object, which can effectively reduce the interference of invalid information on the client, improve the overall efficiency of the system and user experience, simplify the logic of information processing, and make the system more efficient in processing requests. It can provide feedback to the client faster and improve user satisfaction.

[0100] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0101] like Figure 3 , which is a functional module diagram of a device for implementing Hive metadata cache consistency provided by an embodiment of the present invention.

[0102] In the embodiment of the present disclosure, a device for implementing Hive metadata cache consistency is provided, and the device for implementing Hive metadata cache consistency corresponds to the method for implementing Hive metadata cache consistency in the above embodiment. Figure 3 As shown, the Hive metadata cache consistency device 100 can be installed in an electronic device. According to the implemented functions, the Hive metadata cache consistency device 100 includes a configuration information acquisition module 101, a field information acquisition module 102, a validity detection module 103 and a valid information return module 104. The detailed description of each functional module is as follows:

[0103] The configuration information acquisition module 101 is used to acquire serialized configuration information of a preset table from a database;

[0104] A field information acquisition module 102, configured to acquire first field information of the preset table from a preset cache according to the serialization configuration information;

[0105] A validity detection module 103, used to perform a validity detection on the first field information to obtain a detection result;

[0106] The effective information returning module 104 is used to return the first field information and the preset second field information to the client in a response object according to the detection result.

[0107] In one embodiment, when the configuration information acquisition module 101 acquires serialized configuration information of a preset table from a database, it is used to:

[0108] Get serialization rules and serialization strategies in the database;

[0109] Use the preset object-relational mapping tool to query the table structure of the preset table;

[0110] The serialization rule and the table structure are mapped according to the serialization strategy to obtain serialization configuration information.

[0111] In one embodiment, when the configuration information acquisition module 101 acquires serialized configuration information of a preset table from a database, it is used to:

[0112] Generate a serialization target format according to the serialization strategy;

[0113] Mapping the table to the serialization target format according to the serialization rule and the table structure to obtain a mapping relationship;

[0114] The mapping relationship is summarized as serialized configuration information.

[0115] In one embodiment, when the configuration information acquisition module 101 acquires serialized configuration information of a preset table from a database, it is used to:

[0116] Get the serialized data structure;

[0117] Iterate the serialized data structure and access each data type in the serialized data structure one by one;

[0118] The data type is filled into a preset target structure according to the serialization strategy to obtain a serialized target format.

[0119] In one embodiment, when the field information acquisition module 102 acquires the first field information of the preset table from the preset cache according to the serialization configuration information, it is used to:

[0120] Using the field name in the serialized configuration information as a key;

[0121] The key is read in a preset cache to obtain first field information of the preset table.

[0122] In one embodiment, when the validity detection module 103 performs validity detection on the first field information and obtains the detection result, it is used to:

[0123] Determine whether the first field information is empty;

[0124] If the first field information is empty, determining that the detection result is that the first field information is invalid;

[0125] Clear the cache where the first field information is located;

[0126] Obtaining second field information of the preset table in the database;

[0127] Updating the second field information into the cache;

[0128] If the first field information is not empty, it is determined that the detection result is that the first field information is valid.

[0129] In one embodiment, when executing the return of the first field information and the preset second field information to the client as a response object according to the detection result, the valid information returning module 104 is used to:

[0130] Generate a response object according to the detection result;

[0131] If the detection result is that the first field information is invalid, then fill the second field information into the response object;

[0132] If the detection result is that the first field information is valid, filling the first field information into the response object;

[0133] Send the response object to the client.

[0134] In the present invention, in order to achieve Hive metadata cache consistency, first, by obtaining the serialization configuration information of the preset table from the database, the data processing process is simplified, and the efficiency and consistency of data transmission are improved. Then, according to the serialization configuration information, the first field information of the preset table is obtained from the preset cache. By using the field name in the serialization configuration information as a key and using the cache for fast reading, the financial system can improve the response speed while ensuring data accuracy, and perform validity detection on the first field information to obtain the detection result, and can accurately judge the validity of the field information to ensure that the data relied on by subsequent operations is reliable. When the first field information is empty, timely clearing the cache can avoid using outdated or erroneous information, enhance the stability and data consistency of the system, and finally, according to the detection result, the first field information and the preset second field information are returned to the client as a response object, and the response object is generated according to the detection result to ensure the accuracy and pertinence of the response. By judging the validity of the first field information, appropriate data can be flexibly selected to fill in the response object, which can effectively reduce the interference of invalid information on the client and effectively improve the cache consistency between Hive metadata. For the specific limitations on the implementation of the Hive metadata cache consistency device, please refer to the limitations on the method for implementing the Hive metadata cache consistency above, which will not be repeated here. Each module in the above-mentioned Hive metadata cache consistency device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0135] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client through a network connection. When the computer program is executed by the processor, the functions or steps on the server side of the Hive metadata cache consistency method are implemented.

[0136] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Figure 5As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, the functions or steps of the client side of the Hive metadata cache consistency method are implemented.

[0137] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0138] Get the serialized configuration information of the preset table from the database;

[0139] Acquire first field information of the preset table from a preset cache according to the serialization configuration information;

[0140] Performing a validity test on the first field information to obtain a test result;

[0141] The first field information and the preset second field information are returned to the client as a response object according to the detection result.

[0142] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and apparatuses can be implemented in other ways. For example, the system embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0143] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0144] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0145] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0146] In some implementations of this embodiment, a computer-readable storage medium is provided, on which a computer program is stored, characterized in that when the computer program is executed by a processor, the steps of the method described in the above embodiment are implemented.

[0147] The readable storage medium of the present invention stores a computer program, and when the computer program is executed by a processor of an electronic device, the computer program can achieve:

[0148] Get the serialized configuration information of the preset table from the database;

[0149] Acquire first field information of the preset table from a preset cache according to the serialization configuration information;

[0150] Performing a validity test on the first field information to obtain a test result;

[0151] The first field information and the preset second field information are returned to the client as a response object according to the detection result.

[0152] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant descriptions on the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0153] The computer-readable storage medium may also store at least one computer executable program / instruction, which may be, for example, a computer-readable instruction. The computer-readable storage medium includes, but is not limited to, for example, a volatile memory and / or a non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The computer-readable storage medium may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. For example, a non-transitory computer-readable storage medium may be connected to a computing device such as a computer, and then, when the computing device runs the computer-readable instructions stored on the computer-readable storage medium, the various methods described above may be performed.

[0154] In addition, the computer device may also include (but not limited to) a data bus, an input / output (I / O) bus, a display, and input / output devices (eg, keyboard, mouse, speaker, etc.), etc.

[0155] The processor may communicate with external devices via an I / O bus via a wired or wireless network.

[0156] In one embodiment, the at least one computer executable instruction may also be compiled into or constitute a software product / computer program product, wherein one or more computer executable instructions are executed by a processor to perform the various functions and / or method steps in the embodiments described in the present technology.

[0157] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0158] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0159] In the embodiments provided in the present disclosure, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0160] It should be noted that in the present disclosure, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element limited by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0161] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

[0162] It should be noted that if software tools or components other than those of our company appear in the embodiments of the present application, they are only used for illustrative purposes and do not represent actual use.

Claims

1. A method for implementing Hive metadata cache consistency, characterized in that: The method comprises: Get the serialized configuration information of the preset table from the database; Acquire first field information of the preset table from a preset cache according to the serialization configuration information; Performing a validity test on the first field information to obtain a test result; The first field information and the preset second field information are returned to the client as a response object according to the detection result.

2. The method for implementing Hive metadata cache consistency according to claim 1, characterized in that: The step of obtaining serialized configuration information of a preset table from a database includes: Get serialization rules and serialization strategies in the database; Use the preset object-relational mapping tool to query the table structure of the preset table; The serialization rule and the table structure are mapped according to the serialization strategy to obtain serialization configuration information.

3. The method for implementing Hive metadata cache consistency according to claim 2, characterized in that: The mapping of the serialization rule and the table structure according to the serialization strategy to obtain serialization configuration information includes: Generate a serialization target format according to the serialization strategy; Mapping the preset table to the serialization target format according to the serialization rule and the table structure to obtain a mapping relationship; The mapping relationship is summarized as serialized configuration information.

4. The method for implementing Hive metadata cache consistency according to claim 3, characterized in that: Generating a serialization target format according to the serialization strategy includes: Get the serialized data structure; Iterate the serialized data structure and access each data type in the serialized data structure one by one; The data type is filled into a preset target structure according to the serialization strategy to obtain a serialized target format.

5. The method for implementing Hive metadata cache consistency according to claim 1, characterized in that: The acquiring the first field information of the preset table from the preset cache according to the serialization configuration information includes: Using the field name in the serialized configuration information as a key; The key is read in a preset cache to obtain first field information of the preset table.

6. The method for implementing Hive metadata cache consistency according to claim 1, characterized in that: The performing validity detection on the first field information to obtain a detection result includes: Determine whether the first field information is empty; If the first field information is empty, determining that the detection result is that the first field information is invalid; Clear the cache where the first field information is located; Obtaining second field information of the preset table in the database; Updating the second field information into the cache; If the first field information is not empty, it is determined that the detection result is that the first field information is valid.

7. The method for implementing Hive metadata cache consistency according to claim 1, characterized in that: The returning the first field information and the preset second field information to the client as a response object according to the detection result includes: Generate a response object according to the detection result; If the detection result is that the first field information is invalid, then fill the second field information into the response object; If the detection result is that the first field information is valid, filling the first field information into the response object; Send the response object to the client.

8. A device for implementing Hive metadata cache consistency, characterized in that: The device comprises: The configuration information acquisition module is used to obtain the serialized configuration information of the preset table from the database; A field information acquisition module, used to acquire first field information of the preset table from a preset cache according to the serialization configuration information; A validity detection module, used to perform a validity detection on the first field information to obtain a detection result; The effective information returning module is used to return the first field information and the preset second field information to the client in a response object according to the detection result.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for implementing Hive metadata cache consistency as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for implementing Hive metadata cache consistency as described in any one of claims 1 to 7 is implemented.

Images