Memory-based data access system

Abstract

The present invention relates to a memory-based data access system, which belongs to the field of data access technology and solves the problems in the prior art of difficult memory data processing and difficult reuse of processing codes. The system comprises: a memory data source constructor, a query constructor and a memory executor; the memory data source constructor is used to encapsulate the acquired data into a memory table object and then put it into the memory data source; the query constructor is used to construct a query object syntax tree through a chain call method and obtain the query object based on the query object syntax tree; the memory executor is used to obtain the data to be queried according to the memory data source and the incoming query object, and then filter the data to be queried, sort and group the data to be queried to obtain the final query result. The system realizes access to memory data using SQL query thinking.

Description

Technical Field

[0001] The present invention relates to the technical field of data access, and in particular to a memory-based data access system. Background Art

[0002] With the development of Internet systems, various business needs and the amount of data involved are increasing. The storage of data directly affects the efficiency of data query. For data with high usage frequency or low update frequency, if it is read from the database every time, it will not only waste resources but also affect the speed of system execution. Therefore, it is usually stored in memory.

[0003] Some existing technologies store data directly in Java memory, while others utilize in-memory databases. For this in-memory data, developers typically first retrieve the object corresponding to the in-memory data and then use the get method to obtain the specific value, or place the in-memory data in a collection and use collection operations to retrieve the data. If it is necessary to associate or compare multiple objects in the in-memory data, or to interactively match the in-memory object data with data retrieved from a database or with data in a file, developers must first retrieve the required in-memory data based on specific needs and then write a program to process it together with other data.

[0004] The current method of accessing in-memory data does not support SQL (Structured Query Language) programming and requires developers to customize the code based on their needs. This results in low development efficiency, lacks reuse, and cannot guarantee code quality. Summary of the Invention

[0005] In view of the above analysis, an embodiment of the present invention aims to provide a memory-based data access system to solve the existing problems that it is difficult to query and process data in memory and the processing logic code is difficult to reuse.

[0006] The embodiment of the present invention provides a memory-based data access system, comprising: a memory data source constructor, a query constructor and a memory executor; wherein,

[0007] The memory data source constructor is used to encapsulate the acquired data into a memory table object and put it into the memory data source;

[0008] The query builder is used to construct a query object syntax tree through chain calls and obtain a query object based on the query object syntax tree;

[0009] The memory executor is used to obtain the data to be queried based on the memory data source and the incoming query object, and then filter, sort and group the data to be queried to obtain the final query result.

[0010] Furthermore, the memory table object includes: table alias, data list and field accessor;

[0011] The data list includes attribute names and attribute values;

[0012] Field accessors are used to specify one or more attribute names that need to be extracted from a data list.

[0013] Furthermore, the query builder includes multiple interfaces corresponding to SQL components, and in each interface implementation class, methods beginning with start and end are used to indicate the start and end of constructing the SQL component;

[0014] To construct a query object syntax tree, the memory table object and / or constant data set to be queried is treated as a virtual table. Multiple methods starting with start and end are called in a chained manner to construct SQL components. A query object syntax tree describes a complete SQL statement.

[0015] Furthermore, each node on the query object syntax tree is a reference object that implements the same parent reference interface. The reference object includes a method for obtaining an element type, which is used to define the SQL syntax element type corresponding to the reference object.

[0016] Furthermore, reference objects can contain other reference objects and can also derive sub-reference objects;

[0017] Reference objects include: root reference objects, table reference objects, constant query reference objects, table name reference objects, table association reference objects, subquery reference objects, selection column reference objects, grouping reference objects, sorting reference objects and Boolean condition reference objects. Among them, the root reference object is the reference object corresponding to the root node of the syntax tree, and the table name reference object, table association reference object and subquery reference object are child reference objects of the table reference object.

[0018] Furthermore, the memory table object to be queried corresponds to a table reference object, the constant dataset corresponds to a constant query reference object, and can be used as a subquery dataset of a subquery reference object. The constant query reference object implements both the root reference interface and the dataset interface.

[0019] The root reference interface is a child reference interface derived from the parent reference interface and is the interface implemented by the root reference object;

[0020] The dataset interface includes methods for obtaining the dataset header and the data list. The constant query reference object sets the attribute name of the data list to the header in the method for obtaining the dataset header, and uses a two-dimensional array format to store data in the method for obtaining the data list.

[0021] Furthermore, to obtain the query object based on the query object syntax tree, reference objects with the same element type on the syntax tree are placed in a corresponding reference object set, the reference object set is placed in the root reference object of the syntax tree root node, and the root reference object is returned.

[0022] Furthermore, according to the memory data source and the incoming query object, the data to be queried is obtained, including executing the following methods:

[0023] Table sorting is used to traverse the set of table reference objects in the query object and sort the set of table reference objects according to the rule that the left table is the main table and the right table is the subtable in the table association reference object to obtain the sorted set of table reference objects;

[0024] Table merging is used to traverse the sorted table reference object set and identify whether it contains table-related reference objects. If it does not contain table-related reference objects, the memory table objects or constant data sets in the memory data source are obtained, and the data objects are merged in the form of Cartesian products and placed in the wide table set. If it contains table-related reference objects, the association relationship, association method, associated left table name, and right table data are obtained from the child reference objects of the table-related reference objects. Based on the left table name, data objects are obtained from the wide table set, and matching right table data objects are merged into the wide table set according to the association relationship.

[0025] Furthermore, to filter the data to be queried, a set of Boolean condition reference objects in the query object is obtained, the Boolean conditions in the Boolean condition reference object set are connected and encapsulated into a Boolean condition reference object, and then the encapsulated Boolean condition reference object is converted into a Java expression, the wide table set is traversed, and the data objects that do not conform to the Java expression are directly removed from the wide table set.

[0026] Furthermore, sorting and grouping the query data to obtain the final query result includes executing the following methods:

[0027] Get the sorted reference object collection in the query object, and sort the wide table collection using the JDK sorting function based on the chain of responsibility design pattern;

[0028] Get the collection of group reference objects in the query object. Based on the group fields defined in each group reference object, retrieve the data corresponding to the group fields from the wide table collection in turn, calculate the hash value, and use the hash value as the keyword to put the data with the same hash value into the same collection. After the traversal is completed, the final query result is obtained.

[0029] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0030] 1. Following the thinking of writing SQL, chained method calls are used to construct the query object syntax tree, without considering the splicing order. Multi-level sub-query nesting is supported, which reduces the difficulty of constructing memory object queries.

[0031] 2. You can query memory objects, associate memory objects with constant data sets at the same time, or query constant data sets directly.

[0032] 3. Encapsulate the merging, filtering, sorting, and grouping of in-memory data into reusable methods. Developers only need to construct a query object and call it through the in-memory executor, just like SQL statement queries. This improves code reusability, compatibility with multiple data types, and reduces maintenance costs.

[0033] 4. When operating on memory data, references to data objects are stored in the form of two-dimensional arrays to avoid directly storing specific data information, which greatly reduces storage space. At the same time, it reduces the creation and recycling of memory objects and reduces the risk of memory overflow.

[0034] In the present invention, the above-mentioned technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of the present invention will be described in the following description, and some advantages will become apparent from the description or be learned through practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the contents particularly pointed out in the description and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings are only for the purpose of illustrating particular embodiments and are not to be considered limiting of the present invention. Like reference symbols denote like parts throughout the drawings.

[0036] Figure 1 This is a composition diagram of a memory-based data access system in an embodiment of the present invention. DETAILED DESCRIPTION

[0037] The preferred embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, and are not used to limit the scope of the present invention.

[0038] A specific embodiment of the present invention discloses a memory-based data access system, such as Figure 1 As shown, it includes: memory data source builder, query builder and memory executor;

[0039] The memory data source constructor is used to encapsulate the acquired data into a memory table object and put it into the memory data source;

[0040] The query builder is used to construct a query object syntax tree through chain calls and obtain a query object based on the query object syntax tree;

[0041] The memory executor is used to obtain the data to be queried based on the memory data source and the incoming query object, and then filter, sort and group the data to be queried to obtain the final query result.

[0042] During implementation, the query builder is used to construct a query object syntax tree for the memory table object or constant data set placed in the memory data source according to the thinking of writing SQL, automatically generate the query object, and then pass the query object to the memory executor. The query objects are automatically merged into a wide table, and then filtered, sorted, and grouped to obtain the query results.

[0043] Preferably, the memory-based data access system in this embodiment can be packaged into a Jar package and used as a tool class in an enterprise development framework to facilitate use and expansion by various development project teams.

[0044] The following is a detailed introduction to the operating principles and related technical solutions of each module in the memory-based data access system.

[0045] Considering that in-memory data comes from multiple sources, such as business system memory and in-memory databases, processing of in-memory data extends beyond just in-memory data. It can also be used to associate data in files with database query result sets. All of this data can be used for queries or associated queries. Similar to database queries, this data can be accessed like a virtual physical table using a table name or table alias, or encapsulated as a dataset and accessed as a virtual constant table. This data can be accessed directly as a standalone table or as a subtable associated with other main tables.

[0046] The in-memory data source constructor encapsulates acquired data into an in-memory table object and stores it in the in-memory data source. An in-memory table object consists of a table alias, a data list, and field accessors. If the table alias matches an existing table alias in the in-memory data source, it will be overwritten. Data stored in this way in the in-memory data source can be accessed directly using the table alias.

[0047] Specifically, the data list in an in-memory table object consists of attribute names and values. Field accessors are used to specify one or more attribute names to be extracted from the data list. When a data list contains many attributes but only a subset is needed, field accessors can be used to directly select the required data and store it in the in-memory data source, eliminating the need for manual processing.

[0048] If you do not want to put some of the acquired data into the memory data source but need to use it in data access, you can encapsulate it into a reference object that can be recognized by the query builder and attach it to the query object syntax tree.

[0049] It should be noted that the query builder is constructed according to the idea of ​​writing SQL. Considering that the components of an SQL statement include: select column name, query table name from, filter condition where, group field groupby, sort condition orderby, and group condition having, the query builder combines many interfaces, including Select, From, Where, Group, Order, and Having, which correspond to the various components of SQL respectively. In each interface implementation class, methods beginning with start and end indicate the start and end of the constructed component.

[0050] Specifically, the query builder includes the following methods:

[0051] startSelect is used to set the selected column name; endSelect is used to indicate that the current selected column name setting is completed;

[0052] startFrom, used to set the query table; endForm, used to indicate that the table setting for the current query is completed;

[0053] startWhere is used to set the filter conditions; endWhere is used to indicate that the current filter conditions are set;

[0054] startGroup is used to set the group field; endGroup is used to indicate that the current group field setting is completed;

[0055] startOrder is used to set the sorting field and sorting order; endOrder is used to indicate that the current sorting field and sorting order are set;

[0056] startHaving is used to set the grouping condition; endHaving is used to indicate that the current grouping condition setting is completed;

[0057] page, used to set the paging conditions for the query;

[0058] limit, used to set the maximum number of query results;

[0059] It should be noted that the page and limit methods are mutually exclusive. Other methods can be called in any order. Call the above methods according to the SQL syntax combination. After the construction is completed, call the build method to create and generate the query object.

[0060] It's important to note that constructing a query object syntax tree treats the in-memory table object and / or constant dataset to be queried as a virtual table. Using a chained approach, multiple methods prefixed with "start" and "end" are called to construct the SQL components. When the logic is complex, it can be freely constructed in an SQL-like manner, allowing developers to focus on the business without worrying about the order of the various SQL components or whether parentheses match.

[0061] The query builder can use a single statement or multiple statements to construct a syntax tree. A syntax tree describes a complete SQL statement. When combining multiple methods in a single statement to construct multiple SQL components, methods beginning with start and end are called in pairs. If multiple statements are used, each constructing only one SQL component, only the method beginning with start can be called. A line break in the code is equivalent to calling the method beginning with end.

[0062] For example, there is a CUSTOMER memory table in the memory data source. Suppose you want to get the customer name whose customer ID is greater than 1. If you write SQL, you can write it as: "SELECT C.CUSTOMERNAME AS customer name FROM CUSTOMERC WHERE C.ID>1". If you use the query builder, use the HQueryFacade.createQueryBuilder() method to create a query builder, and use one statement to construct the above SQL statement into a syntax tree. The sample code is as follows:

[0063] HQueryquery=

[0064] HQueryFacade.createQueryBuilder().startFrom().tab("CUSTOMER","C").endFrom().startSelect().col(0,"CUSTOMERNAME","Customer Name").endSelect().startWhere().gtr(getCol(0,"ID"),getVal(1)).endWhere().build();

[0065] Use multiple statements to construct the above SQL statement into a syntax tree. The sample code is as follows:

[0066] HQueryBuilderquerybuilder=HQueryFacade.createQueryBuilder().startFrom().tab("CUSTOMER","C").endFrom();

[0067] querybuilder.startSelect().col(0,"CUSTOMERNAME","Customer Name");

[0068] querybuilder.startWhere().gtr(getCol(0,"ID"),getVal(1));

[0069] HQueryquery=querybuilder.build();

[0070] When the query builder constructs the components of SQL, the data of each component is encapsulated as a reference object and placed in the reference object collection corresponding to each component as a node on the syntax tree.

[0071] Preferably, the reference object collection adopts the ArrayList type.

[0072] Reference objects include root reference objects, table reference objects, constant query reference objects, table name reference objects, table join reference objects, subquery reference objects, select column reference objects, group reference objects, sort reference objects, and Boolean condition reference objects. These reference objects implement the same parent reference interface, HValRef, and each includes a method for obtaining element types, which defines the element type corresponding to the reference object. The root reference object is the reference object corresponding to the root node of the syntax tree.

[0073] In-memory table objects in an in-memory data source correspond to table reference objects. For example, in the example code above, a table reference object is constructed using the tab("CUSTOMER", "C") method. "CUSTOMER" is the table name of the table reference object, corresponding to the table alias of the in-memory table object, and "C" is the table alias of the table reference object. This is used when constructing the query object syntax tree, and the table reference object is placed in the table reference object collection, frmList.

[0074] Similarly, in the above example, a selection column reference object is constructed through the col(0, "CUSTOMERNAME", "Customer Name") method, where "0" is the table index to which the column belongs, that is, the first table, "CUSTOMERNAME" is the column name, and "Customer Name" is the column alias. The selection column reference object will be placed in the selection column reference object set selList; a Boolean condition reference object is constructed through gtr(getCol(0, "ID"), getVal(1)), where gtr is the Boolean condition operator, indicating ">", getCol(0, "ID") is the left side of the Boolean condition, indicating that the compared field is the "ID" field of the first table, and getVal(1) is the right side of the Boolean condition, indicating that the compared value is "1". The Boolean condition reference object will be placed in the Bool condition reference object set boolList.

[0075] Preferably, the boolean conditional reference object supports two rvalues, taking into account that there will be two comparison values ​​when using the between operator.

[0076] It should be noted that the parent reference interface derives many child reference interfaces. Each reference object directly implements the corresponding child reference interface, inheriting methods from the parent reference interface while also adding new methods. Taking the Boolean conditional reference object as an example, the sample code for constructing a reference object and placing it into the corresponding reference object collection is as follows, where HBoolExpImpl is the implementation class of the Boolean conditional reference interface HBoolRef, which is a child reference interface derived from the parent reference interface HValRef:

[0077] / / Construct Boolean condition reference object

[0078] HBoolRefbool=

[0079] newHBoolExpImpl(getCol(0,"ID"),GTR,getVal(1),null);

[0080] / / Put the Boolean condition reference object into the Boolean condition reference object collection

[0081] ListboolList = Arrays.asList <hboolref>(bool);

[0082] Reference objects can be associated with other reference objects and can also derive sub-reference objects.

[0083] Constant datasets not stored in the in-memory data source correspond to constant query reference objects, which can serve as subquery datasets for subquery reference objects. Constant query reference objects implement both the root reference interface and the dataset interface. The root reference interface is a child reference interface derived from the parent reference interface and implemented by the root reference object. Implementing this interface enables constant query reference objects to be attached to the query object syntax tree. The dataset interface includes methods for obtaining the dataset header and the data list. In the method for obtaining the dataset header, the constant query reference object sets the attribute name of the data list as the header, and in the method for obtaining the data list, the data list is stored in a two-dimensional array format.

[0084] Specifically, the DataSet interface inherits the List interface and adds two methods, one for obtaining the query result set and the other for obtaining the metadata of the query result. The query result set is in the two-dimensional array format List<Object[]> This structure facilitates the management of data results. For example, to rename a column in a data result, you only need to modify the metadata of the query result, which greatly improves modification efficiency. Using a two-dimensional array to store the query result set can greatly reduce the storage space required.

[0085] By converting a constant data set into a constant query reference object, the object name of the constant query reference object can be directly used as the virtual table name when constructing the syntax tree.

[0086] Considering the need to perform associated queries on in-memory data, which must support leftjoin, rightjoin, innerjoin, and fulljoin, just like multi-table associations in SQL, it is necessary to derive sub-reference objects from the reference object to achieve this. The table name reference object HFrmNRef mentioned above is used to indicate that the association is made directly based on the table name. The table association reference object HFrmJRef is used to define the table reference object, association method, and association condition of the associated left table. The subquery reference object HFrmQRef is used to indicate that a subquery result is associated as a table. These three reference objects are sub-reference objects of the table reference object HFrmRef. When associating tables, both the table name and the query subtable can be directly associated. Therefore, based on the above three sub-reference object interfaces, the associated table name reference object HFrmJNRef is derived to indicate that the associated right table is the direct table name. The associated subtable reference object HFrmJQRef is used to indicate that the associated right table is a subquery.

[0087] For example, memory table object A and memory table object B are left outer joined through object A's AID and object B's BID. The example code for constructing the syntax tree is as follows:

[0088] querybuilder.startFrom()

[0089] .tab("A","Object A")

[0090] .join(0,"AID",JoinType.LEFT_JOIN,"B","Object B","BID").endFrom();

[0091] In the code, a main table name reference object HFrmNRef is constructed through the tab("A","object A") method, and an associated table name reference object HFrmJNRef is constructed through the join(0,"AID",JoinType.LEFT_JOIN,"B","object B","BID") method, which contains the table reference object HFrmRef, the association method JoinType and the Boolean condition reference object HBoolRef, which are used to define the reference object, association relationship and association condition of the associated left table respectively.

[0092] Any reference object that implements the parent reference interface can be attached to the syntax tree. Each reference object can be attached to other reference objects and can also be attached to child syntax trees, thereby achieving infinite expansion of the syntax tree and effectively reducing the difficulty of constructing the syntax tree.

[0093] For example, the attribute to be queried is an attribute in another memory table object. The sample code is as follows:

[0094] querybuilder.startSelect().val(HQerryFacade.createQueryBuilder().startS elect().col(0,"TOTALS","TOTALS").endSelect().startFrom().tab("PRODUC T","P").endFrom().build(),"Sales");

[0095] To obtain a query object based on the query object syntax tree, reference objects with the same element type on the syntax tree are placed in a corresponding reference object set, the reference object set is placed in the root reference object of the syntax tree root node, and the root reference object is returned.

[0096] After obtaining the query object, you can call the memory executor to query the data. The memory executor can use the memory table object in the memory data source to perform specified logical processing on its data.

[0097] According to the memory data source and the passed query object, the data to be queried is obtained, and then the data to be queried is filtered, sorted, and grouped to obtain the final query result.

[0098] Specifically, when obtaining the data to be queried, you need to consider various table association scenarios, including executing the following methods:

[0099] Table sorting is used to traverse the set of table reference objects in the query object and sort the set of table reference objects according to the rule that the left table is the main table and the right table is the subtable in the table association reference object to obtain the sorted set of table reference objects;

[0100] It should be noted that there is no order in which queries are constructed. However, when merging tables, the main table must be loaded before merging sub-tables based on the main table. Therefore, the sorting method prevents developers from constructing sub-tables before the main table, which may cause errors when the main table data cannot be found during subsequent table merges.

[0101] Table merging is used to traverse the sorted table reference object set and identify whether it contains table-related reference objects. If it does not contain table-related reference objects, the memory table object or constant data set is obtained, and the data objects are merged in the form of a Cartesian product and placed in the wide table set. If it contains table-related reference objects, the association relationship, association method, associated left table name, and right table data are obtained from the child reference objects of the table-related reference objects. Based on the left table name, data objects are obtained from the wide table set. According to the association method and association relationship, the matching right table data objects are merged into the wide table set.

[0102] It's important to note that because the tables are sorted first, the data from the primary (left) table is first placed in a wide table set during the merge. When processing table association reference objects, the corresponding data object can be retrieved from the wide table set based on the left table name. The wide table set is a two-dimensional array format. The number of rows in the two-dimensional array represents the number of records to be queried, and each column in each row is an object. This approach eliminates the need to directly store specific data information, significantly reducing storage space.

[0103] Preferably, use list<Object[]> The format stores the merged wide table collection;

[0104] Using the join methods leftjoin, rightjoin, innerjoin, and fulljoin, data from the left and right tables are merged according to the query rules for data join methods in SQL. For example, using leftjoin as an example, based on the join relationship, the data objects in the right table that match the left table are retrieved and placed into the object array in the same row as the left table data objects.

[0105] After merging to obtain the data to be queried, filtering the data to be queried involves obtaining a set of Boolean condition reference objects corresponding to the where condition in the query object, connecting the Boolean conditions in the Boolean condition reference object set into a Boolean condition using "and", encapsulating it into a Boolean condition reference object, and then converting the encapsulated Boolean condition reference object into a Java expression. The wide table collection is traversed, and data objects that do not meet the Java expression are directly removed from the wide table collection.

[0106] It should be noted that, since the data objects have been merged into a wide table set during table merging, the Boolean condition reference object set is processed into a Boolean condition reference object. Therefore, the wide table set only needs to be traversed once. Moreover, each time a row of objects in the wide table set is retrieved, all the filtering conditions can be determined, resulting in high processing efficiency and low performance consumption.

[0107] For example, the following conditions exist in the query object syntax tree:

[0108] Condition 1: startWhere().gtr(getCol(0,"ID"),getVal(1)).endWhere(); / / ID>1

[0109] Condition 2: startWhere().startOr().equ(getCol(0,"TYPE"),getVal(1))

[0110] .neq(getCol(0,"STATUS"),getVal(1))

[0111] .endOr().endWhere(); / / TYPE=1ORSTATUS<>1

[0112] Construct Boolean condition reference objects for condition 1 and condition 2 respectively, and put them into the Boolean condition reference object collection. The sample code is as follows:

[0113]

[0114] Connect the Boolean conditions in the Boolean condition reference object set into one Boolean condition through "and" and encapsulate it into a Boolean condition reference object. The sample code is as follows:

[0115] HBoolReflastbool=newHBoolExpImpl(null,AND,boolList,null);

[0116] This is equivalent to combining the two conditions into: ID>1AND(TYPE=1ORSTATUS<>1).

[0117] Then convert the encapsulated Boolean condition reference object lastbool into a Java expression, traverse the wide table collection, and directly remove the data objects that do not conform to the Java expression from the wide table collection.

[0118] Preferably, the Aviator expression evaluation engine is used to convert the Boolean condition reference object into a Java expression, which is then compiled into an intermediate object of Expression. During traversal, the execution method of Expression is called.

[0119] Sorting and grouping the filtered data to be queried to obtain the final query results includes executing the following methods:

[0120] Get the sorted reference object collection in the query object, and sort the wide table collection using the JDK sorting function based on the chain of responsibility design pattern;

[0121] Get the collection of group reference objects in the query object. Based on the group fields defined in each group reference object, retrieve the data corresponding to the group fields from the wide table collection in turn, calculate the hash value, and use the hash value as the keyword to put the data with the same hash value into the same collection. After the traversal is completed, the final query result is obtained.

[0122] Preferably, when constructing the query object syntax tree, if a paging condition or a maximum number of query results is set, the query results obtained after the above grouping can be further filtered to obtain the final query results.

[0123] Query results can also be encapsulated as a dataset object DataSet to reduce the storage space required and make it easier to manage data information.

[0124] In this embodiment, the in-memory executor processes queries based on the actual reference object sets within the constructed query object. If no grouped reference object sets exist, the query is not processed. By manipulating in-memory data from an SQL processing perspective, a general processing method has been developed to cover common business data scenarios, eliminating the need for developers to rewrite code and achieving high reusability. If specialized business requirements arise, new processing logic can be added by extending the parent reference interface and the in-memory executor interface, providing a flexible expansion mechanism.

[0125] Compared with the existing technology, this embodiment provides a universal memory-based data access system. It uses chained method calls to construct the query object syntax tree in accordance with the idea of ​​writing SQL, without considering the splicing order, and supports multi-level sub-query nesting, which reduces the difficulty of constructing memory object queries; it can not only realize the query of memory objects, but also simultaneously associate memory objects and constant data sets, or directly query the constant data set. The merging, filtering, sorting and grouping of memory data are encapsulated into reusable methods. Developers only need to construct a query object and call it through the memory executor like SQL statement query, which improves the code reuse, the compatibility of multiple data, and reduces maintenance costs; when operating on memory data, the reference to the data object is stored in the form of a two-dimensional array, avoiding the direct storage of specific data information, greatly reducing storage space, while reducing the creation and recycling of memory objects, and reducing the risk of memory overflow.

[0126] Those skilled in the art will appreciate that all or part of the process flow of the above-described embodiment method can be implemented by controlling related hardware through a computer program, and the program can be stored in a computer-readable storage medium, such as a magnetic disk, an optical disk, a read-only memory, or a random access memory.

[0127] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with this technical field within the technical scope disclosed by the present invention should be covered by the scope of protection of the present invention.< / hboolref>

Claims

1. A memory-based data access system, characterized in that: include: Memory data source builder, query builder and memory executor; among them, The memory data source constructor is used to encapsulate the acquired data into a memory table object and then put it into the memory data source; the memory table object includes: a table alias, a data list and a field accessor; The query builder is used to construct a query object syntax tree through a chain call method in accordance with the concept of writing SQL. That is, the memory table object and / or constant data set to be queried is treated as a virtual table. Through the chain call method, multiple groups of methods starting with start and end are called to construct SQL components. The data of each component is encapsulated as a reference object, which is placed as a node on the syntax tree in the reference object set corresponding to each component. The query object is obtained based on the query object syntax tree. The memory executor is used to sort the table reference object set in the query object according to the memory data source and the incoming query object, merge the tables according to the query rules of the Cartesian product or the data association method in SQL, obtain the data to be queried, and then filter, sort and group the data to be queried to obtain the final query result.

2. The memory-based data access system according to claim 1, characterized in that: The data list includes attribute names and attribute values; The field accessor is used to specify one or more attribute names that need to be extracted from the data list.

3. The memory-based data access system according to claim 2, characterized in that: The query builder includes multiple interfaces corresponding to SQL components, and in each interface implementation class, methods starting with start and end are used to indicate the start and end of constructing the SQL component; A query object syntax tree describes a complete SQL statement.

4. The memory-based data access system according to claim 3, characterized in that: Each node on the query object syntax tree is a reference object that implements the same parent reference interface. The reference object includes a method for obtaining an element type, which is used to define the SQL syntax element type corresponding to the reference object.

5. The memory-based data access system according to claim 4, characterized in that: The reference object contains other reference objects, or derives sub-reference objects; The reference objects include: a root reference object, a table reference object, a constant query reference object, a table name reference object, a table association reference object, a subquery reference object, a selection column reference object, a grouping reference object, a sorting reference object, and a Boolean condition reference object, wherein the root reference object is a reference object corresponding to the root node of the syntax tree, and the table name reference object, the table association reference object, and the subquery reference object are child reference objects of the table reference object.

6. The memory-based data access system according to claim 5, characterized in that: The memory table object to be queried corresponds to a table reference object, the constant data set corresponds to a constant query reference object, and the subquery data set serving as the subquery reference object, the constant query reference object implements both a root reference interface and a data set interface; The root reference interface is a child reference interface derived from the parent reference interface and is an interface implemented by the root reference object; The dataset interface includes methods for obtaining a dataset header and a data list. The constant query reference object sets the attribute name of the data list as the header in the method for obtaining a dataset header, and stores data in a two-dimensional array format in the method for obtaining a data list.

7. The memory-based data access system according to claim 6, characterized in that: The method of obtaining the query object based on the query object syntax tree is to place reference objects with the same element type on the syntax tree into a corresponding reference object set, place the reference object set into a root reference object of the syntax tree root node, and return the root reference object.

8. The memory-based data access system according to any one of claims 1 to 7, characterized in that: The method of obtaining the data to be queried based on the memory data source and the incoming query object includes executing the following methods: Table sorting is used to traverse the set of table reference objects in the query object and sort the set of table reference objects according to the rule that the left table is the main table and the right table is the subtable in the table association reference object to obtain the sorted set of table reference objects; Table merging is used to traverse the sorted table reference object set and identify whether it contains table-related reference objects. If it does not contain table-related reference objects, it obtains the memory table object or constant data set in the memory data source, merges the data objects in the form of Cartesian product, and places them into the wide table set. If a table-associated reference object is included, the association relationship, association method, associated left table name, and right table data are obtained from the child reference object of the table-associated reference object. Based on the left table name, data objects are obtained from the wide table set. According to the association relationship, the matching right table data objects are merged into the wide table set.

9. The memory-based data access system according to claim 8, characterized in that: The method of filtering the data to be queried is to obtain a set of Boolean condition reference objects in the query object, connect and encapsulate the Boolean conditions in the Boolean condition reference object set into a Boolean condition reference object, then convert the encapsulated Boolean condition reference object into a Java expression, traverse the wide table set, and directly remove data objects that do not conform to the Java expression from the wide table set.

10. The memory-based data access system according to claim 9, characterized in that: The sorting and grouping of the query data to obtain the final query result includes executing the following methods: Get the sorted reference object collection in the query object, and sort the wide table collection using the JDK sorting function based on the chain of responsibility design pattern; Get the collection of group reference objects in the query object. Based on the group fields defined in each group reference object, retrieve the data corresponding to the group fields from the wide table collection in turn, calculate the hash value, and use the hash value as the keyword to put the data with the same hash value into the same collection. After the traversal is completed, the final query result is obtained.

Images