Java-based Redis structured data table operating system and operating method

Abstract

The invention discloses a Java-based Redis structured data table operating system and operating method, and relates to the technical field of structured data management. The system comprises a table definition module, a data module, an index module, a query grammar module, an aggregation module and a multi-table operation module, and provides that structured data is mapped to a RedJSON storage structure from a traditional table form, index query is realized through RedSearch, and data write-in operation including data verification, data insertion and index synchronization is completed. The method can be used for improving the query performance of a traditional database and is also suitable for replacing partial query functions of a relational database in partial light-weight applications. According to the method, the Redis table system architecture is realized, the underlying capabilities of the Redis JSON and the Redis Search are packaged, and the use experience of the class relational database table is constructed.

Description

Technical Field

[0001] This invention relates to the technical field of structured data management, and more particularly to a Java-based Redis structured data table operating system and operation method. Background Technology

[0002] In the data-driven information technology field, the rapid development of the Internet, cloud computing, and big data applications has made the performance requirements for data storage and access increasingly stringent. Redis (RemoteDictionaryServer), as an open-source, high-performance in-memory database, achieves millisecond-level read and write responses through in-memory operations. It is widely used in high-concurrency, low-latency scenarios, such as caching services and session storage, and is a key component of enterprise-level systems. With its key-value storage as its core data model, its simple structure efficiently supports simple data access, highlighting its high-performance advantages. However, as data volume increases and business logic becomes more complex, user needs extend far beyond simple key-value pair reading and writing, requiring structured data modeling, multi-dimensional index management, and complex conditional queries. Redis's native architecture is not optimized for these needs, lacking the structured data definition capabilities of relational databases (such as MySQL), and does not support multi-table joins or complex aggregation operations. This makes it difficult to meet the demands of deep retrieval and analysis scenarios, thus limiting its application scope. To address these shortcomings, the Redis ecosystem has introduced modules such as RedisJSON and RedisSearch. RedisJSON supports storing structured data in JSON format, and RedisSearch provides full-text search and multi-field query capabilities. However, developers need to manually design and maintain data structures, create and manage indexes, and master specific query syntax, which increases the learning and workload, makes design errors more likely, and increases the complexity of development and maintenance. While traditional relational databases possess robust capabilities for processing structured data, their reliance on disk storage makes disk I / O a bottleneck in high-volume, high-concurrency scenarios, leading to slow queries and response delays. Although some NoSQL databases optimize for high-concurrency storage, they fall short in handling complex queries, making it difficult to balance high performance with complex data processing. In summary, current database technology has shortcomings in meeting the demand for both high performance and complex data processing capabilities. How to balance these two aspects and reduce development and maintenance costs has become an urgent problem to be solved. Summary of the Invention

[0003] To address the aforementioned issues, this invention provides a Java-based Redis structured data table operating system and operation method, which encapsulates the underlying capabilities of RedisJSON and RedisSearch, and builds a user experience similar to relational database tables, thereby resolving the pain points of existing technologies.

[0004] To achieve the above objectives, the technical solution adopted by the present invention is as follows: One aspect provides a Java-based Redis structured data table operating system, including: The module includes a table definition module, a data module, a query syntax module, a multi-table operation module, and an aggregation module. The table definition module is used to define structured tables for Redis data. The structured tables include fields, primary keys, indexes, and foreign keys. The table definition module persists the table definition structure in JSON format, automatically generates RedisSearch index mappings during the table definition stage, and implements data writing, querying, and referencing operations through reusable table definitions. The data module is used to insert and validate data according to the definition of the structured table, and to achieve strong data consistency and business security in Redis through the structure-driven mechanism, and automatically synchronize the relevant field indexes after each data write. The query syntax module is used to construct query condition syntax trees through Java objects. The query condition syntax trees support recursive nested combination of logical relationships and field judgment conditions. The query syntax module automatically converts the constructed query condition syntax trees into RedisSearch query DSL and provides a SQL-like query construction experience. The multi-table operation module is used to automatically implement data referencing, union matching and merging between the main table and the child table based on the foreign key information in the table definition output by the table definition module, simulate foreign key and JOIN behavior, and complete data merging through system logic control. The aggregation module provides the COUNT, SUM, AVG, and GROUPBY aggregate functions, which process the aggregation logic on the query results through the system, and support field filtering, grouping, and chart data structure output.

[0005] Preferably, the table definition module automatically drives data validation, index generation, and foreign key reference operations through table definition.

[0006] Preferably, in the data module, the field index synchronization process after data is written does not require manual intervention from the user.

[0007] Preferably, the query syntax module encapsulates the RedisSearch DSL query logic in the form of a syntax tree, supports the nested combination of complex logical relationships, improves the expressiveness of query conditions, and reduces users' dependence on the details of RedisSearch syntax.

[0008] Preferably, the multi-table operation module constructs a data reference mechanism between the master table and the child table based on Redis, and the union matching and merging process is triggered based on the foreign key relationship.

[0009] Preferably, the aggregation module extends the limitations of RedisSearch's lack of aggregation capabilities, providing lightweight data statistical analysis capabilities for the business, and the chart data structure output supports data source formats that are compatible with common data visualization tools.

[0010] Preferably, during the foreign key verification process of the data module, the existence of the data corresponding to the target foreign key is queried through Redis atomic operations; The aggregation module's GROUPBY grouping operation supports multi-field grouping, and the priority of multi-field grouping is determined by the field order parameter passed by the user.

[0011] Preferably, the query syntax module provides a SQL-like query construction experience and supports constructing query conditions through chained method calls, where the parameters of the chained method call correspond to the keywords of the SQL syntax.

[0012] Preferably, when reusing table definitions, the table definition module loads the persisted JSON format table definition through Java reflection mechanism to adapt to the structural requirements of data writing, querying, and referencing operations.

[0013] In a second aspect, the present invention also provides a method for operating a Redis structured data table based on Java, comprising the following steps: S1: Define the structure information of the Redis structured table through a Java program. The structure information includes field attributes, primary key identifier, index rules and foreign key relationships. Persist the structure information in JSON format and automatically generate RedisSearch index mapping based on the index rules. S2: Obtain the data to be written to Redis. Based on the table definition structure information persisted in step S1, perform field type validation, required field check, primary key uniqueness validation, and foreign key association validation on the data to be written. After the validation is passed, store the data in Redis in JSON format, and automatically synchronize the RedisSearch index of the corresponding fields of the data. S3: Construct a query syntax tree containing recursive nested logical relationships and field judgment conditions through Java objects, automatically convert the query syntax tree into a RedisSearch query DSL, execute the query DSL to obtain the target data in Redis, and provide SQL-like query construction interaction during the construction of the query syntax tree; S4: Based on the foreign key relationships in the structure information defined in step S1, identify the related fields between the main table and the child table, automatically query the child table data corresponding to the main table data, and complete the joint matching and merging of the main table and child table data through system logic, simulating the JOIN operation of traditional databases; S5: For the query results obtained in step S3 or the multi-table data merged in step S4, call the preset COUNT, SUM, AVG, and GROUPBY aggregate functions to perform data statistical calculations. It supports filtering and grouping by specified fields and outputs the calculation results in a data structure adapted to the chart.

[0014] This invention presents a Java-based Redis table system architecture, which, compared to existing technologies, has the following significant advantages: Improve response efficiency in high-frequency query scenarios: This system is built on the high-performance Redis platform. Through structured modeling and index encapsulation, it effectively improves the response speed of traditional SQL queries in high-concurrency scenarios and reduces the load on the main database.

[0015] Enables relational-like structured management capabilities: Users can define fields, indexes, primary keys, and foreign keys through table structures to achieve database-like data modeling and management, improving data consistency and standardization.

[0016] A unified syntax interface supports complex logical queries: The system encapsulates query logic based on a syntax tree structure. Users can construct multi-level nested AND, OR, and NOT conditions through the Java API, which automatically generates RedisSearch query statements, simplifying development complexity.

[0017] It has the ability to perform cross-table joins and combine structured data: Based on the foreign key information in the table definition, the system supports multi-table joins and can combine the query results in a structured way, expanding the application boundaries of Redis in structured scenarios.

[0018] Provides lightweight aggregation computing capabilities: The system has a built-in aggregation module that supports aggregation operations such as COUNT, SUM, AVG, and GROUPBY, making up for Redis's shortcomings in statistical analysis.

[0019] It can run independently without relying on traditional databases: the system has complete data definition, query and analysis capabilities, and can be used as a lightweight database in some business scenarios, reducing dependence on traditional databases. Attached Figure Description

[0020] Figure 1 This is a framework diagram of a Java-based Redis structured data table operating system according to the present invention.

[0021] Figure 2 This is a diagram illustrating the process of mapping structured data from a traditional tabular format to a RedisJSON storage structure, and then using RedisSearch to perform index queries, which is a preferred embodiment of the process.

[0022] Figure 3 This is a core module architecture diagram of a preferred embodiment.

[0023] Figure 4 This is a flowchart illustrating the creation of a table structure and the definition of related fields in a preferred embodiment.

[0024] Figure 5 This is a flowchart of a preferred embodiment of the data writing operation.

[0025] Figure 6 This is a flowchart of a preferred embodiment for querying data.

[0026] Figure 7 This is a flowchart of a preferred embodiment of multi-table query. Detailed Implementation

[0027] Please see Figure 1-7 As shown, this invention provides a Java-based Redis structured data table operating system, including: The module includes a table definition module, a data module, a query syntax module, a multi-table operation module, and an aggregation module. The table definition module is used to define structured tables for Redis data. The structured tables include fields, primary keys, indexes, and foreign keys. The table definition module persists the table definition structure in JSON format, automatically generates RedisSearch index mappings during the table definition stage, and implements data writing, querying, and referencing operations through reusable table definitions. The data module is used to insert and validate data according to the definition of the structured table, and to achieve strong data consistency and business security in Redis through the structure-driven mechanism, and automatically synchronize the relevant field indexes after each data write. The query syntax module is used to construct query condition syntax trees through Java objects. The query condition syntax trees support recursive nested combination of logical relationships and field judgment conditions. The query syntax module automatically converts the constructed query condition syntax trees into RedisSearch query DSL and provides a SQL-like query construction experience. The multi-table operation module is used to automatically implement data referencing, union matching and merging between the main table and the child table based on the foreign key information in the table definition output by the table definition module, simulate foreign key and JOIN behavior, and complete data merging through system logic control. The aggregation module provides the COUNT, SUM, AVG, and GROUPBY aggregate functions, which process the aggregation logic on the query results through the system, and support field filtering, grouping, and chart data structure output.

[0028] This embodiment takes the "product management" business scenario of an e-commerce platform as an example to explain in detail the specific implementation process of the Java-based Redis structured data table operating system and operation methods. The scenario needs to realize the storage of basic product information, multi-dimensional query, product and category table association query, and sales data statistical analysis. In the Spring Boot project, the table definition module, data module, query syntax module, multi-table operation module, and aggregation module are encapsulated as independent JavaBeans and injected into the Spring container through the @Component annotation. Each module interacts with Redis through RedisTemplate. Configuration parameters: Configure Redis connection information in application.yml (host address 127.0.0.1, port 6379, password blank), and set the RedisJSON serialization method to Jackson2JsonRedisSerializer, and unify the RedisSearch index prefix to "idx:".

[0029] The structured definitions for the "Product Table" and "Category Table" are constructed using the table definition module, as detailed below: Product category definition: Field attributes: categoryId (primary key, Long type, not null), categoryName (String type, not null, index field), parentId (Long type, nullable, foreign key related to its own categoryId, used to indicate category level); Create a text index for the categoryName field to support fuzzy searches; The `saveTableDefinition(String tableName, TableStructurestructure)` method stores the table structure in JSON format into the Redis key "table:definition:category", and automatically generates the RedisSearch index "idx:category". The index mapping statement is automatically converted by the module.

[0030] The product table definition includes: Field attributes: productId (primary key, Long type, not null), productName (String type, not null, indexed field), price (Double type, not null, indexed field), stock (Int type, not null), categoryId (Long type, not null, foreign key related to categoryId in the category table); Indexing rules: create a text index for productName and a numeric range index for price; Persistence operation: The same method is called to store the table structure in Redis with the key "table:definition:product", automatically generating the RedisSearch index "idx:product". The index mapping statement includes a foreign key association verification flag to ensure that association verification is triggered when subsequent data is written.

[0031] The data module is used to insert and validate product category data and product data. The specific implementation is as follows: Product category data insertion: Data to be inserted: {"categoryId":1,"categoryName":"electronic products","parentId":0}, {"categoryId":2,"categoryName":"mobile phone","parentId":1}; Data module call: validateData(StringtableName,Map<String,Object> The `data` method verifies that `categoryId` is not null and is of type `Long`, and that `categoryName` is not null. Since `parentId=0` means there is no associated data, the foreign key validation is skipped. Storage and Index Synchronization: After successful verification, the data is stored in Redis in JSON format with keys "data:category:1" and "data:category:2" respectively. The module automatically calls the json.set command of the Redisjson module to synchronize the data to the "idx:category" index. After the JSON.SET data is successfully written, if the data matches the PREFIX rules defined in the table construction -> index creation definition, the index update will be automatically triggered. Product data insertion: Data to be inserted: {"productId":1001,"productName":"XX Brand Mobile Phone Pro","price":4999.0,"stock":500,"categoryId":2}; Validation process: In addition to field type and NOT NULL checks, foreign key validation is performed by querying the category table data through the Redis atomic operation EXISTSdata:category:2. Once the existence of categoryId=2 is confirmed, the validation passes. Storage and index synchronization: When data is stored in the key "data:product:1001", the module automatically synchronizes the "idx:product" index without the need to manually execute the index update command.

[0032] The query syntax module enables multi-dimensional product queries. Taking "querying mobile phone products with category ID 2 and price less than 5000 yuan" as an example, the specific implementation is as follows: Building a query syntax tree: Constructing query conditions through Java object chaining. QueryTree queryTree = QueryTree.builder(); .andCondition("categoryId", Operator.EQ, 2) / / Equals the condition; .andCondition("price", Operator.LT, 5000.0) / / Less than the condition; .build(); The query syntax module calls the `convertToDsl(String tableName, QueryTree queryTree)` method to automatically convert the syntax tree into a RedisSearch query DSL. FT.SEARCH idx:product "@categoryId:[2 2] @price:[0 5000]"; RETURN 0.

[0033] The module executes the DSL and deserializes the query results into a list of Product objects, ultimately returning the product data with product ID 1001. Throughout the process, developers do not need to manually write the DSL; they can complete the query construction simply through a chain of SQL-like calls.

[0034] The multi-table operation module enables related queries between the product table and the product category table. Taking "querying the product with product ID 1001 and its category name" as an example, the specific implementation is as follows: Relationship identification: The module reads the table definition of the product table, identifies categoryId as a foreign key, and associates it with categoryId in the category table; Data joint matching: First, query the product table data: retrieve the product data using the getById("product",1001) method, and get categoryId=2; Automatically associate category tables: The module calls joinTable(String mainTable, String joinTable, ... The `String mainKey, String joinKey, Object mainValue` method queries the category table based on `categoryId=2` and retrieves `categoryName="mobile phone"`. The product data and category data are merged into a ProductWithCategory object, and the returned result includes productId=1001, productName="XX Brand Mobile Phone Pro", and categoryName="Mobile Phone", simulating the INNERJOIN operation of a traditional database.

[0035] The aggregation module performs statistical analysis on the product table data. Taking "statistical analysis of the quantity and average price of products in each product category" as an example, the specific implementation is as follows: Call the `groupByAggregate(String tableName, String groupField, List<AggregateFunction>functions)` method of the aggregation module, set the grouping field to `categoryId`, and the aggregation functions to `COUNT(productId)` (to count the number of products) and `AVG(price)` (to calculate the average price). Aggregation calculation execution: The module first queries all product data through RedisSearch, then groups the data by categoryId in memory, performs statistical calculations, and obtains the results: the product quantity corresponding to categoryId=2 is 1, and the average price is 4999.0. The aggregation results are then converted into a JSON array format. This format can be directly used as the data source for ECharts bar charts, enabling a visual display of the product quantity and average price for each category. Through the above implementation steps, the structured definition, data validation and storage, multi-dimensional querying, cross-table joins, and aggregation statistics of the product and category tables were successfully achieved, meeting the core business needs of e-commerce product management. In a scenario with 100,000 product data entries, the response time for a single product query is ≤5ms, the response time for multi-condition combined queries is ≤20ms, and the response time for aggregation statistics (grouped by category) is ≤50ms. Compared with traditional MySQL databases (query response time ≥100ms under the same data volume), the performance is improved by more than 50%. Usability verification: Developers do not need to manually write RedisSearch index creation statements and DSL query statements. Through table definition reuse and SQL-like query construction, development efficiency is improved by 60%, and the data validation mechanism reduces business anomalies caused by data format errors by more than 80%. This embodiment demonstrates that the Java-based Redis structured data table operating system and operation method of the present invention can effectively make up for the deficiencies of native Redis in structured data management, complex queries and multi-table joins, while maintaining the high performance advantages of Redis. It can be widely used in business scenarios such as e-commerce, finance, and logistics that require high performance and complex data processing.

[0036] In a second aspect, the present invention also provides a method for operating a Redis structured data table based on Java, comprising the following steps: S1: Define the structure information of the Redis structured table through a Java program. The structure information includes field attributes, primary key identifier, index rules and foreign key relationships. Persist the structure information in JSON format and automatically generate RedisSearch index mapping based on the index rules. S2: Obtain the data to be written to Redis. Based on the table definition structure information persisted in step S1, perform field type validation, required field check, primary key uniqueness validation, and foreign key association validation on the data to be written. After the validation is passed, store the data in Redis in JSON format, and automatically synchronize the RedisSearch index of the corresponding fields of the data. S3: Construct a query syntax tree containing recursive nested logical relationships and field judgment conditions through Java objects, automatically convert the query syntax tree into a RedisSearch query DSL, execute the query DSL to obtain the target data in Redis, and provide SQL-like query construction interaction during the construction of the query syntax tree; S4: Based on the foreign key relationships in the structure information defined in step S1, identify the related fields between the main table and the child table, automatically query the child table data corresponding to the main table data, and complete the joint matching and merging of the main table and child table data through system logic, simulating the JOIN operation of traditional databases; S5: For the query results obtained in step S3 or the multi-table data merged in step S4, call the preset COUNT, SUM, AVG, and GROUPBY aggregate functions to perform data statistical calculations. It supports filtering and grouping by specified fields and outputs the calculation results in a data structure adapted to the chart.

[0037] The above embodiments are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A Java-based Redis structured data table operating system, characterized in that, The application relates to a Redis data management system, which comprises a table definition module, a data module, a query syntax module, a multi-table operation module and an aggregation module. The table definition module is used for defining a structured table for Redis data, the structured table comprises fields, a primary key, indexes and foreign keys, the table definition module persists table definition structures in a JSON format, automatically generates RedisSearch index mappings in a table definition stage, and realizes data writing, query and reference operations through reusable table definitions. The data module is used for completing data insertion and verification according to the definition of the structured table, and realizes data strong consistency and business security guarantee in Redis through a structure driving mechanism, and automatically synchronizes related field indexes after each data writing. The query syntax module is used for constructing a query condition syntax tree through a Java object, the query condition syntax tree supports recursive nested combination of logical relations and field judgment conditions, the query syntax module automatically converts the constructed query condition syntax tree into a RedisSearch query DSL, and provides a SQL-like query construction experience. The multi-table operation module is used for automatically realizing data reference, joint matching and merging of a master table and a sub-table based on foreign key information in the table definition output by the table definition module, simulating foreign key and JOIN behaviors, and completing data merging through system logic control. The aggregation module is used for providing COUNT, SUM, AVG and GROUPBY aggregation functions, processing aggregation logic on query results through a system, and supporting field filtering, grouping and chart data structure output. The table definition module automatically drives data verification, index generation and foreign key reference operations through table definition. 2.The Java-based Redis structured data table operation system of claim 1, wherein, In the data module, the field index synchronization process after data writing does not need manual intervention of a user.

3. The Java-based Redis structured data table operating system of claim 1, wherein, The query syntax module encapsulates the DSL query logic of RedisSearch in the form of a syntax tree, supports nested combination of complex logical relations, improves the expression ability of query conditions, and reduces the dependence of a user on RedisSearch syntax details.

4. The Java-based Redis structured data table operating system of claim 1, wherein, The multi-table operation module constructs a Redis-based master table and sub-table data reference mechanism, and the joint matching and merging process is triggered according to foreign key association relations.

5. The Java-based Redis structured data table operating system of claim 1, wherein, The aggregation module extends the limitation of RedisSearch without aggregation capability, provides a lightweight data statistical analysis capability for a business, and supports adaptation of a data source format of a common data visualization tool through chart data structure output.

6. The Java-based Redis structured data table operating system of claim 1, wherein, In the foreign key verification process of the data module, whether target foreign key corresponding data exists is inquired through an atomic operation of Redis.

7. The Java-based Redis structured data table operating system of claim 1, wherein, The GROUPBY grouping operation of the aggregation module supports multi-field combined grouping, and the priority of the multi-field combined grouping is determined through a field order parameter input by a user. The query syntax module provides a SQL-like query construction experience, supports construction of query conditions through a chain calling method, and the parameters of the chain calling method correspond to keywords of a SQL syntax.

8. The Java-based Redis structured data table operating system of claim 1, wherein, ​ 9. The Java-based Redis structured data table operating system of claim 1, wherein, The table definition module loads the persisted JSON format table definition through Java reflection mechanism when multiplexing table definition, and adapts the structure requirement of data writing, querying and referencing operation.

10. A method for operating a Java-based Redis structured data table, the method comprising: receiving a Java-based Redis structured data table; and executing a Java-based Redis structured data table operation on the Java-based Redis structured data table. The method comprises the following steps: S1: defining the structure information of the Redis structured table through a Java program, wherein the structure information comprises field attributes, primary key identification, index rules and foreign key association relationship, persistently storing the structure information in a JSON format, and automatically generating RedisSearch index mapping based on the index rules; S2: obtaining data to be written into Redis, performing field type checking, mandatory item checking, primary key uniqueness checking and foreign key association checking on the data to be written according to the persisted table definition structure information in step S1, storing the data in a JSON format into Redis after the checking is passed, and automatically synchronizing the RedisSearch index of the corresponding field of the data; S3: constructing a query syntax tree containing recursive nested logical relationship and field judgment condition through a Java object, automatically converting the query syntax tree into a RedisSearch query DSL, executing the query DSL to obtain target data in Redis, and providing a class SQL query construction interaction in the process of constructing the query syntax tree; S4: identifying the association fields of the master table and the sub-table based on the foreign key association relationship in the table definition structure information in step S1, automatically querying the sub-table data corresponding to the master table data, completing joint matching and merging of the master table data and the sub-table data through system logic, and simulating JOIN operation of a traditional database; S5: calling preset COUNT, SUM, AVG and GROUPBY aggregation functions to perform data statistical calculation on the query result obtained in step S3 or the multi-table data after merging in step S4, supporting specified field filtering and grouping processing, and outputting the calculation result in a chart-adapted data structure.

Images