Data retrieval system, method and storage medium

Abstract

The present disclosure provides a data retrieval system, method and storage medium, wherein the system comprises an interface management system, a business processing system and an ElasticSearch cluster management system; the interface management system dynamically constructs a user interface to generate query expression information in a preset format; the business processing system analyzes and processes the query expression information to generate ElasticSearch retrieval information; and the ElasticSearch cluster management system mainly comprises ElasticSearch index information management and unified management of storage medium resources. The present disclosure can quickly respond to query requests, is suitable for various query and statistical scenarios, reduces the development workload, reduces the query time consumption and improves the query efficiency, and can accurately analyze the retrieval requirements of users to improve the recall rate and precision rate of retrieval results.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a data retrieval system, method, and storage medium. Background Technology

[0002] Currently, data retrieval methods typically involve inputting different data retrieval needs into a data system to find the corresponding data. However, this approach suffers from high complexity and inaccurate results. For example, foreign trade big data application systems store massive amounts of trade data. As enterprises grow and expand, the volume of trade data and the number of users also increase dramatically. When users perform data retrieval in foreign trade big data application systems, they often cannot accurately describe their information needs, resulting in complex retrieval and statistical expressions, time-consuming retrieval processes, and inaccurate retrieval and statistical results. Summary of the Invention

[0003] In view of this, one technical problem to be solved by the present invention is to provide a data retrieval system, method and storage medium.

[0004] According to a first aspect of this disclosure, a data retrieval system is provided, comprising: an interface management system for dynamically constructing a user interface and parsing interactive information input through the user interface to generate query expression information in a preset format; a business processing system for parsing and processing the query expression information to generate an Elasticsearch retrieval request instance and parsing and processing the Elasticsearch retrieval request instance to generate ElasticSearch retrieval information; and an ElasticSearch cluster management system for using the ElasticSearch retrieval information to query corresponding query result data in an ElasticSearch cluster and returning the query result data to the interface management system through the business processing system so that the query result data is displayed in the user interface.

[0005] Optionally, a data source cluster management system is used to process the source data and send the processed source data to the ElasticSearch cluster management system so that the ElasticSearch cluster management system stores the processed source data in the Elastic Search cluster.

[0006] Optionally, the query expression information includes: a JSON search expression; the interface management system includes: a UI configuration information database for storing and managing system configuration information, the system configuration information including: interface elements and related attributes of the interface elements; an interface generation module for obtaining the interface elements to be displayed and related attributes from the UI configuration information database according to user business requirements, so as to dynamically generate interface components and generate the user interface based on the Vue framework; a front-end parser module for processing the interaction information based on the binding relationship and logical relationship corresponding to the user interface, and generating a JSON search expression in a preset format; wherein, multiple query requirement description modules are set in the JSON search expression, including: a query condition module, a query range module, a query statistics content module, a sorting module, a user information module, and an interface operation module.

[0007] Optionally, the business processing system includes: an index configuration information database for storing and managing ElasticSearch index information, wherein the ElasticSearch index information includes: index name, configuration attributes, field content, aliases, word segmentation method, and retrieval method; a logic processor module for parsing each query requirement description module in the JSON retrieval expression, obtaining the corresponding ElasticSearch index information from the index configuration information database based on the parsing result, and selecting a corresponding processing algorithm to generate a retrieval request instance containing query requirement content corresponding to each query requirement description module; the processing algorithm includes: language conversion algorithm, hierarchical setting algorithm, term reorganization algorithm, and query matching algorithm; and an ES parser module for parsing each query requirement content in the retrieval request instance, generating the ElasticSearch retrieval information, and sending it to the ElasticSearch cluster management system.

[0008] Optionally, the logic processor module is used to perform validity checks on the JSON search expression, and to perform corresponding parsing processing on each query requirement description module in the JSON search expression that passes the validity check to obtain search condition information and search content information; to obtain the corresponding ElasticSearch index information based on the search condition information and search content information, and to select the corresponding processing algorithm to generate the search request instance.

[0009] Optionally, the ES resolver module is specifically used to review the IP address corresponding to the search request instance; if the IP address passes the review, the module parses the search condition information in the search request instance, determines the relationship information between the sub-conditions in the search condition information and the priority of each sub-condition based on the parsing result; it parses the search content information in the search request instance, obtains the search content, search fields, and sorting rules based on the parsing result, and generates an Elasticsearch search expression; it generates an ES request object based on the relationship information between the sub-conditions, the priority of each sub-condition, and the Elasticsearch search expression, and sends it to the Elasticsearch cluster management system.

[0010] Optionally, the ElasticSearch cluster management system is used to establish ElasticSearch index information for the data stored in the ElasticSearch server cluster, set corresponding parsers for each ElasticSearch index information, manage the ElasticSearch index information in the index configuration information database, and perform data query operations in the ElasticSearch cluster based on the ES request object; wherein, the ElasticSearch cluster includes: multiple ElasticSearch servers storing source data.

[0011] Optionally, the data source cluster management system includes: a data source server cluster for storing the source data; and a data cleaning and processing system deployed in a data cluster based on the Hadoop distributed framework, used to import new source data from the data source server cluster into the data cluster, so that the data cluster distributes the new source data to various storage nodes, forming HDFS files and generating a data warehouse; processing the data in the HDFS files and sending the processed data to the ElasticSearch cluster for storage; wherein the processing of the data in the HDFS files includes: data consistency checks, invalid and missing value detection, erroneous data removal, code unification processing, and naming convention processing.

[0012] Optionally, the query results data include: import and export trade data of at least one country.

[0013] According to a second aspect of this disclosure, a data retrieval method is provided, comprising: an interface management system dynamically constructing a user interface and parsing interactive information input through the user interface to generate query expression information in a preset format; a business processing system parsing the query expression information to generate an Elasticsearch retrieval request instance and parsing the Elasticsearch retrieval request instance to generate ElasticSearch retrieval information; an ElasticSearch cluster management system using the ElasticSearch retrieval information to query corresponding query result data in an Elastic Search cluster, and returning the query result data to the interface management system through the business processing system so that the query result data is displayed in the user interface.

[0014] Optionally, the data source cluster management system processes the source data and sends the processed source data to the ElasticSearch cluster management system, so that the ElasticSearch cluster management system stores the processed source data in the Elastic Search cluster.

[0015] Optionally, the query expression information includes: a JSON search expression; the interface management system dynamically constructs the user interface by: obtaining the interface elements to be displayed and related attributes from the UI configuration information database according to user business needs, so as to dynamically generate interface components and generate the user interface based on the Vue framework; wherein, the UI configuration information database is used to store and manage system configuration information, and the system configuration information includes: interface elements and related attributes of interface elements; the step of parsing the interactive information input through the user interface and generating query expression information in a preset format includes: processing the interactive information based on the binding relationship and logical relationship corresponding to the user interface to generate a JSON search expression in a preset format; wherein, multiple query requirement description modules are set in the JSON search expression, including: query condition module, query range module, query statistics content module, sorting module, user information module, and interface operation module.

[0016] Optionally, generating ElasticSearch retrieval information includes: parsing each query requirement description module in the JSON retrieval expression; obtaining the corresponding ElasticSearch index information from the index configuration information database based on the parsing results and selecting the corresponding processing algorithm to generate a retrieval request instance containing query requirement content corresponding to each query requirement description module; wherein, the index configuration information database is used to store and manage ElasticSearch index information, and the ElasticSearch index information includes: index name, configuration attributes, field content, alias, word segmentation method, and retrieval method; parsing each query requirement content in the retrieval request instance to generate the ElasticSearch retrieval information; wherein, the processing algorithm includes: language conversion algorithm, hierarchical setting algorithm, term reorganization algorithm, and query matching algorithm.

[0017] Optionally, generating the retrieval request instance includes: performing a validity check on the Json retrieval expression; parsing each query requirement description module in the Json retrieval expression that passes the validity check to obtain retrieval condition information and retrieval content information; obtaining the corresponding ElasticSearch index information based on the retrieval condition information and retrieval content information, and selecting the corresponding processing algorithm to generate the retrieval request instance.

[0018] Optionally, the Elasticsearch retrieval information includes: an ES request object; generating the Elasticsearch retrieval information includes: auditing the IP address corresponding to the retrieval request instance; if the IP address passes the audit, parsing the retrieval condition information in the retrieval request instance, determining the relationship information between each sub-condition in the retrieval condition information and the priority of each sub-condition itself based on the parsing result; parsing the retrieval content information in the retrieval request instance, obtaining the retrieval content, retrieval fields, and sorting rules based on the parsing result, to generate an Elasticsearch retrieval expression; and generating an ES request object based on the relationship information between each sub-condition, the priority of each sub-condition itself, and the Elasticsearch retrieval expression.

[0019] Optionally, the step of using the ElasticSearch retrieval information to query the corresponding query result data in the Elastic Search cluster includes: establishing ElasticSearch index information for the data stored in the ElasticSearch server cluster, and setting a corresponding parser for each ElasticSearch index; wherein, the Elastic Search cluster includes: multiple Elasticsearch servers storing source data; managing the ElasticSearch index information in the index configuration information database; and performing data query operations in the Elastic Search cluster based on the ES request object.

[0020] Optionally, the data source cluster management system processes the source data and sends the processed source data to the ElasticSearch cluster management system, including: importing new source data from the data source server cluster into the data cluster based on the Hadoop distributed framework, so that the data cluster distributes the new source data to various storage nodes, forming HDFS files and generating a data warehouse; wherein, the data source server cluster is used to store the source data; processing the data in the HDFS file and sending the processed data to the ElasticSearch cluster for storage; wherein, the processing of the data in the HDFS file includes: data consistency checks, invalid and missing value detection, erroneous data removal, code unification processing, and naming convention processing.

[0021] According to a third aspect of this disclosure, a computer-readable storage medium is provided that stores computer instructions which are executed by a processor as described above.

[0022] The data retrieval system, method, and storage medium disclosed herein can dynamically construct user interfaces and parse user needs based on a configured database, quickly respond to query requests, and are applicable to various query and statistical scenarios, reducing development workload; by querying data through ElasticSearch indexes, query time is reduced and query efficiency is improved; it can accurately parse user retrieval needs, improving the recall and precision of retrieval results; it reduces development costs and improves user experience. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram of modules according to an embodiment of the data retrieval system of this disclosure;

[0025] Figure 2 This is a schematic diagram of a module according to another embodiment of the data retrieval system of this disclosure;

[0026] Figure 3 This is a schematic diagram of modules according to yet another embodiment of the data retrieval system of this disclosure;

[0027] Figure 4 This is a flowchart illustrating an embodiment of the data retrieval method according to the present disclosure;

[0028] Figure 5 This is a schematic diagram of data processing in a logic processor module according to one embodiment of the data retrieval method of this disclosure;

[0029] Figure 6 This is a schematic diagram of data processing in an ES parser module according to one embodiment of the data retrieval method of this disclosure. Detailed Implementation

[0030] The present disclosure will now be described more fully with reference to the accompanying drawings, which illustrate exemplary embodiments of the present disclosure. The technical solutions of the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present disclosure, and not all embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present disclosure. The technical solutions of the present disclosure will be described in various aspects below with reference to the various figures and embodiments.

[0031] In one embodiment, such as Figure 1 As shown, this disclosure provides a data retrieval system. The source data retrieved by the data retrieval system can be import and export trade data from multiple countries, etc. The data retrieval system includes an interface management system 10, a business processing system 20, and an ElasticSearch cluster management system 30. ElasticSearch, abbreviated as ES, is a highly scalable distributed full-text search engine based on Apache Lucene, capable of storing and retrieving data in near real-time.

[0032] The interface management system 10 dynamically constructs the user interface and parses the interactive information input through the user interface to generate query expression information in a preset format. The user interface can be a webpage, etc., and the query expression information can be of various types, such as JSON search expressions.

[0033] The business processing system 20 parses and processes the query expression information, generates an Elasticsearch retrieval request instance, and then parses and processes the Elasticsearch retrieval request instance to generate ElasticSearch retrieval information.

[0034] The ElasticSearch cluster management system 30 uses ElasticSearch to retrieve information and query corresponding query results data in the Elastic Search cluster. The query results data is then returned to the interface management system through the business processing system for display in the user interface. The query results data can be import and export trade data of one or more countries, etc.

[0035] like Figure 2 As shown, the data retrieval system includes a data source cluster management system 40. The data source cluster management system 40 processes the source data (e.g., import and export trade data of multiple countries) and sends the processed source data to the ElasticSearch cluster management system 30, so that the ElasticSearch cluster management system 30 stores the processed source data in the Elastic Search cluster.

[0036] In one embodiment, such as Figure 3 As shown, the interface management system 10 includes an interface generation module 11, a front-end parser module 12, and a UI configuration information database 13. The UI configuration information database 13 can be any type of database, used to store and manage system configuration information, including interface elements and their related attributes. The interface generation module 11 retrieves the interface elements to be displayed and their related attributes from the UI configuration information database 13 according to user business requirements, dynamically generating interface components and generating the user interface based on the Vue framework. Vue is a progressive JavaScript framework for building user interfaces; other web frameworks can also be used for the user interface.

[0037] The interface generation module 11, in conjunction with the UI configuration information database 13, dynamically generates and loads the user interface. Business modules can include, for example, a transaction data retrieval module, which can be selected by the user. Based on the business module selected by the user according to business needs, the interface generation module 11 dynamically constructs a user graphical interface corresponding to the business module by reading the interface elements and related attributes from the UI configuration information database 13. Interface elements can include dialog boxes, buttons, etc.

[0038] The interface generation module 11 retrieves the elements to be displayed on the user's page and their related attributes from the UI configuration information database 13 based on the ID of the business module selected by the user. Using existing JavaScript technology, it generates interface components according to the information of each element and loads them onto the user page. The user page provides users with various services such as data query, statistical analysis, data display, and system management.

[0039] The interface generation module 11 dynamically generates a graphical interface based on the Vue progressive framework, which can directly interact with the user and provide system interaction services to the user through the user interface. This improves the reusability of system code, makes the program more scalable and readable, and more flexible. It greatly reduces the workload of developers, improves work efficiency, and reduces the company's development costs. In addition, it provides users with a dynamically loaded user interface, which can provide a faster and more flexible user experience.

[0040] In one embodiment, the front-end parser module 12 processes the interactive information based on the binding and logical relationships corresponding to the user interface, generating a JSON search expression in a preset format. The JSON search expression includes multiple query requirement description modules, such as a query condition module, a query scope module, a query statistics module, a sorting module, a user information module, and a user interface operation module.

[0041] The front-end parser module 12 is configured with parsing algorithm code written in Vue.js to parse query requirements from user interface input, generating JSON search expressions in a specific format from the user's request description. The front-end parser module 12 can directly obtain user query requirements from the user interface, respond in real-time to changes in user input data, and use existing methods and pre-set binding and logical relationships in the parsing algorithm code to analyze and reorganize the user input, generating JSON search expressions in a specific format.

[0042] The front-end parser module 12 defines JSON search expressions with a specific format, which can clearly describe user needs and the operations required by the system. The JSON search expression includes at least six query requirement description modules: SearchModelArr (query condition module), PageModel (query range module), StatisticsModel (query (statistical) content module), SortModel (sorting module), UserModel (user information module), and OperaModel (interface operation module). Each of these six modules contains its own specific keywords or expressions to express or describe user needs and the operations required by the system.

[0043] The front-end parser module 12 generates a complete JSON search expression from the interactive information input by the user through the user interface according to the preset JSON format, adds legal identifiers to the JSON search expression, encrypts and encapsulates it, and sends it to the business processing system 20.

[0044] For example, when a user queries US import trade data, the query criteria include: the trade transaction time is between "202001" and "202204", the goods description contains "ECG MACHINE" or "ECG HOLTER", the customs code starts with "90", the country of origin is "CHINA" in China, and the names and trade volumes of the top 20 purchasing companies with the largest trade volume.

[0045] For the user's search request above, the front-end parser module 12 generates the following JSON search expression:

[0046]

[0047]

[0048] The SearchModelArr module describes the user's search criteria and the logical relationships between them. Keywords are set within the SearchModelArr module to describe the fields the user needs to query, the content of the query, the algorithm used, and the logical relationships between the query results.

[0049] For example, in the SearchModelArr module of the JSON search expression above, the query conditions are: trade transaction time between "202001" and "202204", goods description containing "ECG MACHINE" or "ECG HOLTER", customs code starting with "90", and country of origin "CHINA". The system sets the first-level query conditions as time, goods description, customs code, and country of origin, with each condition connected by an AND relationship.

[0050] For each query condition, select an algorithm, set query fields, and set keywords. Taking setting a time query condition as an example: Set the query field: "FieldESKey":["Months"], the field name is obtained from the information configuration database; Set the query content: "FieldValue":["202001","202204"], the query content is directly read from the user's data text box; Set the query type: "FieldType":"1", the query type includes various options such as fuzzy query, exact query, prefix query, term transformation query, etc.; Set the enabled algorithm: "AlgFlag":"1", set to 0 if the algorithm is not enabled.

[0051] Set the algorithm name: "AlgValue":"Conj". The system offers several algorithms such as Conj, Near, and Not for users to choose from. Set the search term interval: "AlgNum":null. After selecting an algorithm, users can set the interval between search terms. Set whether to query from newly added data: "IsNew":false. After setting this option, the system will automatically query trade data added in the recent period.

[0052] The PageModel module describes the user's search scope and the parameters for handling the returned result set. The PageModel module sets parameters such as index name, number of rows returned in the query results, whether to summarize the query results, whether to query month-over-month comparisons, and whether to export to Excel.

[0053] For example, in the PageModel module of the JSON search expression above, "PageSize" indicates that the user has set up pagination for the query, displaying 10 records per page; "CurrentPage" indicates that the user wants to query the data on the third page; "Fields" indicates the field names of the content the user wants to query; "TableName" indicates the country identifier of the country index the user wants to query; "IsXls" indicates whether the user wants to export the query results to an Excel spreadsheet; "IfTotal" indicates whether the user wants to summarize the query results; and "Ifcompare" indicates whether the user wants to calculate the month-on-month comparison.

[0054] The StatisticsModel module is used to describe the user's search (statistical) content information. The StatisticsModel module contains keywords to describe the fields used in the query, statistical content, statistical items, statistical levels, the number of statistical items, statistical algorithms, and other information.

[0055] For example, in the StatisticsModel module of the JSON search expression above, "StaColumn": importer is the field name of the buyers that the user wants to query and count; "StaNum": 20 indicates the number of buyers the user wants to count; "StaItem": is the item content and statistical item that the user wants to count. In this item, the user can calculate the sum (sum), average (avg), maximum (max), minimum (min), etc. of the statistical items.

[0056] The SortModel module describes the sorting information used by the user's search, including the sorting field and sorting method. For example, in the SortModel module of the JSON search expression above, "SortField" sets the user's sorting field; "SortType" sets the user's sorting method; and UserModel describes the user's personal information, including user ID, system version, system language, user IP address, and a unique code for this operation.

[0057] In the UserModel module of the JSON search expression above, "UserID" represents the user's unique identifier; "Lang" represents the user's system language; "Version" represents the user's system version; and "GUID" is the unique identifier of the query operation performed by the user. OpeModel describes the query operation information, such as whether the query was added to favorites, the query name, etc.

[0058] In the OpeModel module of the above JSON retrieval expression, "saveOpe" indicates whether the user saves the query to their favorites; "opeMainType" represents the business module of the user's query operation.

[0059] In one embodiment, such as Figure 3 As shown, the business processing system 20 includes an index logic processor module 21, an ES parser module 22, and a configuration information database 23. The index configuration information database 23 stores and manages Elasticsearch index information, including index name, configuration attributes, field content, aliases, tokenization methods, and retrieval methods. The index configuration information database 23 can be various databases, such as relational databases. Database administrators can automatically create indexes by configuring index names, sharding formats, number of replicas, and different tokenizers. Multiple methods can be used to manage and create Elasticsearch index information.

[0060] For example, when creating an index on Russian import data, the database administrator sets the index name, field content, data volume, number of shards, number of replicas, and other configuration information in the index configuration information database 23. Then, a Python-based index creation tool is invoked. This tool connects to the index configuration information database 23, reads the Russian import configuration information, configures it according to the ElasticSearch index creation syntax, and creates the index. ElasticSearch index information enables application systems to automatically match field names, tokenizers, and retrieval methods during queries or statistics, making the query function applicable to all query and statistical analysis modules in the system. It is flexible and suitable for retrieving trade data from different countries, in various languages, and with different index structures.

[0061] In one embodiment, the logic processor module 21 performs corresponding parsing processing on each query requirement description module in the JSON retrieval expression, obtains the corresponding ElasticSearch index information from the index configuration information database based on the parsing results, and selects the corresponding processing algorithm to generate a retrieval request instance containing query requirement content corresponding to each query requirement description module. The processing algorithms include language conversion algorithms, hierarchical setting algorithms, term reorganization algorithms, and query matching algorithms.

[0062] For example, the logic processor module 21 can perform validity checks on JSON search expressions. For JSON search expressions that pass the validity check, it performs corresponding parsing processing on each query requirement description module to obtain search condition information and search content information. Based on the search condition information and search content information, the logic processor module 21 obtains the corresponding ElasticSearch index information and selects the corresponding processing algorithm to generate a search request instance. The information in the search request instance includes search condition information and search content information, etc.

[0063] In one embodiment, the logic processor module 21 is used for logical processing of query and statistical analysis business, and is configured with processing algorithms such as system verification, source data calculation, business rules, and logical processing. The logic processor module 21 receives the user request (JSON search expression) sent by the front-end parser module 12, and after parsing, logical analysis, and algorithm selection, forms a GitsSearch request instance (retrieval request instance) in a specific format, which is then sent to the ES parser module 22.

[0064] The logic processor module 21 receives the JSON search expression sent by the front-end parser module 12. It can diagnose and predict the JSON search expression, checking the legality of the user's request. The logic processor module 21 decodes the JSON search expression and performs parsing, logical processing, and algorithm selection on each module according to the format of the JSON search expression, including query conditions, query content, and query scope. The logic processor module 21 then configures the user's request based on the index information in the index configuration information database 23.

[0065] Logic processor module 21 is equipped with a variety of logic processing and text parsing algorithms:

[0066] Multilingual Conversion: The logic processor module 21 incorporates a language conversion algorithm. Logic processor module 21 supports mutual translation between Chinese and English, Japanese, Korean, Russian, French, Spanish, Portuguese, German, Vietnamese, and Arabic, accurately and efficiently converting source language text into target language text. Logic processor module 21 converts query terms in the query conditions into multiple languages, then performs OR operations using logical relational operators to make the query conditions in various languages ​​OR relationships, thus expanding the scope of the query plugin. The language conversion algorithm converts the query content into multiple languages, allowing query operations to be performed within the indexes of data in different languages, significantly increasing the range of the result set.

[0067] Multi-level setting: The logic processor module 21 includes a multi-level setting algorithm. This algorithm sets the conditions in the query criteria according to their logical relationships, using a three-level (AND-OR-AND) structure. Specifically, it sets the first-level conditions to an AND relationship; the second-level conditions to an OR relationship; and the third-level conditions to an AND relationship. This multi-level setting clearly and comprehensively expresses the user's query intent, ensuring a clear hierarchy and logical flow between the query conditions.

[0068] For example, the query conditions are: trade period from February 2021 to January 2022; and the product description must begin with "zinc" and contain "sulfate", or contain "plastic toy". For this query condition, the hierarchy setting algorithm, based on the user's query intent, sets "trade period" and "product description" as the first level, with an AND relationship between the two conditions; sets the relationship between the two "product descriptions" as the second level, with an OR relationship between the two conditions; and sets "zinc" and "contains sulfate" in the first product description query condition as the third level, with an AND relationship between the two conditions.

[0069] Precise and Fuzzy Matching: The logic processor module 21 provides users with various search algorithms for both precise and fuzzy matching. Precise queries are relatively simple, offering a "what you input is what you get" approach without word segmentation analysis; the document must contain the entire search term. For fuzzy queries, the logic processor module 21 offers various fuzzy algorithms, such as full-text search, wildcard algorithms, prefix algorithms, stemming algorithms, Conj algorithms, Near algorithms, and term recombination algorithms. When parsing the user's query expression, the logic processor module 21 selects an algorithm based on the query expression and then performs logical operations on the query terms.

[0070] Term Reorganization: The logic processor module 21 includes a term reorganization algorithm. This algorithm recombines the contents of the query conditions according to a permutation algorithm, forming new query terms. For example, if the query terms are the three words A, B, and C, the term reorganization algorithm calculates and reorganizes the query terms, making the query conditions into query pairs: AB C, ACB, BAC, BC A, CA B, CB A, ABC, ACB, BAC, BCA, CAB, CBA. The term reorganization algorithm can reorganize each query term to the maximum extent, making each combination an independent query condition. Furthermore, it sets OR relationships between the query conditions to expand the query scope and better meet the user's query needs.

[0071] After parsing the JSON search expression, the logic processor module 21, in conjunction with the index information such as index name, field name, and analysis method in the index configuration information database 23, replaces and sets the query range, query content, and query algorithm in the JSON search expression according to the index information. The logic processor module 21 then reconstructs the query conditions, query range, query content, and system operations into a fixed-format and keyword-based GitsSearch request instance (search request instance), as shown in the figure below:

[0072]

[0073]

[0074] In one embodiment, the ES parser module 22 parses the query requirements in the retrieval request instance, generates ElasticSearch retrieval information, and sends it to the ElasticSearch cluster management system 30. The ES parser module 22 verifies the IP address corresponding to the retrieval request instance. If the IP address passes verification, the ES parser module 22 parses the retrieval condition information in the retrieval request instance, and determines the relationship information between the sub-conditions in the retrieval condition information and the priority of each sub-condition based on the parsing results.

[0075] The ES parser module 22 parses the search content information in the search request instance, and obtains the search content, search fields, and sorting rules based on the parsing results to generate an Elasticsearch search expression. The ES parser module 22 generates an ES request object based on the relationships between sub-conditions, the priority of each sub-condition, and the Elasticsearch search expression, and sends it to the Elasticsearch cluster management system 30.

[0076] In one embodiment, the ES parser module 22 parses the GitsSearch request instance sent by the logic processor module 21 into a request with a syntax format that Elasticsearch can recognize, and sends it to the Elasticsearch cluster management system 30 to retrieve data. The ES parser module 22 rewrites and encapsulates the algorithms of the publicly available Elasticsearch API by rewriting and encapsulating the Elasticsearch distributed full-text search API, making it applicable to data querying and statistical analysis in various industries based on Elasticsearch data storage. The ES parser module 22 receives the GitsSearch request instance sent by the logic processor module 21 synchronously or asynchronously, parses and runs the content of each module according to the information modules contained in the request instance, forming a request (GitsSearch request instance) with a syntax that Elasticsearch can recognize, and submits it to the Elasticsearch distributed cluster to perform data query operations on the corresponding data index. The ES parser module 22 obtains the response search result data and returns it to the logic processor module 21.

[0077] In one embodiment, such as Figure 3 As shown, the Elasticsearch cluster management system 30 establishes Elasticsearch index information for the data stored in the Elasticsearch server cluster and sets corresponding parsers for each Elasticsearch index. The Elasticsearch cluster management system 30 manages the Elasticsearch index information in the index configuration information database 23. Based on ES request objects, the Elasticsearch cluster management system 30 performs data query operations in the Elasticsearch cluster, which includes multiple Elasticsearch servers storing source data, including trade data from multiple countries.

[0078] To quickly respond to query requests and improve query flexibility, ElasticSearch technology was introduced into the data retrieval system, and the data sources in the data retrieval system were stored in ElasticSearch. ElasticSearch is a Lucene-based search server that provides a distributed, multi-user, full-text search engine. Deployed at the bottom layer of the application system, it stores trade data from various countries. The ElasticSearch cluster management system 30 manages the index configuration information database 23, sets cluster parameters, and provides index management functions such as dynamic index creation, viewing cluster status, viewing all indexes, searching documents, and deleting indexes. Multiple Linux servers equipped with the ElasticSearch distributed, highly scalable, and real-time-enabled search and data analysis engine are deployed in the ElasticSearch cluster management system 30 to store trade data from various countries.

[0079] In the Elasticsearch cluster management system 30, indexes are created based on country identifiers and import / export identifiers in the data. When creating indexes, the Elasticsearch cluster management system 30 sets the shard size and number of replicas according to a formula based on the data volume. Simultaneously, the Elasticsearch cluster management system 30 sets a corresponding Gits parser for each index based on the different languages ​​used in the trade data, the differences in data content, and the different retrieval requirements. The Gits parser can segment documents according to various forms such as language, stemming, singular / plural, synonyms, and antonyms, enabling users to accurately find the information they need during retrieval.

[0080] In one embodiment, such as Figure 3 As shown, the data source cluster management system 40 includes a data cleaning and processing system 41 and a data source server cluster 42. The data source server cluster 42 stores the source data, and the data cleaning and processing system 41 is deployed in a data cluster based on the Hadoop distributed framework. It imports new source data from the data source server cluster 41 into the data cluster, so that the data cluster distributes the new source data to various storage nodes, forming HDFS files and generating a data warehouse.

[0081] The data cleaning and processing system 41 processes the data in the HDFS file and sends the processed data to the ElasticSearch cluster for storage. The processing of the data in the HDFS file includes data consistency checks, invalid and missing value detection, erroneous data removal, code unification processing, and naming convention processing.

[0082] The source data includes import and export trade data from multiple countries. The company's original import and export trade data is divided by country and stored in separate databases and servers. The storage of import and export trade data from multiple countries is based on a relational data source server cluster 42. The processing of the original import and export trade data is carried out on a Linux cluster equipped with the Hadoop distributed data processing framework. A data cleaning and processing system 41 (e.g., the GitsDataMG data processing system) performs operations such as importing and exporting new data and data cleaning. The data source server cluster 42 is separated from other systems and physically isolated from the external network, ensuring the security and stability of the data source.

[0083] The data cleaning and processing system 41 is deployed in a Linux cluster with a Hadoop distributed data processing framework and is configured with a series of data preprocessing algorithms. The data cleaning and processing system 41 performs data processing on HDFS in Hadoop, such as data consistency checks, handling invalid and missing values, removing erroneous data (duplicate data), code standardization, and naming conventions.

[0084] When new data is added, the data cleaning and processing system 41 imports the data from the data source server cluster 42 into the Hadoop cluster via the internal network. The Hadoop cluster distributes the data across various storage nodes, forming HDFS files. The data cleaning and processing system 41 processes the HDFS data to generate a data warehouse. The data cleaning and processing system 41 then uses scheduled tasks to periodically import the processed data into the ElasticSearch data cluster system 30, decoupling data addition and querying, thereby enabling flexible querying, elastic synchronization, and efficient updates of massive amounts of data.

[0085] Figure 4 This is a flowchart illustrating an embodiment of the data retrieval method according to the present disclosure, as follows: Figure 4 As shown:

[0086] Step 401: The interface management system dynamically constructs the user interface and parses the interactive information input through the user interface to generate query expression information in a preset format.

[0087] Step 402: The business processing system parses the query expression information, generates an Elasticsearch retrieval request instance, and parses the Elasticsearch retrieval request instance to generate ElasticSearch retrieval information.

[0088] Step 403: The ElasticSearch cluster management system uses ElasticSearch to retrieve information and query the corresponding query results data in the ElasticSearch cluster. The query results data is then returned to the interface management system through the business processing system for display in the user interface.

[0089] In one embodiment, the data source cluster management system processes the source data and sends the processed source data to the ElasticSearch cluster management system, so that the ElasticSearch cluster management system stores the processed source data in the Elastic Search cluster.

[0090] The query expression information includes JSON search expressions. The UI management system can dynamically construct user interfaces using various methods. For example, based on user business requirements, it can retrieve the required UI elements and related attributes from the UI configuration information database to dynamically generate UI components and create the user interface based on the Vue framework. The UI configuration information database stores and manages system configuration information, including UI elements and their related attributes.

[0091] The interface management system can generate preset-format query expressions using various methods. For example, it can process interactive information based on the binding and logical relationships corresponding to the user interface to generate preset-format JSON search expressions.

[0092] There are several methods for business processing systems to generate Elasticsearch retrieval information. For example, one method involves parsing each query requirement description module in a JSON search expression, retrieving the corresponding Elasticsearch index information from the index configuration database based on the parsing results, and selecting the appropriate processing algorithm to generate a retrieval request instance containing query requirement content corresponding to each query requirement description module. The retrieval request instance then parses and processes each query requirement content to generate the Elasticsearch retrieval information.

[0093] For example, the logic parser module receives a JSON search expression, combines it with the index configuration database, parses and reassembles the request to form a GtisSearch request instance, and sends it to the ES parser module. Figure 5 This is a data processing diagram of a logic processor module in one embodiment of the data retrieval method according to the present disclosure, as shown below. Figure 5 As shown:

[0094] Step 501: The front-end parser module parses the user request, generates a JSON retrieval expression, and sends it to the logic processor module.

[0095] The front-end parser module obtains the user's query request interaction information from the dynamically generated user graphical interface, generates a JSON-formatted request expression containing multiple modules based on the interaction information, and sends it to the business logic processor.

[0096] For example, a user might initiate a trade data query request through a graphical user interface, specifying the following criteria: "Time range (January 2021 to December 2021), customs code starting with 9503, and product description containing the keywords 'TOY' or 'PEDAL CAR'." This query aims to find information on the top 10 suppliers by total export value in various countries. The front-end parser module parses the user's query criteria. Based on the obtained content, it combines the query criteria, query range, and query content into a specific format, keywords, and logical expressions, adding a valid unique identifier, and sends this string in a specific JSON format to the logic processor module.

[0097] Step 502: The logic parser module receives the JSON search expression.

[0098] Step 503: Check if the request is valid. If it is, proceed to step 504. If not, return and inform the customer to make modifications and additions.

[0099] Step 504: Parse the JSON search expression; request the index configuration database to obtain the required configuration items; select the algorithm and set the operators and operation logic.

[0100] Step 505: Generate a GitsSearch request instance.

[0101] Step 506: Send to the ES parser module and log it.

[0102] After receiving a JSON search expression, the logic parser module performs a validity check on the request represented by the JSON search expression to determine whether the request path is valid, whether it contains illegal injection attacks, and whether the unique identifier is valid. Once the validity check is passed, the logic parser module decodes and parses the JSON search expression to obtain the user's search scope, content, fields, field parser, applied algorithm, size of the returned result set, sorting fields, and sorting method.

[0103] The logic parser module connects to the index configuration information database and retrieves information such as the index name, field name, and tokenizer from the Elasticsearch cluster based on the query information in the JSON search expression. The logic parser module extracts and identifies the part-of-speech tag and position of the "search words" or "search expressions" in the search conditions. Based on the priority, logical relationships, and applicable query algorithms of each condition in the query, the logic parser module reorganizes and calculates the request. Finally, the logic parser module logically processes the search scope, fields, sorting fields, and sorting methods in the search results according to unified standards and rules.

[0104] For example, a JSON search expression is converted into a JsonObject, and the user's query conditions are obtained according to the object's properties. Parsing determines that the user needs to query export data from all countries, and the query conditions include three criteria: time range, customs code, and product description. Each of these three criteria has its own algorithm, and they are related by AND. The product description criterion contains two sub-conditions that are ORed.

[0105] After the logic parser parses the user's search criteria, it parses the query content. Through parsing, it determines that the query content consists of all information about the top 10 suppliers in terms of total export value across all countries under the given query criteria. Once the query criteria and query content are obtained, the corresponding algorithm is selected and calculated according to a fixed syntax and keywords.

[0106] The logic parser module merges search criteria and search content, encapsulating them into a GtisSearch search instance (search request instance) containing specific operators and a standard syntax format. A GtisSearch search instance is a pre-formatted, specifically encoded template of request parameters that accurately and clearly describes the mapping relationship between the user's search content and search logic, fully expressing the user's search intent. The logic parser adds a valid identifier to the GtisSearch request instance, encrypts it using a specific algorithm, and then sends the request instance to the ES parser.

[0107] In one embodiment, Elasticsearch retrieval information includes an ES request object. Various methods can be used to generate Elasticsearch retrieval information. For example, the IP address corresponding to the retrieval request instance is audited. If the IP address passes the audit, the retrieval condition information in the retrieval request instance is parsed. Based on the parsing results, the relationships between the sub-conditions and the priority of each sub-condition are determined. The retrieval content information in the retrieval request instance is parsed, and the retrieval content, retrieval fields, and sorting rules are obtained based on the parsing results to generate an Elasticsearch retrieval expression. Finally, an ES request object is generated based on the relationships between the sub-conditions, the priority of each sub-condition, and the Elasticsearch retrieval expression.

[0108] For example, the ES parser module receives GtisSearch request instances, parses the GtisSearch request instances into a syntax format that ElasticSearch recognizes, queries data from the Elastic Search cluster, and responds to user requests. Figure 6 This is a schematic diagram of data processing in an ES parser module according to one embodiment of the data retrieval method of this disclosure, as shown below. Figure 6 As shown:

[0109] Step 601: Check if the request is valid. If it is, proceed to step 603. If not, proceed to step 602: Throw an exception and return it to the client.

[0110] The ES resolver module receives GtisSearch request instances, verifies the request IP address that sent the GtisSearch request instance, and determines whether the request IP address is a valid IP address. If the request IP address is invalid, the system will throw an exception and stop the data retrieval request.

[0111] Step 603: Decode the GitsSearch request instance.

[0112] Step 604: Select the appropriate algorithm based on the GitsSearch request instance.

[0113] Step 605: Parse the GitsSearch request instance to generate a Gits search object (ES request object); parse the query conditions and set the priority and logical relationship between the conditions based on the keywords; parse the query content and set the corresponding algorithm based on the query dimensions and content.

[0114] Step 606: Retrieve the indexes in the ElasticSearch cluster according to the user's search requirements.

[0115] Step 607: Structure the result set and return it; write it to the log file.

[0116] After the validity check is passed, the ES parser module parses the GtisSearch request instance. Using preset keywords, the ES parser module analyzes the GtisSearch request instance, obtains the user's target based on the keywords, selects the appropriate algorithm module, and parses the search conditions and search content accordingly.

[0117] The ES parser module obtains the hierarchical relationship of GtisSearch request instances and parses the query conditions at each level from low to high priority, setting a corresponding algorithm for each query condition. The ES parser module creates query condition instances, extracts keywords and query algorithm code from each query condition, and selects the appropriate query algorithm based on the algorithm proxy. It pre-sets various query algorithms, including exact match, full-text search, prefix search, wildcard search, regular expression search, and range search. The ES parser module reorganizes and packages these query algorithms to form the query algorithms used for foreign trade big data.

[0118] The relationships between the various conditions set in the GitsSearch request instance are analyzed according to the hierarchy and priority of the conditions. The parsed conditions are then processed in sequence and logically, and the parsed condition bodies are organized according to a predetermined format and rules to form a complete logical operation condition suitable for ElasticSearch.

[0119] The Elasticsearch parser module obtains the dimensions of the query content based on the description of the search content in the GitsSearch request instance, and parses the statistics of each dimension. The Elasticsearch parser module retrieves the search content and search fields from the GitsSearch request instance, creates an aggregate query instance, parses the search content, search fields, sorting rules, etc., into expressions that the Elasticsearch database can recognize, and adds them to the aggregate query instance.

[0120] The Elasticsearch parser combines the search criteria and search content to form a GtisES request object, which is then sent to the Elasticsearch cluster server index to retrieve the response body from the Elasticsearch cluster server search engine. This data query method using Elasticsearch indexes avoids traversing and querying the massive amounts of basic data in the server cluster, enabling fast and accurate queries across multiple complex query dimensions, reducing query time, and improving query efficiency.

[0121] In one embodiment, this disclosure provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the method as described in any of the above embodiments.

[0122] The data retrieval system, method, and storage medium provided in the above embodiments can dynamically build user interfaces and parse user needs based on a configured database, quickly respond to query requests, and are applicable to various query and statistical scenarios, reducing development workload; querying data through ElasticSearch indexes reduces query time and improves query efficiency, has a wide range of applications, and is easy to operate; it can accurately parse user retrieval needs, improving the recall and precision of retrieval results; it reduces development costs and improves user experience.

[0123] The methods and systems of this disclosure can be implemented in many ways. For example, they can be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the methods is for illustrative purposes only, and the steps of the methods of this disclosure are not limited to the order specifically described above, unless otherwise specifically stated. Furthermore, in some embodiments, this disclosure may also be implemented as a program recorded on a recording medium, the program including machine-readable instructions for implementing the methods according to this disclosure. Thus, this disclosure also covers recording media storing programs for performing the methods according to this disclosure.

[0124] The description in this disclosure is provided for illustrative and descriptive purposes only and is not intended to be exhaustive or to limit the disclosure to its forms. Many modifications and variations will be apparent to those skilled in the art. The embodiments were chosen and described in order to better illustrate the principles and practical application of this disclosure and to enable those skilled in the art to understand this disclosure and to design various embodiments with various modifications suitable for a particular purpose.

Claims

1. A data retrieval system, comprising: The interface management system is used to dynamically construct user interfaces and parse the interactive information input through the user interface to generate query expression information in a preset format. The business processing system is used to parse and process the query expression information, generate an Elasticsearch retrieval request instance, and parse and process the Elasticsearch retrieval request instance to generate ElasticSearch retrieval information. The query expression information includes a JSON search expression, which contains multiple query requirement description modules, including a query condition module, a query range module, a query statistics content module, a sorting module, a user information module, and a user interface operation module; the ElasticSearch search information includes an ES request object. The business processing system includes: The index configuration information database is used to store and manage ElasticSearch index information; The logic processor module is used to parse and process each query requirement description module in the JSON search expression to obtain search condition information and search content information; based on the search condition information and search content information, it retrieves the corresponding ElasticSearch index information from the index configuration information database and selects the corresponding processing algorithm to generate a search request instance containing query requirement content corresponding to each query requirement description module; wherein, the information in the search request instance includes search condition information and search content information; the processing algorithm includes a language conversion algorithm, a hierarchy setting algorithm, a term recombination algorithm, and a query matching algorithm; The ES parser module is used to parse the search condition information in the search request instance, determine the relationship information between the sub-conditions in the search condition information and the priority of each sub-condition based on the parsing results; parse the search content information in the search request instance, obtain the search content, search fields and sorting rules based on the parsing results, and generate an Elasticsearch search expression; generate an ES request object based on the relationship information between the sub-conditions, the priority of each sub-condition and the Elasticsearch search expression, and send it to the Elasticsearch cluster management system. The ElasticSearch cluster management system is used to retrieve corresponding query result data in the ElasticSearch cluster using the ElasticSearch retrieval information, and then return the query result data to the interface management system through the business processing system so that the query result data can be displayed in the user interface.

2. The system of claim 1, further comprising: The data source cluster management system is used to process the source data and send the processed source data to the ElasticSearch cluster management system so that the ElasticSearch cluster management system stores the processed source data in the Elastic Search cluster.

3. The system as described in claim 2, wherein, The interface management system includes: A UI configuration information database is used to store and manage system configuration information, which includes: interface elements and related attributes of the interface elements; The interface generation module is used to obtain the interface elements to be displayed and related attributes from the UI configuration information database according to the user's business requirements, so as to dynamically generate interface components and generate the user interface based on the Vue framework. The front-end parser module is used to process the interactive information based on the binding and logical relationships corresponding to the user interface, and generate the JSON search expression in a preset format.

4. The system as described in claim 3, wherein, The ElasticSearch index information includes: index name, configuration attributes, field content, aliases, tokenization method, and retrieval method.

5. The system as described in claim 4, wherein, The logic processor module is used to perform validity checks on the JSON search expression, and to perform corresponding parsing processing on each query requirement description module in the JSON search expression that passes the validity check.

6. The system of claim 5, wherein, The ES resolver module is specifically used to review the IP address corresponding to the search request instance; if the IP address passes the review, the search condition information in the search request instance is parsed.

7. The system of claim 6, wherein, The ElasticSearch cluster management system is used to establish ElasticSearch index information for data stored in the ElasticSearch server cluster, and set corresponding parsers for each ElasticSearch index; manage the ElasticSearch index information in the index configuration information database; and perform data query operations in the ElasticSearch cluster based on the ES request object; wherein, the ElasticSearch cluster includes: multiple ElasticSearch servers storing source data.

8. The system as claimed in claim 2, wherein, The data source cluster management system includes: A data source server cluster is used to store the source data; The data cleaning and processing system is deployed in a data cluster based on the Hadoop distributed framework. It imports new source data from the data source server cluster into the data cluster, enabling the data cluster to distribute the new source data across various storage nodes, forming HDFS files and generating a data warehouse. The system then processes the data in the HDFS files and sends the processed data to the ElasticSearch cluster for storage. The processing of the data in the HDFS files includes: data consistency checks, invalid and missing value detection, erroneous data removal, code standardization, and naming convention processing.

9. The system as claimed in any one of claims 1 to 8, wherein, The query results include import and export trade data for at least one country.

10. A data retrieval method, comprising: The interface management system dynamically constructs the user interface and parses the interactive information input through the user interface to generate query expression information in a preset format. The business processing system parses the query expression information to generate an Elasticsearch retrieval request instance, and then parses the Elasticsearch retrieval request instance to generate ElasticSearch retrieval information. The query expression information includes a JSON search expression, which contains multiple query requirement description modules, including a query condition module, a query range module, a query statistics content module, a sorting module, a user information module, and a user interface operation module; the ElasticSearch search information includes an ES request object. The generated ElasticSearch retrieval information includes: The JSON search expression is parsed to obtain search condition information and search content information. Based on the search condition information and search content information, the corresponding ElasticSearch index information is retrieved from the index configuration information database, and a corresponding processing algorithm is selected to generate a search request instance containing search content corresponding to each search requirement description module. The information in the search request instance includes search condition information and search content information. The processing algorithms include language conversion algorithms, hierarchical setting algorithms, term recombination algorithms, and query matching algorithms. The search condition information in the search request instance is parsed, and the relationship information between each sub-condition and the priority of each sub-condition are determined based on the parsing results. The search content information in the search request instance is parsed, and the search content, search fields and sorting rules are obtained based on the parsing results to generate an Elasticsearch search expression. An ES request object is generated based on the relationship information between each sub-condition, the priority of each sub-condition and the Elasticsearch search expression, and sent to the Elasticsearch cluster management system. The ElasticSearch cluster management system uses the ElasticSearch retrieval information to query the corresponding query result data in the Elastic Search cluster, and returns the query result data to the interface management system through the business processing system so that the query result data can be displayed in the user interface.

11. The method of claim 10, further comprising: The data source cluster management system processes the source data and sends the processed source data to the ElasticSearch cluster management system, so that the ElasticSearch cluster management system stores the processed source data in the Elastic Search cluster.

12. The method of claim 11, wherein, The interface management system dynamically constructs the user interface, including: Based on user business requirements, the UI configuration information database is used to retrieve the interface elements to be displayed and related attributes, so as to dynamically generate interface components and generate the user interface based on the Vue framework. The UI configuration information database is used to store and manage system configuration information, which includes: interface elements and related attributes of the interface elements; The step of parsing the interactive information input through the user interface to generate query expression information in a preset format includes: The interactive information is processed based on the binding and logical relationships corresponding to the user interface to generate the JSON search expression in a preset format.

13. The method of claim 12, wherein, The ElasticSearch index information includes: index name, configuration attributes, field content, aliases, tokenization method, and retrieval method.

14. The method of claim 13, further comprising: Perform a validity check on the JSON search expression; For each query requirement description module in the JSON search expression that passes the validity check, corresponding parsing and processing are performed.

15. The method of claim 14, further comprising: The IP address corresponding to the search request instance is subject to review processing; If the IP address passes the verification, the search criteria information in the search request instance is parsed.

16. The method of claim 15, wherein, The step of using the ElasticSearch retrieval information to query the corresponding query result data in the ElasticSearch cluster includes: Create ElasticSearch index information for the data stored in the ElasticSearch server cluster; Configure the corresponding parser for each ElasticSearch index; The Elastic Search cluster includes: multiple Elastic Search servers that store source data; Manage the ElasticSearch index information in the index configuration information database; Data query operations are performed in the Elastic Search cluster based on the ES request object.

17. The method of claim 11, wherein, The data source cluster management system processes the source data and sends the processed source data to the ElasticSearch cluster management system, including: New source data is imported from the data source server cluster into a data cluster based on the Hadoop distributed framework, so that the data cluster distributes the new source data to various storage nodes, forming HDFS files and generating a data warehouse; wherein, the data source server cluster is used to store the source data; The data in the HDFS file is processed, and the processed data is sent to the ElasticSearch cluster for storage. The processing of data in HDFS files includes: data consistency checks, invalid and missing value detection, erroneous data removal, code standardization, and naming convention processing.

18. A computer-readable storage medium that non-transitoryly stores computer instructions, which are executed by a processor according to any one of claims 10 to 17.

Images