Big data bucket query method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a big data bucket query method and device, electronic equipment and a storage medium, and belongs to the technical field of digital medical treatment. The method comprises the steps that query information is obtained, and the query information comprises an index name, user data and a query condition; when the query condition is discrete query, performing statistical processing on the user data to obtain a dimension value of an index name; when the dimension value is greater than a preset dimension threshold value, performing bucket dividing processing on the query condition, and updating the query condition; generating bucket query statements according to the query conditions, and sending the bucket query statements to a database; and receiving a data query result returned by the database. According to the embodiment of the invention, the bucket query statements meeting the dimension threshold can be automatically generated according to different query conditions and dimension values thereof, the reference value of a big data query result is improved, the situation that data statistics is too dispersed is avoided, and the accuracy and timeliness of data query are improved.

Description

Technical Field

[0001] The present application relates to the field of digital medical technology, and in particular to a big data bucket query method and device, an electronic device, and a storage medium. Background Art

[0002] With the growth of business volume, the amount of data in digital medical databases is getting larger and larger. Front-end personnel need to perform statistical analysis on the data, and statistical analysis of the data requires the use of database aggregate query. The aggregate query performance of traditional databases has performance risks and poor compatibility. The statistical analysis of digital medical big data cannot make humanized choices of discrete and bucket queries according to actual needs, which affects the query performance of the analysis system. Bucket query is used to perform aggregate analysis and statistics on digital medical data. It can perform grouping, counting, summing, averaging and other operations according to specified conditions and fields, so as to quickly generate various data analysis reports and charts. However, since the data values ​​in the digital medical database are too divergent, customized queries need to be performed according to actual conditions. The existing big data bucket query method is difficult to implement this customized data bucket analysis, and it is also impossible to achieve dynamic interval statistics.

[0003] At present, the existing bucket query method has the problem of poor compatibility, which affects the accuracy and timeliness of data query. Therefore, how to improve the accuracy and timeliness of data query has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to propose a big data bucket query method and device, an electronic device and a storage medium, aiming to improve the accuracy and timeliness of data query.

[0005] To achieve the above purpose, a first aspect of an embodiment of the present application proposes a big data bucket query method, the method comprising: Acquire query information, wherein the query information includes an index name, user data, and query conditions; When the query condition is a discrete query, statistical processing is performed on the user data to obtain a dimension value of the index name; When the dimension value is greater than a preset dimension threshold, the query condition is bucketed and the query condition is updated; Generate a bucket query statement according to the query condition, and send the bucket query statement to the database; Receive the data query result returned by the database.

[0006] In some embodiments, obtaining query information includes: Acquire the user data and obtain the data type of the user data; According to the data type, the query condition corresponding to the index name is obtained.

[0007] In some embodiments, after obtaining the query information, the method further includes: When the query condition is a bucket query, the bucket configuration of the index name is obtained according to the query condition; Generate a bucket query statement according to the bucket configuration.

[0008] In some embodiments, performing statistical processing on the user data to obtain the dimension value of the index name includes: According to the index name and the query condition, obtaining the number of entries whose index name meets the query condition; According to the number of entries, the dimension value of the index name is obtained.

[0009] In some embodiments, the bucketing the query condition and updating the query condition includes: According to the data type of the user data, automatically generate a corresponding target bucket for the index name; Update the query condition according to the target bucket.

[0010] In some embodiments, automatically generating a corresponding target bucket for the index name according to the data type of the user data includes: Obtaining a numerical range of the user data; According to the numerical interval and the data type, a partition operation is performed on the user data to obtain partitions of the user data, wherein the number of the partitions is less than or equal to the dimension threshold; The user data that meets the requirements between the partitions is allocated to corresponding buckets to obtain the target buckets.

[0011] In some embodiments, the method further comprises: When the dimension value is less than or equal to the dimension threshold, a discrete query statement is generated according to the query condition; Sending the discrete query statement to the database.

[0012] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application proposes a large data bucket query device, the device comprising: An acquisition module, used to acquire query information, wherein the query information includes an index name, user data and query conditions; A statistical module, used for performing statistical processing on the user data to obtain the dimension value of the index name when the query condition is a discrete query; An updating module, configured to perform bucket processing on the query condition and update the query condition when the dimension value is greater than a preset dimension threshold; A generation module, used to generate a bucket query statement according to the query condition, and send the bucket query statement to the database; The receiving module is used to receive the data query result returned by the database.

[0013] To achieve the above objectives, a third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0014] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0015] The big data bucket query method and device, electronic device and storage medium proposed in this application obtain query information, and when the query condition is a discrete query, perform statistical processing on user data to obtain the dimension value of the index name; when the dimension value is greater than the preset dimension threshold, perform bucket processing on the query condition and update the query condition; generate bucket query statements according to the query condition, send the bucket query statements to the database; receive the data query results returned by the database. By automatically generating bucket query statements that meet the dimension threshold for different query conditions and their dimension values, the reference value of big data query results is improved, the situation where data statistics are too dispersed is avoided, and the accuracy and timeliness of data queries are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flowchart of the big data bucket query method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0018] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0020] First, some nouns involved in this application are analyzed: In a possible implementation, the user data is medical data, such as personal health records, prescriptions, examination reports, and other data.

[0021] In a possible implementation, the natural language text is a medical text, and the medical text may be an electronic healthcare record, an electronic personal health record, including a series of electronic records with preservation value such as medical records, electrocardiograms, and medical images.

[0022] Information query has become a channel for users to quickly obtain the information they need in many scenarios. For example, in the medical field, the medical record information required by users can be queried from massive electronic medical records based on artificial intelligence models, which helps to provide users with medical record references.

[0023] Structured Query Language (SQL) is a special purpose programming language, a database query and programming language used to access data and query, update and manage relational database systems. SQL is a high-level non-procedural programming language that allows users to work on high-level data structures. It does not require users to specify the storage method for data, nor does it require users to understand the specific data storage method. Therefore, different database systems with completely different underlying structures can use the same structured query language as an interface for data input and management. Structured query language statements can be nested, which makes it extremely flexible and powerful.

[0024] A database is a warehouse for storing data. It has a large storage space and can store millions, tens of millions, or even hundreds of millions of data. However, the database does not store data randomly, but has certain rules, otherwise the query efficiency will be very low. Today's world is an Internet world full of data, which is full of a large amount of data. That is, this Internet world is a data world. There are many sources of data, such as travel records, consumption records, browsed web pages, sent messages, etc. In addition to text-type data, images, music, and sounds are all data. A database is a computer software system that stores and manages data according to data structure. The concept of a database actually includes two meanings: First, a database is an entity, which is a warehouse that can reasonably store data. Users store transaction data to be managed in the warehouse. The two concepts of data and library are combined into a database. Secondly, a database is a new method and technology for data management. It can organize data more appropriately, maintain data more conveniently, control data more strictly, and use data more effectively. As the most important basic software, the database is the cornerstone to ensure the stable operation of the computer system.

[0025] Java is an object-oriented programming language. It not only absorbs the advantages of C++, but also abandons the difficult-to-understand concepts of multiple inheritance and pointers in C++. Therefore, Java has two characteristics: powerful functions and simple and easy to use. As a representative of static object-oriented programming languages, Java has implemented object-oriented theory very well, allowing programmers to perform complex programming with an elegant way of thinking. Java has the characteristics of simplicity, object-oriented, distributed, robust, secure, platform-independent and portable, multi-threaded, and dynamic. Java can be used to write desktop applications, Web applications, distributed systems, and embedded system applications.

[0026] Based on this, the embodiments of the present application provide a big data bucket query method and device, an electronic device and a storage medium, aiming to improve the accuracy and timeliness of data query.

[0027] The big data bucket query method and device, electronic device and storage medium provided in the embodiments of the present application are specifically illustrated through the following embodiments. First, the big data bucket query method in the embodiments of the present application is described.

[0028] The big data bucket query method provided in the embodiment of the present application relates to the field of digital medical technology. The big data bucket query method provided in the embodiment of the present application can be applied to the terminal, can also be applied to the server side, and can also be software running in the terminal or the server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or it can be configured as a server cluster or distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the big data bucket query method, etc., but is not limited to the above forms.

[0029] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0030] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0031] Figure 1 is an optional flowchart of the big data bucket query method provided in the embodiment of the present application. Figure 1 The method may include but is not limited to steps S1000 to S5000.

[0032] Step S1000, obtaining query information, wherein the query information includes an index name, user data and query conditions.

[0033] Step S2000, when the query condition is a discrete query, statistical processing is performed on the user data to obtain the dimension value of the index name.

[0034] Step S3000, when the dimension value is greater than the preset dimension threshold, the query condition is bucketed and the query condition is updated.

[0035] Step S4000, generate a bucket query statement according to the query conditions, and send the bucket query statement to the database.

[0036] Step S5000, receiving the data query result returned by the database.

[0037] Steps S1000 to S5000 shown in the embodiment of the present application obtain query information, and when the query condition is a discrete query, perform statistical processing on the user data to obtain the dimension value of the index name; when the dimension value is greater than the preset dimension threshold, perform bucket processing on the query condition and update the query condition; generate a bucket query statement based on the query condition, send the bucket query statement to the database; receive the data query result returned by the database. By automatically generating bucket query statements that meet the dimension threshold for different query conditions and their dimension values, the reference value of big data query results is improved, the situation where data statistics are too dispersed is avoided, and the accuracy and timeliness of data queries are improved.

[0038] In some embodiments, step S1000 may include but is not limited to steps S1100 to S1200: Step S1100, acquiring user data and obtaining the data type of the user data.

[0039] Step S1200: Obtain the query condition corresponding to the index name according to the data type.

[0040] In step S1100 of some embodiments, the data type of the user data is obtained by acquiring the user data. For example, when the user data is of time type, the data format is yyyy-MM-dd HH:mm:ss; when the user data is of age, the data format is 1-120 years old. In practical applications, the user data and the data type can be directly selected or specified by the user. By acquiring the data type of the user data, the user data can be analyzed and counted according to the data type to ensure the accuracy of the data query.

[0041] In step S1200 of some embodiments, after obtaining the data type, the dialog box of the query condition can be accurately opened according to the data type, so that the user can enter the query condition corresponding to the index name according to the data type. In actual applications, the index name of the database refers to the unique identifier assigned when the index is created in the database. The role of the database index name is to uniquely identify and manage the index, improve query performance, and facilitate the maintenance and management of the index. By obtaining the query condition corresponding to the index name, the user data corresponding to the index name in the database can be accurately obtained.

[0042] In some embodiments, step S1000 may include but is not limited to steps S1300 to S1400: Step S1300, when the query condition is a bucket query, the bucket configuration of the index name is obtained according to the query condition.

[0043] Step S1400, generating a bucket query statement according to the bucket configuration.

[0044] In step S1300 of some embodiments, bucket query is a commonly used technique in big data processing, mainly used to optimize query performance and data processing efficiency. Bucket query simplifies the query processing process by decomposing a data set into several smaller buckets, each containing a portion of data.

[0045] It should be noted that in actual applications, when the user chooses bucket query, a corresponding prompt box will pop up, and the user will fill in the customized data bucket configuration or interval distribution. That is, if custom bucketing is selected, a prompt box for configuring partitions will pop up. The user can divide the attribute values ​​of the user data into any number of bucket intervals according to actual needs. There cannot be any intersection between bucket intervals, and the principle of left closing and right opening is adopted.

[0046] In step S1400 of some embodiments, bucket query statements are generated according to bucket configurations. In practical applications, bucket query statements are generated through SQL statements. According to the bucket configuration, rule parameters are parsed to realize the generation of SQL for bucket query. In practical applications, a set of parsing logic is implemented in Java code to realize dynamic conversion according to bucket configuration when dynamically querying statistical fields. Among them, the bucket configuration is stored in a relational database. Among them, SQL statements are languages ​​used to interact with relational databases, used to query, insert, update and delete data, and to create and manage database objects (tables, views, indexes, etc.). Some common SQL statements include SELECT, INSERT, UPDATE, DELETE, CREATE TABLE, etc. Java statements refer to statements used in the Java programming language to implement program logic, control flow, define variables, call methods, etc. Java statements generally include declaration statements, assignment statements, conditional statements, loop statements, etc., as well as calling methods in Java class libraries. Bucket is a concept used to group and aggregate data. It can be used to group, count and analyze search results. Data buckets can be grouped according to different conditions, such as date, range, term, nested object, etc., and can be further divided into sub-buckets and sub-buckets, providing users with rich aggregation functions to help users conduct more in-depth analysis and statistics of data.

[0047] In some embodiments, step S2000 may include but is not limited to steps S2100 to S2200: Step S2100, according to the index name and the query condition, obtain the number of entries whose index name meets the query condition.

[0048] Step S2200, obtaining the dimension value of the index name according to the number of entries.

[0049] In step S2100 of some embodiments, when the query condition is a discrete query, in order to avoid the situation where the query results are too divergent due to too many latitude values, it is necessary to determine the dimension value of the index name. Discrete query generally refers to the process of querying discrete sensor data in a database. Discrete sensor data is generally used to monitor and record the status of equipment, such as temperature, humidity, etc., and these data exist in the form of discrete values.

[0050] It should be noted that, according to the index name and the query condition, the number of entries whose index name meets the query condition is obtained. For example, when the user data is age, and the query condition is to divide the interval into years, the number of entries whose index name meets the query condition is the number of different ages in the user data. For large-scale medical data, the number of entries whose index name meets the query condition often reaches dozens or even hundreds. Specifically, in the age range of 1-100 years old, there are user data at various ages, and the number of entries whose index name meets the query condition reaches 100, that is, in the discrete query of age, the number of age categories is 100. In actual applications, the database can be quickly queried through instructions to determine the number of entries whose index name meets the query condition, thereby avoiding the situation where the query results are too discrete.

[0051] In step S2200 of some embodiments, the dimension value of the index name is obtained according to the number of entries. As shown in the above steps, when the number of entries of the index name that meet the query condition is obtained, the number of categories of user data under the query condition, that is, the dimension value of the index name, can be determined.

[0052] In some embodiments, step S3000 may include but is not limited to steps S3100 to S3200: Step S3100, automatically generate corresponding target buckets for the index name according to the data type of the user data.

[0053] Step S3200, update the query conditions according to the target bucket.

[0054] In step S3100 of some embodiments, when the dimension value is greater than the preset dimension threshold, that is, the query result is too discrete, it is necessary to generate corresponding target buckets based on the data type of the user data to make the query results of the user data more aggregated. Through bucketing, large-scale data sets can be broken down into smaller parts, so that query operations can be performed on smaller data sets, thereby improving query efficiency. At the same time, bucketed data is easier to manage and process, especially when performing JOIN operations, which can reduce the amount of data to be processed and increase processing speed. In addition, bucketing makes sampling queries more efficient, and sampling can be performed in some buckets to obtain representative results.

[0055] It should be noted that bucketing technique is a commonly used technique in computer science and data processing, which is used to divide data into different buckets or intervals to process data more efficiently.

[0056] It is understandable that in data query, especially in query operations for big data, bucketing technology is usually used to process sequence data. Its main uses include: sequence length bucketing, word frequency bucketing and value range bucketing. Among them, sequence length bucketing is to divide the input sequence into different buckets according to its length. This is very useful for processing sequence data of different lengths, because medical data usually requires fixed length input. By placing samples with similar sequence lengths in the same bucket, padding operations can be reduced and the processing efficiency of big data can be improved. Word frequency bucketing is to divide words into different buckets according to their word frequency in the corpus. This is very useful for processing medical databases with large vocabulary, because some common words may appear frequently, while some uncommon words may appear less frequently. By placing words with similar word frequencies in the same bucket, sparsity can be reduced and the data processing efficiency and generalization ability of medical databases can be improved. Value range bucketing is to divide numerical data into different buckets according to its value range. This is very useful for processing continuous numerical features, because continuous numerical features need to be discretized during data processing and analysis so that medical databases can be better processed. By placing data with similar numerical ranges in the same bucket, the dimension and complexity of the data can be reduced, and the data processing speed and generalization ability of the medical database can be improved. Therefore, bucketing technology can help improve the data processing speed and generalization ability of the medical database, reduce the complexity and amount of data processing during the query process, and has a significant performance improvement effect, especially when processing large-scale data and complex models.

[0057] In step S3200 of some embodiments, after obtaining the target bucket, the query condition needs to be updated to avoid obtaining overly discrete query results due to the updated query condition.

[0058] In some embodiments, step S3100 may include, but is not limited to, steps S3110 to S3130: Step S3110, obtaining the numerical range of user data.

[0059] Step S3120: divide the user data according to the numerical range and the data type to obtain partitions of the user data, wherein the number of partitions is less than or equal to the dimension threshold.

[0060] Step S3130, distribute the user data that meets the requirements between partitions into corresponding buckets to obtain target buckets.

[0061] In step S3110 of some embodiments, in order to avoid the occurrence of discrete query conditions with equal values ​​in the above step S3100, it is necessary to obtain the numerical interval of the user data. An interval generally refers to a set of real numbers: if x and y are two numbers in the set, then any number between x and y also belongs to the set. When the maximum and minimum values ​​in the user data are found, the numerical interval of the user data can be accurately obtained. For example, when the user data is age, the maximum age of the user in the medical database is 100, and the minimum age of the user is 1, then the numerical interval of the user data can be determined to be [1,100], and this numerical interval contains the age range of all user data in the medical database.

[0062] In step S3120 of some embodiments, after obtaining the numerical interval, the user data is divided according to the data type to obtain the partition interval of the user data. That is, the division is performed on the numerical interval, and its working principle is consistent with the above-mentioned step S1300, which also ensures that there is no intersection between the numerical intervals and the numerical intervals, and adopts the principle of left closing and right opening. The difference is that the division operation of the user data is automatically performed here, and it is no longer divided into any number of numerical intervals. The principle of dividing the numerical interval here is to ensure that the number of partition intervals is less than or equal to the dimension threshold to avoid the query result from being too discrete. In specific applications, controlling the number of partition intervals through instructions belongs to the prior art and will not be repeated here.

[0063] It should be noted that the general bucket configuration is stored in a relational database, such as postgresql, while the user data is stored in an online analytical processing (OLAP) database. Among them, a relational database refers to a database that uses a relational model to organize data. It stores data in the form of rows and columns to facilitate user understanding. This series of rows and columns in a relational database is called a table, and a group of tables constitutes a database. Users retrieve data from the database through queries, and a query is an execution code used to limit certain areas in the database. The relational model can be simply understood as a two-dimensional table model, and a relational database is a data organization composed of two-dimensional tables and the relationships between them.

[0064] It should be noted that online analytical processing is a technology used to organize large business databases and support business intelligence. OLAP databases are divided into one or more cubes, each of which is organized and designed by the cube administrator to accommodate the way data is retrieved and analyzed so that it is easier to create and use the required PivotTable and PivotChart reports. OLAP is a database technology that has been optimized for querying and reporting rather than processing transactions. The source data for OLAP is a database that is usually stored in a data warehouse for online transaction processing. OLAP data is derived from this historical data and aggregated into a structure that allows complex analysis. OLAP data is also organized hierarchically and stored in cubes rather than in tables. It is a complex technology that uses a multidimensional structure to provide fast access to data for analysis. With this organization, a PivotTable or PivotChart report can easily display a high-level summary, such as the total sales for an entire region, and display details for sites where sales are particularly strong or weak. OLAP databases are designed to speed up the retrieval of data. Because the OLAP server calculates summary values, less data needs to be sent to Excel when creating or changing reports. This approach enables you to handle much larger volumes of source data than you would if you organized the data in a traditional database, where Excel retrieves all the individual records and then calculates summary values. An OLAP database contains two basic types of data: measures (numeric data), quantities and averages used to make informed business decisions, and dimensions, which are the categories used to organize these measures. OLAP databases use the same categories you are familiar with to analyze data, helping to organize data at multiple levels of detail.

[0065] In step S3130 of some embodiments, after obtaining the partitions of the user data, the user data that meets the partitions is allocated to the corresponding buckets to obtain the target buckets, and a bucket query statement is generated according to the bucket configuration. The specific process is consistent with the above step S1400 and will not be repeated here.

[0066] In some embodiments, step S3000 may include but is not limited to steps S3300 to S3400: Step S3300: When the dimension value is less than or equal to the dimension threshold, a discrete query statement is generated according to the query condition.

[0067] Step S3400: sending a discrete query statement to a database.

[0068] In step S3300 of some embodiments, when the dimension value is less than or equal to the dimension threshold, a query can be performed directly according to the discrete query conditions set by the user.

[0069] It should be noted that the most typical discrete query is the query for metadata. In the context of graph computing and graph databases, metadata refers to vertices or edges, because these two are the data with the smallest granularity. They are usually identified by unique IDs and do not need to be further subdivided. All operations that are only oriented to vertices or edges are typical discrete type operations. It should be pointed out that the definition of metadata is, on the one hand, the smallest granularity and indivisibility, and on the other hand, there is a unique matching ID to locate it. In graph databases, only vertices and edges meet these two points. However, the expression of edges is indeed more complicated, because in addition to the ID, the edges are also associated with the starting point and end point, direction, and other attributes. Some graph computing systems will cut edges, that is, the two vertices associated with an edge may be cut and stored in different shards, but this kind of edge cutting design system is relatively rare and has very low computational efficiency.

[0070] In step S3400 of some embodiments, after obtaining the discrete query statement, the discrete query statement needs to be sent to the database, and the database responds to the discrete query statement and returns the query result. The database query result is a data set retrieved from the database through an SQL query, which is usually displayed in a table format, including the fields specified in the query and the records that meet the query conditions.

[0071] It is understandable that generating discrete query statements according to query conditions and sending them to a database belongs to the prior art and will not be described in detail here.

[0072] In step S4000 of some embodiments, a bucket query statement is generated according to the query condition, and the bucket query statement is sent to the database. It should be noted that the database includes the target bucket indicated in the bucket query statement. After sending the bucket query statement to the query database using the SQL language, the database performs the following processing: receiving the bucket query statement; determining the user data to be queried based on the index name and query conditions indicated by the bucket query statement; aggregating the query information based on the target bucket indicated by the bucket query statement, performing data query in the target bucket, and obtaining the query result.

[0073] In step S5000 of some embodiments, after receiving the data query result returned by the database, it is necessary to perform code conversion on the data query result to obtain a conversion query result in Java language; the conversion query result is sent to a business system in Java language, where the business system issues a query request. Specifically, after obtaining the data query result, code conversion can be performed to convert it into a conversion query result in Java language, and this conversion process needs to involve the conversion of the data query result. After the conversion is completed, the conversion query result can be sent to a business system in Java language, and the business system can process and display the query results according to its own language characteristics and requirements, thereby improving interactivity and usability.

[0074] It should be noted that the embodiment of the present application also supports bucketing and aggregation statistics of multiple different index names. Each index name can be bucketed, and the buckets of multiple index names can be aggregated and counted together. For example, if the bucket of index name A has 3 intervals and the bucket of index name B has 3 intervals, the final analysis result should have 3*3=9 dimensional analysis statistics, and can also be grouped and counted together by bucketing and discrete dimensions. For example, if index name A is age, index name B is time, and index name C is gender, the specific statistics are as follows: Bucket + Bucket: Teenagers, daytime; Teenagers, nighttime; Middle-aged people, daytime; Middle-aged people, nighttime...etc.

[0075] Bucket + Discrete: Teenagers, male; Teenagers, female; Middle-aged, male; Middle-aged, female... etc.

[0076] In the embodiment of the present application, by obtaining query information, when the query condition is a discrete query, statistical processing is performed on the user data to obtain the dimension value of the index name; when the dimension value is greater than the preset dimension threshold, the query condition is bucketed and the query condition is updated; a bucketed query statement is generated according to the query condition, and the bucketed query statement is sent to the database; and the data query result returned by the database is received. By automatically generating bucketed query statements that meet the dimension threshold for different query conditions and their dimension values, the reference value of big data query results is improved, the situation where data statistics are too dispersed is avoided, and the accuracy and timeliness of data queries are improved.

[0077] In a possible implementation, the user data is medical data, such as personal health records, prescriptions, examination reports, and other data.

[0078] Information query has become a channel for users to quickly obtain the information they need in many scenarios. For example, in the medical field, the medical record information required by users can be queried from massive electronic medical records based on artificial intelligence models, which helps to provide users with medical record references.

[0079] First, when performing a query, select the query type, including discrete query or custom data bucketing. Discrete query means no bucketing, statistical analysis is performed according to the divergent dimension values, and one statistical record is recorded for each dimension value. If custom data bucketing is selected, proceed to the next step. Next, fill in the custom data bucketing configuration and interval distribution. In the pop-up configuration partition prompt box, the attribute value of the user data can be divided into any number of bucket intervals according to the actual query needs. There should be no intersection between buckets, and the principle of left-closed and right-open data intervals should be followed.

[0080] For example, when the user data is the user's age. Dynamically generate bucket SQL according to the bucket rule configuration, dynamically bucket the user data according to the bucket rule during query, and generate SQL for processing the user data. Specifically, according to the parameters of the set bucket rule, write a dimension SQL dynamic generation class, which parses the bucket rule parameters and generates a string of SQL returns, and can generate corresponding SQL for the data type of the dimension field, such as numeric type and time type. For example, according to the age in the medical data, the interval bucket is divided, 0-18 years old (not including 18 years old) belongs to minors, 18-45 years old belongs to young people, and over 45 years old (not including 45 years old) belongs to middle-aged and elderly people. This can dynamically change according to the actual application query scenario, and generate the following after reading the rules: select case when a.age>0 and a.age<18 then 'minor' when a.age>=18 and a.age<=45 then 'young person' when a.age>45 then 'middle-aged and elderly' else 'other' end as age group by age Next, the bucket processing SQL for user data obtained in the previous step can be spliced ​​with the original business query SQL, such as filtering conditions, time range, etc., to form a complete query SQL; if it is a discrete query, the query is directly grouped according to the field, such as group by age; if it is a bucket query, the query SQL is dynamically spliced ​​according to the custom partition, and finally the alias field after group by processing is used.

[0081] The present application also provides a big data bucket query device, which can implement the above big data bucket query method, and the device includes: An acquisition module, used to acquire query information, wherein the query information includes an index name, user data and query conditions; The statistics module is used to perform statistical processing on user data when the query condition is a discrete query to obtain the dimension value of the index name; An update module is used to bucket the query conditions and update the query conditions when the dimension value is greater than a preset dimension threshold; A generation module is used to generate bucket query statements according to query conditions and send the bucket query statements to the database; The receiving module is used to receive the data query results returned by the database.

[0082] The specific implementation of the big data bucket query device is basically the same as the specific implementation of the above-mentioned big data bucket query method, and will not be repeated here.

[0083] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned big data bucket query method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0084] The hardware structure of the electronic device of another embodiment includes: The processor may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application; The memory can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory can store an operating system and other applications. When the technical solution provided in the embodiments of this specification is implemented by software or firmware, the relevant program code is stored in the memory, and the processor calls and executes the big data bucket query method of the embodiments of this application; Input / output interface, used to realize information input and output; Communication interface, used to realize communication interaction between this device and other devices. Communication can be realized through wired methods (such as USB, network cable, etc.) or wireless methods (such as mobile network, WIFI, Bluetooth, etc.); The bus, which transfers information between the various components of the device (such as the processor, memory, input / output interfaces, and communication interfaces); The processor, memory, input / output interface and communication interface are connected to each other through a bus inside the device.

[0085] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the above-mentioned big data bucket query method is implemented.

[0086] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0087] The big data bucket query method, big data bucket query device, electronic device and storage medium provided in the embodiment of the present application obtain query information, and when the query condition is a discrete query, perform statistical processing on the user data to obtain the dimension value of the index name; when the dimension value is greater than the preset dimension threshold, perform bucket processing on the query condition and update the query condition; generate a bucket query statement according to the query condition, send the bucket query statement to the database; receive the data query result returned by the database. By automatically generating a bucket query statement that meets the dimension threshold for different query conditions and their dimension values, the reference value of the big data query result is improved, the situation where the data statistics are too dispersed is avoided, and the accuracy and timeliness of the data query are improved.

[0088] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0089] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0090] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0091] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0092] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0093] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0094] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0095] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0096] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0097] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0098] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A big data bucket query method, characterized in that: The method comprises: Acquire query information, wherein the query information includes an index name, user data, and query conditions; When the query condition is a discrete query, statistical processing is performed on the user data to obtain a dimension value of the index name; When the dimension value is greater than a preset dimension threshold, the query condition is bucketed and the query condition is updated; Generate a bucket query statement according to the query condition, and send the bucket query statement to the database; Receive the data query result returned by the database.

2. The big data bucket query method according to claim 1, characterized in that: The obtaining of query information includes: Acquire the user data and obtain the data type of the user data; According to the data type, the query condition corresponding to the index name is obtained.

3. The big data bucket query method according to claim 2 is characterized in that: After obtaining the query information, the method further includes: When the query condition is a bucket query, the bucket configuration of the index name is obtained according to the query condition; Generate a bucket query statement according to the bucket configuration.

4. The big data bucket query method according to claim 1, characterized in that: The performing statistical processing on the user data to obtain the dimension value of the index name includes: According to the index name and the query condition, obtaining the number of entries whose index name meets the query condition; According to the number of entries, the dimension value of the index name is obtained.

5. The big data bucket query method according to claim 2, characterized in that: The bucketing of the query condition and updating the query condition includes: According to the data type of the user data, automatically generate a corresponding target bucket for the index name; Update the query condition according to the target bucket.

6. The big data bucket query method according to claim 5, characterized in that: The automatically generating a corresponding target bucket for the index name according to the data type of the user data includes: Obtaining a numerical range of the user data; According to the numerical interval and the data type, a partition operation is performed on the user data to obtain partitions of the user data, wherein the number of the partitions is less than or equal to the dimension threshold; The user data that meets the requirements between the partitions is allocated to corresponding buckets to obtain the target buckets.

7. The big data bucket query method according to claim 1, characterized in that: The method further comprises: When the dimension value is less than or equal to the dimension threshold, a discrete query statement is generated according to the query condition; Sending the discrete query statement to the database.

8. A big data bucket query device, characterized in that: The device comprises: An acquisition module, used to acquire query information, wherein the query information includes an index name, user data and query conditions; A statistical module, used for performing statistical processing on the user data to obtain the dimension value of the index name when the query condition is a discrete query; An updating module, configured to perform bucket processing on the query condition and update the query condition when the dimension value is greater than a preset dimension threshold; A generation module, used to generate a bucket query statement according to the query condition, and send the bucket query statement to the database; The receiving module is used to receive the data query result returned by the database.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the big data bucket query method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the big data bucket query method described in any one of claims 1 to 7 is implemented.

Images