Data query method and system, computer equipment and readable storage medium

CN118939610APending Publication Date: 2024-11-12SHENZHEN YISHIHUOLALA TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411031909.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-30
Publication Date
2024-11-12

Smart Images

  • Figure CN118939610A_ABST
    Figure CN118939610A_ABST
Patent Text Reader

Abstract

The invention provides a data query method and system, computer equipment and a readable storage medium, and the method comprises the steps: monitoring a data change event of a database through an application service, the database storing model data; analyzing the data change event through the application service, and converting all or local data of the model data into document data of a search engine when data change occurs in the model data; and finally, in response to the data query request, performing data query from the search engine. According to the method, all or partial data of the model data are converted into the document data of the search engine, so that when a user performs data query through a service system in a client, the user does not need to perform complex list query operation from a database, but can directly perform data query from the search engine. Therefore, by utilizing the dynamic characteristics of the search engine, not only is the data query operation simplified, but also the data query efficiency and the data query flexibility are remarkably improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a data query method, system, computer equipment and readable storage medium. Background Art

[0002] The EAV (Entity–Attribute–Value) model is a model for dynamically adding or removing attributes from a data model. Its flexibility means that when adding new attributes, there is no need to redesign the table structure. When expanding the database, attributes can be added vertically without changing the data structure. However, as the amount of data increases, the efficiency of data retrieval becomes very low; and because the data is relatively scattered, querying the records of a complete entity requires multiple table JOIN queries. More importantly, when the EAV model is applied to large amounts of data, the same set of EAV modeled data descriptions requires matrix pivoting (row-column conversion), which is error-prone and CPU (Central Processing Unit, CPU) intensive. Therefore, in the EAV model, how to improve the efficiency of database queries is a major problem that needs to be solved urgently. Summary of the invention

[0003] In view of the above-mentioned shortcomings of the prior art, an object of the present invention is to provide a data query method, system, computer device and readable storage medium to solve the technical problems existing in the prior art.

[0004] To achieve the above object and other related objects, the present invention provides a data query method, comprising the following steps:

[0005] Monitoring data change events of a database, where the database stores model data;

[0006] Parsing the data change event, and converting all or part of the model data into document data of the search engine when the model data changes;

[0007] In response to a data query request, a data query is performed from the search engine; wherein the data query request is used to query the model data.

[0008] In one embodiment of the present invention, if the model data includes entity attribute value model data, when data changes occur in the entity attribute value model data, the process of converting all or part of the entity attribute value model data into document data of the search engine includes:

[0009] When data changes occur in the value table of the entity attribute value model data, the column fields of the entity table are obtained through the foreign key entity number of the value table, and the attribute data of the attribute table are obtained through the foreign key attribute number of the value table;

[0010] Associating the column fields of the entity table with the attribute data of the attribute table as search engine graph fields;

[0011] The data in the value table are parsed item by item, and the row data in the value table are converted into column data of the search engine, and the document data of the search engine is generated by combining the search engine graph field and the column data of the search engine, and the document data is stored in the search engine.

[0012] In one embodiment of the present invention, the process of monitoring the data change event of the database includes:

[0013] Obtaining a configuration file of the database;

[0014] Adjust the log parameters in the configuration file to be turned on, and specify the log file name and storage location; wherein the log parameters include binary log parameters;

[0015] Using a command line tool determined in advance or in real time, reading a log file according to a specified log file name and storage location, parsing the content of the log file, and identifying a data event type of the database;

[0016] If the data event type includes insert, update or delete, determining that there is a data change event in the database;

[0017] If the data event type does not include insert, update and delete, continue to parse the log file content to identify the data event type of the database.

[0018] In an embodiment of the present invention, when converting the row data in the value table into column data of the search engine, the method further includes: automatically expanding the fields based on the dynamic characteristics of the search engine.

[0019] In an embodiment of the present invention, the method further includes: monitoring data change events of a database through an application service, and parsing the data change events through the application service.

[0020] In one embodiment of the present invention, if the model data includes entity attribute value model data, before monitoring the data change event of the database, the method further includes:

[0021] Obtaining basic information of a preset product and storing the basic information in an entity table of an entity attribute value model; and,

[0022] Obtaining the attribute name of the preset product, and storing the attribute name in the attribute table of the entity attribute value model; and,

[0023] Obtaining attribute values ​​of the preset product, and storing the attribute values ​​in a value table of an entity attribute value model;

[0024] The basic information includes: a unique identifier of the product and a product name;

[0025] The attribute name includes: a unique identifier of the attribute and an attribute name;

[0026] The attribute value includes: a unique identifier of the value, a foreign key referencing an entity table, a foreign key referencing an attribute table, and an actual value of the attribute.

[0027] The present invention also provides a data query system, the system comprising:

[0028] A monitoring module, used for monitoring data change events in a database, wherein the database stores model data;

[0029] A conversion module, used to parse the data change event and convert all or part of the model data into document data of the search engine when the model data changes;

[0030] A query module is used to perform data query from the search engine in response to a data query request; wherein the data query request is used to query the model data.

[0031] In one embodiment of the present invention, if the model data includes entity attribute value model data, then when data changes occur in the entity attribute value model data, the process of the conversion module converting all or part of the entity attribute value model data into document data of the search engine includes:

[0032] When data changes occur in the value table of the entity attribute value model data, the column fields of the entity table are obtained through the foreign key entity number of the value table, and the attribute data of the attribute table are obtained through the foreign key attribute number of the value table;

[0033] Associating the column fields of the entity table with the attribute data of the attribute table as search engine graph fields;

[0034] The data in the value table are parsed item by item, and the row data in the value table are converted into column data of the search engine, and the document data of the search engine is generated by combining the search engine graph field and the column data of the search engine, and the document data is stored in the search engine.

[0035] The present invention also provides a computer device, comprising a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of any one of the data query methods described above.

[0036] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the data query methods described above are implemented.

[0037] As described above, the present invention provides a data query method, system, computer device and readable storage medium, which have the following beneficial effects: the present invention monitors the data change events of the database through the application service, and the database stores the model data; and parses the data change events through the application service, and when the model data changes, converts all or part of the model data into the document data of the search engine; finally, by responding to the data query request, the data query is performed from the search engine; wherein the data query request is used to query the model data. It can be seen that the present invention converts all or part of the model data into the document data of the search engine, and when the user performs data query through the business system in the client, there is no need to perform complex joint table query operations from the database, but data query can be performed directly from the search engine. Therefore, by utilizing the dynamic characteristics of the search engine, the present invention not only simplifies the data query operation, but also significantly improves the data query efficiency and data query flexibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A schematic diagram of an exemplary system architecture for applying the technical solutions in one or more embodiments of the present invention;

[0039] Figure 2 A schematic diagram of a flow chart of a data query method provided by an embodiment of the present invention;

[0040] Figure 3 A schematic diagram of a flow chart of a data query method provided in another embodiment of the present invention;

[0041] Figure 4 A schematic diagram of the hardware structure of a data query system provided by an embodiment of the present invention;

[0042] Figure 5 The figure is a schematic diagram of the hardware structure of a computer device suitable for implementing one or more embodiments of the present invention. DETAILED DESCRIPTION

[0043] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0044] It should be noted that the illustrations provided in this embodiment are only used to illustrate the basic concept of the present invention in a schematic manner. Therefore, the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0045] Figure 1 FIG. 1 is a schematic diagram showing an exemplary system architecture to which the technical solution in one or more embodiments of the present invention can be applied. Figure 1 As shown, the system architecture 100 may include a terminal device 110, a network 120, and a server 130. The terminal device 110 may include various electronic devices such as a smart phone, a tablet computer, a laptop computer, and a desktop computer. The server 130 may be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The network 120 may be a communication medium of various connection types that can provide a communication link between the terminal device 110 and the server 130, such as a wired communication link or a wireless communication link.

[0046] According to the implementation requirements, the system architecture in the embodiment of the present invention may have any number of terminal devices, networks and servers. For example, the server 130 may be a server group composed of multiple server devices. In addition, the technical solution provided in the embodiment of the present invention may be applied to the terminal device 110, or to the server 130, or may be implemented by the terminal device 110 and the server 130 together, and the present invention does not make any special limitation on this.

[0047] In one embodiment of the present invention, the terminal device 110 or server 130 of the present invention can monitor data change events of a database through an application service, and the database stores model data; and parse the data change events through the application service, and when data changes occur in the model data, convert all or part of the model data into document data of the search engine; finally, by responding to a data query request, perform a data query from the search engine; wherein the data query request is used to query the model data. By executing the data query method using the terminal device 110 or the server 130, all or part of the model data can be converted into document data of the search engine, and when the user performs a data query through the business system in the client, there is no need to perform a complex joint table query operation from the database, but data query can be performed directly from the search engine. Therefore, by utilizing the dynamic characteristics of the search engine, the present invention not only simplifies the data query operation, but also significantly improves the data query efficiency and data query flexibility.

[0048] The above section introduces the contents of an exemplary system architecture that applies the technical solution of the present invention. Next, the data query method of the present invention will be introduced.

[0049] Figure 2 FIG. 1 is a flow chart of a data query method provided by an embodiment of the present invention. Specifically, in an exemplary embodiment, Figure 2 As shown, this embodiment provides a data query method, which includes the following steps:

[0050] S210, monitoring data change events of a database, wherein the database stores model data. As an example, in this embodiment or other embodiments, the database includes but is not limited to a relational database management system MySQL, and the model data includes but is not limited to entity attribute value model data or EAV model data.

[0051] S220, parsing the data change event, and converting all or part of the model data into document data of the search engine when the model data changes. As an example, in this embodiment or other embodiments, the search engine includes but is not limited to Elasticsearch.

[0052] S230, in response to the data query request, perform a data query from a search engine; wherein the data query request is used to query the model data.

[0053] It can be seen that, by converting all or part of the model data into document data of the search engine, when the user performs data query through the business system in the client, there is no need to perform complex joint table query operations in the database, but data query can be performed directly from the search engine. Therefore, by utilizing the dynamic characteristics of the search engine, this embodiment not only simplifies the data query operation, but also significantly improves the data query efficiency and data query flexibility.

[0054] According to the above records, in an exemplary embodiment, if the model data includes entity attribute value model data, when the entity attribute value model data has data changes, the process of converting all or part of the entity attribute value model data into document data of the search engine includes: when the value table of the entity attribute value model data has data changes, the column field of the entity table is obtained through the foreign key entity number of the value table, and the attribute data of the attribute table is obtained through the foreign key attribute number of the value table; the column field of the entity table and the attribute data of the attribute table are associated as search engine graph fields; the data of the value table is parsed one by one, and the row data in the value table is converted into the column data of the search engine, and the document data of the search engine is generated by combining the search engine graph field and the column data of the search engine, and the document data is stored in the search engine. As an example, the search engine in this embodiment includes but is not limited to Elasticsearch. It can be seen that this embodiment not only retains the flexible expansion capability of the database EAV model by converting the row data of the EAV model into the column data of Elasticsearch for storage, but also the row-to-column storage method greatly reduces the number of data rows in Elasticsearch, and the reduction in the number of data rows will inevitably lead to an improvement in query efficiency. Assuming that the total amount of data in the Value table of the EAV model is N, and the number of attributes involved is M, due to the row-to-column operation, the number of data rows actually stored in Elasticsearch is about N / M. Therefore, when users query data through the business system in the client, they no longer need to perform complex joint table query operations in the database, but can directly query data from Elasticsearch. By utilizing the dynamic characteristics of Elasticsearch, not only does it simplify data query operations, but it also significantly improves data query efficiency and flexibility.

[0055] According to the above records, in an exemplary embodiment, the process of monitoring data change events of a database includes: obtaining a configuration file of the database; adjusting the log parameters in the configuration file to be turned on, and specifying the log file name and storage location; wherein the log parameters include binary log parameters; using a command line tool determined in advance or in real time, reading the log file according to the specified log file name and storage location, and parsing the content of the log file to identify the data event type of the database; if the data event type includes insert, update or delete, determining that there is a data change event in the database; if the data event type does not include insert, update and delete, continuing to parse the log file content to identify the data event type of the database.

[0056] As an example, in this embodiment, if the database in this embodiment is MySQL, this embodiment implements monitoring of MySQL data change events by monitoring binlog. Specifically, obtain the configuration file of the MySQL database, modify the configuration file of MySQL, ensure that the log_bin parameter is set to ON, and specify the name and storage location of the binlog file. Configure binlog format: There are three formats of binlog: STATEMENT, ROW and MIXED. The ROW format records the changes of each row of data, and the monitoring of data changes is more accurate. Restart MySQL service: After modifying the configuration file, you need to restart the MySQL service for the configuration to take effect. Select and configure a binlog monitoring tool, such as mysqlbinlog command line tool, Canal, Maxwell, Debezium, etc.; and configure MySQL connection information, binlog monitoring location, data parsing rules, etc. according to the requirements of the selected tool. The monitoring tool captures MySQL binlog events and parses them into recognizable data formats (such as JSON, messages in message queues, etc.). The parsing process includes identifying event types (such as inserting INSERT, updating UPDATE, deleting DELETE), extracting change data, etc. If the data event type includes INSERT, UPDATE, or DELETE, it is determined that there is a data change event in the database.

[0057] According to the above records, in an exemplary embodiment, when the row data in the value table is converted into the column data of the search engine, the embodiment may also include: automatically expanding the field based on the dynamic characteristics of the search engine. As an example, in this embodiment or other embodiments, if the search engine is Elasticsearch, the dynamic characteristics (dynamic) of Elasticsearch can be used to dynamically expand the index field. At the same time, by utilizing this dynamic characteristic, the EAV model row data of the database can be converted into the column data of Elasticsearch for storage, and by transferring the data query from the database to Elasticsearch, the data query efficiency can be improved.

[0058] According to the above records, in an exemplary embodiment, this embodiment may also include: monitoring the data change event of the database through the application service, and parsing the data change event through the application service. As an example, in this embodiment or other embodiments, the application service includes but is not limited to services written in languages ​​such as Java, Python, and C++.

[0059] According to the above records, in an exemplary embodiment, if the model data includes entity attribute value model data, before monitoring the data change event of the database, this embodiment may also include: obtaining the basic information of the preset product, and storing the basic information in the entity table of the entity attribute value model; wherein the basic information includes: the unique identifier of the product, the product name. And, obtaining the attribute name of the preset product, and storing the attribute name in the attribute table of the entity attribute value model; wherein the attribute name includes: the unique identifier of the attribute, the attribute name. And, obtaining the attribute value of the preset product, and storing the attribute value in the value table of the entity attribute value model; wherein the attribute value includes: the unique identifier of the value, the foreign key referencing the entity table, the foreign key referencing the attribute table, and the actual value of the attribute.

[0060] In another exemplary embodiment of the present invention, Figure 3 As shown, this embodiment also provides a data query method, comprising the following steps:

[0061] Step 1: Monitor the data changes of the database in real time through the application service. As an example, for example, in the database MySQL, this embodiment is implemented by monitoring binlog. Step 2: After the application service receives the data change event of the Value table of the EAV model, the application service will monitor the corresponding event and start parsing the data change content;

[0062] Step 3: After the application service parses the changed data in the Value table, it needs to convert the data in the EAV model into document data in Elasticsearch for storage. This includes the following four steps:

[0063] a) Get the column field of the Entity table in the EAV model through the foreign key entity_id of the Value table in the EAV model, and use it as the field of Elasticsearch Mapping.

[0064] b) Get the attribute data of the Attribute table in the EAV model through the foreign key attribute_id of the Value table in the EAV model, and use it as the field of Elasticsearch Mapping;

[0065] c) Parse the data in the Value table in the EAV model one by one, convert the row data in the Value table in the EAV model into column data of Elasticsearch, and thus obtain document data;

[0066] d) Update the processed document data to Elasticsearch for storage.

[0067] Specifically, this embodiment can insert the following EAV model data into the database:

[0068] Table 1 Product table Goods (corresponding to Entity in EAV)

[0069] Id Name 1 cell phone 2 computer

[0070] Table 2 Attribute table Goods_attribute (corresponding to Attribute in EAV)

[0071] Id Goods_id attribute 1 1 color 2 2 cpu 3 2 ram

[0072] Table 3 Value table Goods_value (corresponding to Value in EAV)

[0073] Id Goods_id Attribute_id value 1 1 1 red 2 2 2 GTX3090 3 2 3 8G

[0074] As shown in Tables 1 to 3 above, for operation step a, the name of the Goods table can be found through the Goods_id of the Goods_value table, and it can be regarded as a field of Elasticsearch mapping, corresponding to Goods_name of Elasticsearch. For operation step b, the attribute of the Goods_attribute table can be found through the Attribute_id of the Goods_value table, and it can be converted into Elasticsearch mapping, corresponding to color, cpu, ram, etc. of Elasticsearch. For operation step c, the actual row data of the Goods_value table can be converted into document data of Elasticsearch mapping. For example, the first row of Goods_name is entity = mobile phone, color is attribute = red, cpu and ram are also attributes, but the data in the value table are empty.

[0075] The converted document data is stored in Elasticsearch to obtain the final row-to-column conversion effect. Specifically, after the application service parses the EAV model data, the data stored in Elasticsearch is shown in Table 4 below.

[0076] Table 4 Data in Elasticsearch

[0077] Id Goods_id Goods_name color cpu ram 1 1 cell phone red 2 2 computer GTX3090 8G

[0078] Step 4: When the user queries the data set of the EAV model through the business system, the data query is directly performed from Elasticsearch in response to the data query request generated by the user in real time, and no complex joint table query is performed from the database. Therefore, this embodiment not only simplifies the data query operation, but also significantly improves the data query efficiency and data query flexibility.

[0079] In summary, the present invention provides a data query method, which monitors data change events of a database through an application service, and the database stores model data; and parses data change events through an application service, and when data changes occur in the model data, converts all or part of the model data into document data of a search engine; finally, in response to a data query request, performs a data query from a search engine; wherein the data query request is used to query the model data. It can be seen that, by converting all or part of the model data into document data of a search engine, when a user performs a data query through a business system in a client, there is no need to perform a complex joint table query operation from the database, but data can be directly queried from the search engine. Therefore, by utilizing the dynamic characteristics of the search engine, the method not only simplifies the data query operation, but also significantly improves the data query efficiency and data query flexibility. In addition, although the existing technology uses a wide data table solution to perform data query, it does not support flexible field expansion. The present method dynamically expands index fields through the dynamic characteristics of Elasticsearch. By utilizing this dynamic characteristic, the EAV model row data of the database can be converted into Elasticsearch column data for storage, and the query can be transferred from the database to Elasticsearch, thereby improving data query efficiency.

[0080] In another exemplary embodiment of the present invention, Figure 4 As shown, this embodiment also provides a data query system, including:

[0081] The monitoring module 410 is used to monitor data change events of a database, where the database stores model data. As an example, in this embodiment or other embodiments, the database includes but is not limited to a relational database management system MySQL, and the model data includes but is not limited to entity attribute value model data or EAV model data.

[0082] The conversion module 420 is used to parse the data change event and convert all or part of the model data into document data of the search engine when the model data changes. As an example, in this embodiment or other embodiments, the search engine includes but is not limited to Elasticsearch.

[0083] The query module 430 is used to perform data query from a search engine in response to a data query request, wherein the data query request is used to query model data.

[0084] It can be seen that, by converting all or part of the model data into document data of the search engine, when the user performs data query through the business system in the client, there is no need to perform complex joint table query operations in the database, but data query can be performed directly from the search engine. Therefore, by utilizing the dynamic characteristics of the search engine, this embodiment not only simplifies the data query operation, but also significantly improves the data query efficiency and data query flexibility.

[0085] According to the above records, in an exemplary embodiment, if the model data includes entity attribute value model data, the process of converting all or part of the entity attribute value model data into the document data of the search engine when the entity attribute value model data changes includes: when the value table of the entity attribute value model data changes, the column field of the entity table is obtained through the foreign key entity number of the value table, and the attribute data of the attribute table is obtained through the foreign key attribute number of the value table; the column field of the entity table and the attribute data of the attribute table are associated as search engine graph fields; the data of the value table is parsed one by one, and the row data in the value table is converted into the column data of the search engine, and the document data of the search engine is generated by combining the search engine graph field and the column data of the search engine, and the document data is stored in the search engine. As an example, the search engine in this embodiment includes but is not limited to Elasticsearch. It can be seen that this embodiment not only retains the flexible expansion capability of the database EAV model by converting the row data of the EAV model into the column data of Elasticsearch for storage, but also greatly reduces the number of data rows in Elasticsearch by the row-to-column storage method, and the reduction in the number of data rows will inevitably lead to an improvement in query efficiency. Assuming that the total amount of data in the Value table of the EAV model is N, and the number of attributes involved is M, due to the row-to-column operation, the number of data rows actually stored in Elasticsearch is about N / M. Therefore, when users query data through the business system in the client, they no longer need to perform complex joint table query operations in the database, but can directly query data from Elasticsearch. By utilizing the dynamic characteristics of Elasticsearch, not only does it simplify data query operations, but it also significantly improves data query efficiency and flexibility.

[0086] According to the above records, in an exemplary embodiment, the process of monitoring data change events of a database includes: obtaining a configuration file of the database; adjusting the log parameters in the configuration file to be turned on, and specifying the log file name and storage location; wherein the log parameters include binary log parameters; using a command line tool determined in advance or in real time, reading the log file according to the specified log file name and storage location, and parsing the content of the log file to identify the data event type of the database; if the data event type includes insert, update or delete, determining that there is a data change event in the database; if the data event type does not include insert, update and delete, continuing to parse the log file content to identify the data event type of the database.

[0087] As an example, in this embodiment, if the database in this embodiment is MySQL, this embodiment implements monitoring of MySQL data change events by monitoring binlog. Specifically, obtain the configuration file of the MySQL database, modify the configuration file of MySQL, ensure that the log_bin parameter is set to ON, and specify the name and storage location of the binlog file. Configure binlog format: There are three formats of binlog: STATEMENT, ROW and MIXED. The ROW format records the changes of each row of data, and the monitoring of data changes is more accurate. Restart MySQL service: After modifying the configuration file, you need to restart the MySQL service for the configuration to take effect. Select and configure a binlog monitoring tool, such as mysqlbinlog command line tool, Canal, Maxwell, Debezium, etc.; and configure MySQL connection information, binlog monitoring location, data parsing rules, etc. according to the requirements of the selected tool. The monitoring tool captures MySQL binlog events and parses them into recognizable data formats (such as JSON, messages in message queues, etc.). The parsing process includes identifying event types (such as inserting INSERT, updating UPDATE, deleting DELETE), extracting change data, etc. If the data event type includes INSERT, UPDATE, or DELETE, it is determined that there is a data change event in the database.

[0088] According to the above records, in an exemplary embodiment, when the row data in the value table is converted into the column data of the search engine, the embodiment may also include: automatically expanding the field based on the dynamic characteristics of the search engine. As an example, in this embodiment or other embodiments, if the search engine is Elasticsearch, the dynamic characteristics (dynamic) of Elasticsearch can be used to dynamically expand the index field. At the same time, by utilizing this dynamic characteristic, the EAV model row data of the database can be converted into the column data of Elasticsearch for storage, and by transferring the data query from the database to Elasticsearch, the data query efficiency can be improved.

[0089] According to the above records, in an exemplary embodiment, this embodiment may also include: monitoring the data change event of the database through the application service, and parsing the data change event through the application service. As an example, in this embodiment or other embodiments, the application service includes but is not limited to services written in languages ​​such as Java, Python, and C++.

[0090] According to the above records, in an exemplary embodiment, if the model data includes entity attribute value model data, before monitoring the data change event of the database, this embodiment may also include: obtaining the basic information of the preset product, and storing the basic information in the entity table of the entity attribute value model; wherein the basic information includes: the unique identifier of the product, the product name. And, obtaining the attribute name of the preset product, and storing the attribute name in the attribute table of the entity attribute value model; wherein the attribute name includes: the unique identifier of the attribute, the attribute name. And, obtaining the attribute value of the preset product, and storing the attribute value in the value table of the entity attribute value model; wherein the attribute value includes: the unique identifier of the value, the foreign key referencing the entity table, the foreign key referencing the attribute table, and the actual value of the attribute.

[0091] In summary, the present invention provides a data query system, which monitors data change events of a database through an application service, and the database stores model data; and parses data change events through the application service, and when data changes occur in the model data, converts all or part of the model data into document data of the search engine; finally, by responding to a data query request, a data query is performed from the search engine; wherein the data query request is used to query the model data. It can be seen that the system converts all or part of the model data into document data of the search engine, and when the user performs data query through the business system in the client, there is no need to perform complex joint table query operations from the database, but data query can be performed directly from the search engine. Therefore, by utilizing the dynamic characteristics of the search engine, the system not only simplifies the data query operation, but also significantly improves the data query efficiency and data query flexibility. In addition, although the existing technology uses a wide data table solution to perform data query, it does not support flexible field expansion. The system dynamically expands the index field through the dynamic characteristics of Elasticsearch. By utilizing this dynamic characteristic, the EAV model row data of the database can be converted into Elasticsearch column data for storage, and the query can be transferred from the database to Elasticsearch, thereby improving data query efficiency.

[0092] It should be noted that the data query system provided in the above embodiment and the data query method provided in the above embodiment belong to the same concept, wherein the specific manner in which the data query method performs the operation has been described in detail in the above method embodiment, and will not be repeated here. In actual application, the data query system provided in the above embodiment can allocate the above functions to different functional modules as needed, that is, divide the internal structure of the data query system into different functional modules, and then implement all or part of the functions of the corresponding functional modules through the data query method described in the above embodiment, and this is not specifically limited here.

[0093] The embodiment of the present invention further provides a computer device, which may include a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program so that the computer device executes Figure 2 and / or Figure 3 The steps of the data query method. Figure 5 FIG. 1 is a schematic diagram showing a structure of a computer device 1000. Figure 5 As shown, the computer device 1000 includes: a processor 1010 , a memory 1020 , a power supply 1030 , a display unit 1040 , and an input unit 1060 .

[0094] The processor 1010 is the control center of the computer device 1000. It uses various interfaces and lines to connect various components, and executes various functions of the computer device 1000 by running or executing computer programs / instructions stored in the memory 1020, thereby monitoring the computer device 1000 as a whole. In the embodiment of the present invention, when the processor 1010 calls the computer program stored in the memory 1020, the following is executed: Figure 2 and / or Figure 3 The steps of the data query method. Optionally, the processor 1010 may include one or more processing units; preferably, the processor 1010 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application, etc., and the modem processor mainly processes wireless communication. In some embodiments, the processor and the memory may be implemented on a single chip, and in some embodiments, they may also be implemented separately on separate chips.

[0095] The memory 1020 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, various applications, etc.; the data storage area may store instruction data, etc. created according to the use of the computer device 1000. In addition, the memory 1020 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices, etc.

[0096] The computer device 1000 also includes a power source 1030 (such as a battery) for supplying power to various components. The power source can be logically connected to the processor 1010 through a power management system, so that the power management system can manage functions such as charging, discharging, and power consumption.

[0097] The display unit 1040 may be used to display information input by the user or information provided to the user and various menus of the computer device 1000, and in the embodiment of the present invention, is mainly used to display the display interface of each application in the computer device 1000 and objects such as text and pictures displayed in the display interface. The display unit 1040 may include a display panel 1050. The display panel 1050 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc.

[0098] The input unit 1060 may be used to receive information such as numbers or characters input by the user. The input unit 1060 may include a touch panel 1070 and other input devices 1080. The touch panel 1070, also known as a touch screen, may collect touch operations of the user on or near it (such as operations of the user using a finger, a stylus, or any other suitable object or accessory on or near the touch panel 1070).

[0099] Specifically, the touch panel 1070 can detect the user's touch operation, detect the signal brought by the touch operation, convert these signals into touch point coordinates, send them to the processor 1010, and receive and execute the command sent by the processor 1010. In addition, the touch panel 1070 can be implemented in various types such as resistive, capacitive, infrared and surface acoustic wave. Other input devices 1080 can include but are not limited to one or more of a physical keyboard, function keys (such as volume control keys, power on / off keys, etc.), a trackball, a mouse, a joystick, etc.

[0100] Of course, the touch panel 1070 can cover the display panel 1050. When the touch panel 1070 detects a touch operation on or near it, it is transmitted to the processor 1010 to determine the type of touch event, and then the processor 1010 provides a corresponding visual output on the display panel 1050 according to the type of touch event. Figure 5 In the embodiment, the touch panel 1070 and the display panel 1050 are two independent components to implement the input and output functions of the computer device 1000, but in some embodiments, the touch panel 1070 and the display panel 1050 can be integrated to implement the input and output functions of the computer device 1000.

[0101] The computer device 1000 may further include one or more sensors, such as a pressure sensor, a gravity acceleration sensor, a proximity light sensor, etc. Of course, according to the needs of specific applications, the computer device 1000 may also include other components such as a camera.

[0102] The embodiment of the present invention further provides a computer-readable storage medium, in which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the above-mentioned device can perform the present invention as follows: Figure 2 and / or Figure 3 The steps of the data query method.

[0103] It can be understood by those skilled in the art that Figure 5This is only an example of a computer device and does not constitute a limitation on the device. The device may include more or fewer components than shown in the figure, or combine certain components, or different components. For the convenience of description, the above parts are divided into modules (or units) according to their functions and described separately. Of course, when implementing the present invention, the functions of each module (or unit) can be implemented in the same or multiple software or hardware.

[0104] Those skilled in the art should understand that the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The present invention is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be applied to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0105] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A data query method, characterized in that: The method comprises the following steps: Monitoring data change events of a database, where the database stores model data; Parsing the data change event, and converting all or part of the model data into document data of the search engine when the model data changes; In response to a data query request, a data query is performed from the search engine; wherein the data query request is used to query the model data.

2. The data query method according to claim 1, characterized in that: If the model data includes entity attribute value model data, when data changes occur in the entity attribute value model data, the process of converting all or part of the entity attribute value model data into document data of the search engine includes: When data changes occur in the value table of the entity attribute value model data, the column fields of the entity table are obtained through the foreign key entity number of the value table, and the attribute data of the attribute table are obtained through the foreign key attribute number of the value table; Associating the column fields of the entity table with the attribute data of the attribute table as search engine graph fields; The data in the value table are parsed item by item, and the row data in the value table are converted into column data of the search engine, and the document data of the search engine is generated by combining the search engine graph field and the column data of the search engine, and the document data is stored in the search engine.

3. The data query method according to claim 1 or 2, characterized in that: The process of monitoring database data change events includes: Obtaining a configuration file of the database; Adjust the log parameters in the configuration file to be turned on, and specify the log file name and storage location; wherein the log parameters include binary log parameters; Using a command line tool determined in advance or in real time, reading a log file according to a specified log file name and storage location, parsing the content of the log file, and identifying a data event type of the database; If the data event type includes insert, update or delete, determining that there is a data change event in the database; If the data event type does not include insert, update and delete, continue to parse the log file content to identify the data event type of the database.

4. The data query method according to claim 2, characterized in that: When converting the row data in the value table into column data of the search engine, the method further comprises: automatically expanding the fields based on the dynamic characteristics of the search engine.

5. The data query method according to claim 1, characterized in that: The method further includes: monitoring data change events of a database through an application service, and parsing the data change events through the application service.

6. The data query method according to claim 1, characterized in that: If the model data includes entity attribute value model data, before monitoring the data change event of the database, the method further includes: Obtaining basic information of a preset product and storing the basic information in an entity table of an entity attribute value model; and, Obtaining the attribute name of the preset product, and storing the attribute name in the attribute table of the entity attribute value model; and, Obtaining attribute values ​​of the preset product, and storing the attribute values ​​in a value table of an entity attribute value model; The basic information includes: a unique identifier of the product and a product name; The attribute name includes: a unique identifier of the attribute and an attribute name; The attribute value includes: a unique identifier of the value, a foreign key referencing an entity table, a foreign key referencing an attribute table, and an actual value of the attribute.

7. A data query system, characterized in that: The system comprises: A monitoring module, used for monitoring data change events in a database, wherein the database stores model data; A conversion module, used to parse the data change event and convert all or part of the model data into document data of the search engine when the model data changes; A query module is used to perform data query from the search engine in response to a data query request; wherein the data query request is used to query the model data.

8. The data query system according to claim 7, characterized in that: If the model data includes entity attribute value model data, the process in which the conversion module converts all or part of the entity attribute value model data into document data of the search engine when data changes occur in the entity attribute value model data includes: When data changes occur in the value table of the entity attribute value model data, the column fields of the entity table are obtained through the foreign key entity number of the value table, and the attribute data of the attribute table are obtained through the foreign key attribute number of the value table; Associating the column fields of the entity table with the attribute data of the attribute table as search engine graph fields; The data in the value table are parsed item by item, and the row data in the value table are converted into column data of the search engine, and the document data of the search engine is generated by combining the search engine graph field and the column data of the search engine, and the document data is stored in the search engine.

9. A computer device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the data query method according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the steps of the data query method described in any one of claims 1 to 6 are implemented.