A data processing method and device

Abstract

The embodiment of the present invention provides a data processing method and device, which relates to the field of data processing technology, and in particular to the field of medical data processing, to obtain the business data of the first user to be stored; extract the data corresponding to the preset index field in the business data as the first index data; obtain the primary key of the business data as the first primary key; store the first primary key and the first index data in the first database in correspondence; use the first primary key as a row key and store the first non-index data belonging to the preset non-index field in the business data in the second database in correspondence. In this way, it is possible to achieve the storage and reading of a large amount of business data, and improve the reading efficiency of business data.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, in particular to the field of medical data processing, and more particularly to a data processing method and device. Background Art

[0002] In people's work and life, a large amount of data is generated every day and needs to be stored. The stored data often has great value. For example, the stored data may be the medical business data of the patient, or the travel business data of the passenger. Summary of the invention

[0003] The purpose of the embodiments of the present invention is to provide a data processing method and device, which can realize the storage and reading of a large amount of business data and improve the reading efficiency of business data. The specific technical solution is as follows:

[0004] In a first aspect, in order to achieve the above-mentioned object, an embodiment of the present invention discloses a data processing method, the method comprising:

[0005] Acquire the service data of the first user to be stored;

[0006] Extracting data corresponding to a preset index field in the business data as first index data;

[0007] Obtaining the primary key of the business data as the first primary key;

[0008] storing the first primary key and the first index data in a first database correspondingly;

[0009] The first primary key is used as a row key and is stored in the second database correspondingly with the first non-index data belonging to a preset non-index field in the business data.

[0010] Optionally, before storing the first primary key and the first index data in the first database correspondingly, the method further includes:

[0011] Obtaining a user identifier of the first user in the current sub-service system as a first sub-system user identifier;

[0012] Based on a preset correspondence between a subsystem user identifier and a system-wide user identifier, determining a system-wide user identifier corresponding to the first subsystem user identifier as a first system-wide user identifier;

[0013] The storing the first primary key and the first index data in a first database correspondingly includes:

[0014] The first primary key, the first subsystem user identifier, the first full-system user identifier and the first index data are stored in a first database correspondingly.

[0015] Optionally, the storing the first primary key as first non-index data belonging to a preset non-index field in the business data in the second database in correspondence includes:

[0016] The first primary key is used as a row key and is correspondingly stored in the second database with the first non-index data belonging to a preset non-index field in the business data, the first subsystem user identifier and the first full-system user identifier.

[0017] Optionally, the method further includes:

[0018] Generate metadata; wherein the metadata is used to indicate that preset index fields are stored in the first database, and preset non-index fields are stored in the second database.

[0019] Optionally, the first database is an ElasticSearch database, and the second database is an HBase database.

[0020] In a second aspect, in order to achieve the above-mentioned purpose, an embodiment of the present invention discloses a data processing method, the method comprising:

[0021] Receive business data reading requests sent by clients;

[0022] In the first database, the data of the condition field carried in the business data read request is used as index data, and the corresponding primary key is queried as the first primary key;

[0023] In the second database, the first primary key is used as a row key to query the corresponding non-index data as a non-index query result;

[0024] Based on the non-index query result, a final query result is sent to the client.

[0025] Optionally, in the first database, before using the data of the condition field carried in the business data read request as index data and querying the corresponding primary key as the first primary key, the method further includes:

[0026] Determine whether the query field and the condition field carried in the business data read request exist in the metadata; wherein the metadata is used to indicate that a preset index field is stored in the first database, and that a preset non-index field is stored in the second database;

[0027] The method of using the data of the condition field carried in the business data read request as index data and querying the corresponding primary key as the first primary key in the first database includes:

[0028] In the case that the query field and the condition field exist in the metadata, in the first database, the data of the condition field is used as index data to query the corresponding primary key as the first primary key.

[0029] Optionally, in the second database, before querying the corresponding non-index data using the first primary key as a row key and taking the non-index query result as the result, the method further includes:

[0030] Determine whether the query field carried in the business data reading request includes a preset non-index field;

[0031] The querying of the corresponding non-index data in the second database using the first primary key as the row key as the non-index query result includes:

[0032] In the case where the query field includes a preset non-index field, in the second database, the first primary key is used as a row key to obtain non-index data corresponding to the preset non-index field included in the query field as a non-index query result.

[0033] Optionally, before sending the final query result to the client based on the non-index query result, the method further includes:

[0034] Determine whether the query field carried in the business data reading request contains a preset index field;

[0035] If the query field includes a preset index field, then in the first database, the data corresponding to the data in the condition field and belonging to the preset index field included in the query field are searched as the index query result;

[0036] The sending a final query result to the client based on the non-index query result includes:

[0037] Sending to the client a final query result including the non-index query result and the index query result.

[0038] Optionally, the first database is an ElasticSearch database, and the second database is an HBase database.

[0039] In a third aspect, in order to achieve the above-mentioned purpose, an embodiment of the present invention discloses a data processing device, the device comprising:

[0040] A business data acquisition module, used to acquire the business data of the first user to be stored;

[0041] A first index data acquisition module, used to extract data corresponding to a preset index field in the business data as first index data;

[0042] A first primary key acquisition module, used to acquire the primary key of the business data as the first primary key;

[0043] A first storage module, used for storing the first primary key and the first index data in a first database correspondingly;

[0044] The second storage module is used to store the first primary key as a row key and the first non-index data belonging to a preset non-index field in the business data in a second database accordingly.

[0045] Optionally, the device further comprises:

[0046] A first subsystem user identification acquisition module, configured to acquire a user identification of the first user in the current sub-business system as a first subsystem user identification before storing the first primary key and the first index data in the first database in correspondence with each other;

[0047] A first system-wide user identifier acquisition module, configured to determine, based on a preset correspondence between a subsystem user identifier and a system-wide user identifier, a system-wide user identifier corresponding to the first subsystem user identifier as a first system-wide user identifier;

[0048] The first storage module is specifically configured to store the first primary key, the first subsystem user identifier, the first full-system user identifier, and the first index data in a first database correspondingly.

[0049] Optionally, the first storage module is specifically used to use the first primary key as a row key, and store it in the second database correspondingly with the first non-index data belonging to a preset non-index field in the business data, the first subsystem user identifier and the first full-system user identifier.

[0050] Optionally, the device further comprises:

[0051] The metadata generation module is used to generate metadata; wherein the metadata is used to indicate that the first database stores preset index fields, and the second database stores preset non-index fields.

[0052] Optionally, the first database is an ElasticSearch database, and the second database is an HBase database.

[0053] In a fourth aspect, in order to achieve the above-mentioned purpose, an embodiment of the present invention discloses a data processing device, the device comprising:

[0054] A business data reading request acquisition module is used to receive a business data reading request sent by a client;

[0055] A first primary key query module is used to query the corresponding primary key in the first database using the data of the condition field carried in the business data read request as index data as the first primary key;

[0056] A non-index query result acquisition module, used to query the corresponding non-index data in the second database using the first primary key as a row key as a non-index query result;

[0057] The final query result sending module is used to send the final query result to the client based on the non-index query result.

[0058] Optionally, the device further comprises:

[0059] A first judgment module is used to use the data of the condition field carried in the business data read request as index data in the first database to query the corresponding primary key, and before using it as the first primary key, to judge whether the query field and the condition field carried in the business data read request exist in the metadata; wherein the metadata is used to indicate that a preset index field is stored in the first database, and that a preset non-index field is stored in the second database;

[0060] The first primary key query module is specifically used to query the corresponding primary key as the first primary key in the first database using the data of the condition field as index data when the query field and the condition field exist in the metadata.

[0061] Optionally, the device further comprises:

[0062] A second judgment module is used to query the corresponding non-index data in the second database using the first primary key as a row key, and before determining whether the query field carried in the business data read request contains a preset non-index field as a non-index query result;

[0063] The non-index query result acquisition module is specifically used to obtain, in the second database, non-index data corresponding to the preset non-index field contained in the query field using the first primary key as the row key, as a non-index query result when the query field contains a preset non-index field.

[0064] Optionally, the device further comprises:

[0065] A third judgment module is used to detect whether the query field carried in the business data reading request contains a preset index field before sending the final query result to the client based on the non-index query result;

[0066] An index query result acquisition module, configured to, if the query field includes a preset index field, query the first database for data corresponding to the data in the condition field and belonging to the preset index field included in the query field as the index query result;

[0067] The final query result sending module is specifically configured to send a final query result including the non-index query result and the index query result to the client.

[0068] Optionally, the first database is an ElasticSearch database, and the second database is an HBase database.

[0069] In another aspect of the present invention, in order to achieve the above-mentioned purpose, an embodiment of the present invention further discloses an electronic device, the electronic device comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus;

[0070] The memory is used to store computer programs;

[0071] The processor is used to implement any of the data processing methods described above when executing the program stored in the memory.

[0072] In another aspect of the present invention, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the data processing method as described above is implemented.

[0073] An embodiment of the present invention further provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute any of the above-mentioned data processing methods.

[0074] Beneficial effects of the embodiments of the present invention:

[0075] The data processing method provided by the embodiment of the present invention can obtain the business data of the first user to be stored; extract the data corresponding to the preset index field in the business data as the first index data; obtain the primary key of the business data as the first primary key; store the first primary key and the first index data in the first database correspondingly; use the first primary key as the row key and store it in the second database correspondingly with the first non-index data belonging to the preset non-index field in the business data.

[0076] Based on the above processing, the user's business data can be stored in different databases respectively, and the data in multiple databases can be associated through primary keys, so that a large amount of business data can be stored and read. In addition, the data corresponding to the preset index field is stored in the first database, which can realize the rapid retrieval of the business data belonging to the preset index field and improve the reading efficiency of the business data.

[0077] Of course, it is not necessary to achieve all of the advantages described above at the same time to implement any product or method of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0079] Figure 1 A flowchart of a data processing method provided by an embodiment of the present invention;

[0080] Figure 2 A flowchart of another data processing method provided by an embodiment of the present invention;

[0081] Figure 3 A schematic diagram of a data processing flow provided by an embodiment of the present invention;

[0082] Figure 4 A flowchart of a data processing method provided by an embodiment of the present invention;

[0083] Figure 5 A flowchart of another data processing method provided by an embodiment of the present invention;

[0084] Figure 6 A flowchart of another data processing method provided by an embodiment of the present invention;

[0085] Figure 7 A flowchart of another data processing method provided by an embodiment of the present invention;

[0086] Figure 8 A schematic diagram of a data processing flow provided by an embodiment of the present invention;

[0087] Fig. 9 A structural diagram of a data processing device provided by an embodiment of the present invention;

[0088] Fig.10 A structural diagram of a data processing device provided by an embodiment of the present invention;

[0089] Fig.11 A structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0090] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field based on this application belong to the scope of protection of the present invention.

[0091] With the rapid development of computer technology, medical institutions have gradually built medical business data indexed by patient information. Medical business data can include: the patient's name, the patient's mobile phone number, the patient's medical card number, the patient's visit time, the patient's doctor, and the doctor's prescription for the patient. These medical business data can be used by the corresponding medical institution to query its patient information. At present, the medical institution has built a patient master index (Enterprise Master Patient Index, which refers to a code within a specific domain that is used to identify each patient / patient instance in the domain and maintain its uniqueness) with patient information as the main index, but the business data related to patients are distributed in different business systems, which increases the unified association sharing.

[0092] In addition, medical business data is generally stored independently using relational databases according to business characteristics, and retrieved in a unified manner through big data Hbase. However, the EMPI patient master index data and the business data generated during the patient's medical treatment are largely separated, and medical business data can only be stored and retrieved in a small area using relational databases, which makes it difficult to store and retrieve massive data in a large area. The unified retrieval of big data Hbase has a single retrieval field and limited application business scenarios.

[0093] Based on this, the present invention proposes a data processing method and device to solve some or all of the above technical problems. It should be noted that the present invention can be applied to the processing of medical business data, and can also be applied to the processing of business in related fields that are common and similar to medical business data, such as passenger travel business data in a transportation system.

[0094] The embodiment of the present invention provides a data processing method, which can be applied to a server, and the server can be a data storage server, that is, used to store data, wherein the stored data can be the medical service data of the above-mentioned patient. Figure 1 , Figure 1A flowchart of a data processing method provided by an embodiment of the present invention, the method may include the following steps:

[0095] S101: Acquire service data of a first user to be stored.

[0096] S102: Extracting data corresponding to a preset index field in the business data as first index data.

[0097] S103: Obtain the primary key of the business data as the first primary key.

[0098] S104: Store the first primary key and the first index data in the first database in correspondence.

[0099] S105: Using the first primary key as a row key and storing it in the second database in correspondence with the first non-index data belonging to the preset non-index field in the business data.

[0100] The data processing method provided by the embodiment of the present invention can store the business data of the user in different databases respectively. The business data in multiple databases are associated through primary keys, so that the storage and reading of business data with a large amount of data can be realized. In addition, the data corresponding to the preset index field is stored in the first database, which can realize the rapid retrieval of the business data belonging to the preset index field and improve the reading efficiency of the business data.

[0101] With respect to step S101 , different business scenarios correspond to different business data.

[0102] For example, in the scenario of a patient visiting a doctor, the business data may be the patient's medical business data. The medical business data may include: the patient's name, the patient's mobile phone number, the patient's medical card number, the time of the patient's visit, the doctor the patient visited, the prescription the doctor gave the patient, and the patient's medical history.

[0103] For the scenario of passenger travel, the business data may be the passenger's travel business data. The travel business data may include: the passenger's name, the passenger's mobile phone number, the passenger's ID number, the passenger's travel time, the passenger's travel vehicle, the passenger's departure place and the passenger's destination, etc.

[0104] With respect to step S102 , the preset index field may be set by a technician according to business requirements.

[0105] In one implementation, the preset index field may be a field with a relatively small amount of business data, or may be a field that requires rapid retrieval.

[0106] For example, for a patient's visit to a doctor, the preset index fields may include: the patient's name, the patient's mobile phone number, the patient's medical card number, and the time the patient visited the doctor. Alternatively, the preset index fields may include: the patient's name, the patient's mobile phone number, the patient's medical card number, and the doctor the patient visited.

[0107] For step S103, the primary key is a unique identifier for determining the business data. Specifically, the primary key of the business data can be a field contained therein, or a combination of multiple fields. For example, for a patient's visit, the primary key of the business data can be represented by a combination of the patient's medical card number and the time of the patient's visit.

[0108] In addition, if there is no field in the business data that can be used as the primary key, a unique identifier can be generated for the business data as its primary key.

[0109] For step S105, other fields in the business data except the preset index field can be used as preset non-index fields. In one implementation, the preset non-index field can be a field with a large amount of business data, or a field that does not require fast retrieval.

[0110] For example, for a patient visit scenario, the preset non-index fields may include: the prescription written by the doctor for the patient and the patient's medical records.

[0111] The embodiment of the present invention does not limit the execution order of step S104 and step S105.

[0112] In one embodiment, the first database is an ElasticSearch database, and the second database is an HBase database.

[0113] Based on the above processing, the ElasticSearch database can be used to realize multi-field rapid retrieval of business data of preset index fields, and the HBase database can be used to realize large-scale storage of business data of preset non-index fields.

[0114] In addition, based on the distributed and highly available characteristics of the ElasticSearch database and the HBase database, dynamic scaling of business data storage can be achieved. In addition, based on the ElasticSearch database, word segmentation and fuzzy retrieval of preset index fields can also be achieved, and support can be provided for business data reading requests (for example, like SQL (Structured Query Language) statements).

[0115] In one embodiment, see Figure 2 ,exist Figure 1On the basis of, before the above step S104, the method may further include the following steps:

[0116] S106: Obtain a user identifier of the first user in the current sub-service system as a first sub-system user identifier.

[0117] S107: Based on the preset correspondence between the subsystem user identifier and the whole system user identifier, determine the whole system user identifier corresponding to the first subsystem user identifier as the first whole system user identifier.

[0118] Accordingly, the above step S104 may include:

[0119] S1041: Store the first primary key, the first subsystem user identifier, the first full-system user identifier, and the first index data in the first database in correspondence.

[0120] The subsystem user ID may be a unique ID assigned to the user by the sub-business system, and the whole system user ID may be a unique ID assigned to the user by the sub-business system.

[0121] In the embodiment of the present invention, the business system may be divided based on the size of the scope.

[0122] For example, for the scenario of patient consultation, the hospitals in the entire urban area can be regarded as a full business system, and each hospital in the urban area can be regarded as a sub-business system.

[0123] Correspondingly, a patient has a corresponding user ID in each hospital's sub-business system (ie, a sub-system user ID), and also has a corresponding user ID in the city's full-business system (ie, a full-system user ID).

[0124] It is understandable that the subsystem user IDs corresponding to the patient in each sub-business system may be different. That is, when recording the business data of the patient, each hospital usually uses the corresponding user ID in its own sub-business system to mark the patient.

[0125] Therefore, in order to achieve intercommunication of the business data respectively recorded by each sub-business system, the correspondence between the first sub-system user identifier and the first full-system user identifier may also be recorded in the first database.

[0126] For example, for the scenario of patient visits, EMPI (Enterprise Master Patient Index) can be obtained. Through EMPI, the correspondence between the subsystem user ID and the system-wide user ID can be obtained. EMPI represents a code within a specific domain that is used to identify each patient instance within the specific domain and maintain its uniqueness.

[0127] In one embodiment, accordingly, the above step S105 may include the following steps:

[0128] The first primary key is used as a row key and is correspondingly stored in the second database with the first non-index data belonging to the preset non-index field in the business data, the first subsystem user identifier and the first full-system user identifier.

[0129] In the embodiment of the present invention, after the first subsystem user identifier and the first full-system user identifier are determined, they may be stored correspondingly in the second database to achieve intercommunication of the business data respectively recorded in each sub-business system.

[0130] In one embodiment, the method may further include the following steps:

[0131] Generate metadata.

[0132] The metadata is used to indicate that the first database stores preset index fields, and the second database stores preset non-index fields.

[0133] In an embodiment of the present invention, metadata corresponding to the business data can also be generated. For example, the metadata can record the identifiers of preset index fields and preset non-index fields, and the data of the preset index fields are recorded in a first database, and the data of the preset non-index fields are recorded in a second database.

[0134] Based on the generated metadata, the management efficiency of the stored business data can be improved, and the efficiency of business data retrieval can be improved.

[0135] See also Figure 3 , Figure 3 A schematic diagram of a data processing flow provided by an embodiment of the present invention.

[0136] Based on EMPI, the system-wide user ID corresponding to the subsystem user ID can be obtained.

[0137] Generate metadata: used to indicate that the first database (ie, the ElasticSearch database) stores preset index fields, and the second database (ie, the HBase database) stores preset non-index fields.

[0138] Determine whether there is a field that serves as the primary key in the business data. If so, use the field as the primary key. If not, generate a primary key.

[0139] Then, the business data can be stored in the database. That is, the primary key, subsystem user ID, system-wide user ID and index data belonging to the preset index field in the business data are stored in the ElasticSearch database in correspondence; the primary key is used as the row key, and the non-index data belonging to the preset non-index field in the business data, the subsystem user ID and the system-wide user ID are stored in the HBase database in correspondence.

[0140] Corresponding to the above data processing method, an embodiment of the present invention further provides a data processing method, which can be applied to a server, which can be a data reading server, that is, used to read data, and the data reading server and the above data processing server can be the same server. Figure 4 , Figure 4 A flowchart of a data processing method provided by an embodiment of the present invention, the method may include the following steps:

[0141] S401: Receive a business data reading request sent by a client.

[0142] S402: In the first database, the data of the condition field carried in the business data read request is used as index data to query the corresponding primary key as the first primary key.

[0143] S403: In the second database, the first primary key is used as a row key to query corresponding non-index data as a non-index query result.

[0144] S404: Based on the non-index query result, send the final query result to the client.

[0145] The data processing method provided by the embodiment of the present invention can associate business data stored in different databases through primary keys, thereby enabling storage and reading of business data of a large amount of data. In addition, the first database stores index data corresponding to the conditional field, that is, it can achieve rapid retrieval of business data corresponding to the conditional field, thereby improving the reading efficiency of business data.

[0146] The business data stored in the first database and the second database in the embodiment of the present invention may be obtained based on the above-mentioned data processing method applied to the data storage server.

[0147] In one embodiment, the first database is an ElasticSearch database, and the second database is an HBase database.

[0148] Based on the above processing, the ElasticSearch database can be used to realize multi-field rapid retrieval of business data of conditional fields, and the HBase database can be used to realize large-scale storage of non-indexed data.

[0149] In addition, based on the distributed and highly available features of the ElasticSearch database and the HBase database, dynamic scaling of business data storage can be achieved.

[0150] In one embodiment, in order to further improve the management efficiency of business data and improve the efficiency of reading business data, metadata of the business data stored in the database may also be recorded.

[0151] Accordingly, see Figure 5 ,exist Figure 4 On the basis of, before the above step S402, the method may further include the following steps:

[0152] S405: Determine whether the query field and condition field carried in the business data reading request exist in the metadata.

[0153] The metadata is used to indicate that the first database stores preset index fields, and the second database stores preset non-index fields.

[0154] Accordingly, the above step S402 may include the following steps:

[0155] S4021: When there are query fields and condition fields in the metadata, in the first database, the data of the condition field is used as index data to query the corresponding primary key as the first primary key.

[0156] In the embodiment of the present invention, the service data reading request may be a SQL-like statement.

[0157] In one implementation, the query field indicates the field that the user requests to query, and the condition field indicates the query condition when the user requests to query. For example, if the user needs to query the time of the patient's medical card number A's visit, the query field is "the time of the patient's visit" and the condition field is "the patient's medical card number".

[0158] Therefore, after receiving a business data read request, the query field and condition field carried therein can be obtained, and it can be determined whether there is a corresponding record in the metadata.

[0159] If so, it indicates that the first database and the second database store corresponding business data, and corresponding query can be performed.

[0160] In one embodiment, if there is no corresponding record in the metadata, it indicates that the first database and the second database do not store the corresponding business data. Accordingly, it can be directly prompted that the query fails.

[0161] In one embodiment, see Figure 6 ,exist Figure 4On the basis of, before the above step S403, the method may further include the following steps:

[0162] S406: Determine whether the query field carried in the business data reading request includes a preset non-index field.

[0163] S4031: When the query field includes a preset non-index field, in the second database, using the first primary key as the row key, obtain non-index data corresponding to the preset non-index field included in the query field as a non-index query result.

[0164] In an embodiment of the present invention, after determining the query field carried in the business data reading request, if it is determined that the query field contains a preset non-index field, it indicates that the user needs to query the business data of the non-index field stored in the second database, and then, a corresponding query can be performed in the second database to obtain a non-index query result.

[0165] If it is determined that the query field does not include a preset non-index field, it indicates that the user does not need to query the business data of the non-index field stored in the second database, and the query may not be performed in the second database.

[0166] In one embodiment, see Figure 7 ,exist Figure 4 On the basis of, before the above step S404, the method may further include the following steps:

[0167] S407: Determine whether the query field carried in the business data reading request includes a preset index field.

[0168] S408: If the query field includes a preset index field, then the first database is searched for data corresponding to the data in the condition field and belonging to the preset index field included in the query field as the index query result.

[0169] Accordingly, the above step S404 may include the following steps:

[0170] S4041: Sending the final query result including the non-index query result and the index query result to the client.

[0171] In the embodiment of the present invention, after determining the query field carried in the business data read request, if the query field includes a preset index field (which may be referred to as a target index field), it indicates that the user needs to query the business data corresponding to the target index field recorded in the first database. Then, the data corresponding to the data in the condition field and belonging to the target index field may be obtained in the first database to obtain the index query result.

[0172] Furthermore, the non-index query result and the index query result may be merged and sent to the client.

[0173] See also Figure 8 , Figure 8 A schematic diagram of a data processing flow provided by an embodiment of the present invention.

[0174] After receiving the business data reading request sent by the client, it can be determined whether the business data reading request complies with the specification.

[0175] For example, the method provided in the embodiment of the present invention only allows one data table to be queried at a time, for example, only a patient's medical treatment data table can be queried at a time, or only a patient's hospitalization data table can be queried at a time.

[0176] If the business data read request is a request for multiple data tables at the same time, it can be determined that it does not meet the specifications and a reminder message is returned to the client.

[0177] If the business data reading request complies with the specification, it can be determined whether there is a query field in the metadata, that is, whether the field requested by the client is stored.

[0178] If it does not exist, a reminder message can be returned to the client.

[0179] If so, it is possible to continue to determine whether there is a condition field in the metadata, that is, to determine whether the query condition for the client query is stored.

[0180] If it does not exist, a reminder message may be returned to the client.

[0181] If it exists, you can perform an index field query, that is, obtain the primary key corresponding to the condition field in the ElasticSearch database. In addition, you can also obtain the business data of the preset index field included in the query field corresponding to the condition field in the ElasticSearch database to obtain the index query result.

[0182] In addition, it is also possible to determine whether the query field contains a preset non-index field.

[0183] If so, a non-index field query can be performed, that is, in the Hbase database, the business data corresponding to the preset non-index field contained in the query field is obtained according to the primary key to obtain the non-index query result.

[0184] Then, the index query result and the non-index query result may be merged and sent to the client.

[0185] Based on the same inventive concept, the embodiment of the present invention further provides a data processing device, see Fig. 9 , Fig. 9 A structural diagram of a data processing device provided in an embodiment of the present invention, the device comprising:

[0186] A service data acquisition module 901 is used to acquire service data of a first user to be stored;

[0187] A first index data acquisition module 902, used to extract data corresponding to a preset index field in the business data as first index data;

[0188] A first primary key acquisition module 903, used to acquire the primary key of the business data as the first primary key;

[0189] A first storage module 904, configured to store the first primary key and the first index data in a first database correspondingly;

[0190] The second storage module 905 is used to store the first primary key as a row key and the first non-index data belonging to a preset non-index field in the business data in a corresponding manner in a second database.

[0191] Optionally, the device further comprises:

[0192] A first subsystem user identification acquisition module, configured to acquire a user identification of the first user in the current sub-business system as a first subsystem user identification before storing the first primary key and the first index data in the first database in correspondence with each other;

[0193] A first system-wide user identifier acquisition module, configured to determine, based on a preset correspondence between a subsystem user identifier and a system-wide user identifier, a system-wide user identifier corresponding to the first subsystem user identifier as a first system-wide user identifier;

[0194] The first storage module 904 is specifically configured to store the first primary key, the first subsystem user identifier, the first full-system user identifier, and the first index data in a first database correspondingly.

[0195] Optionally, the first storage module 904 is specifically used to use the first primary key as a row key, and store it in the second database correspondingly with the first non-index data belonging to a preset non-index field in the business data, the first subsystem user identifier and the first full-system user identifier.

[0196] Optionally, the device further comprises:

[0197] The metadata generation module is used to generate metadata; wherein the metadata is used to indicate that the first database stores preset index fields, and the second database stores preset non-index fields.

[0198] Optionally, the first database is an ElasticSearch database, and the second database is an HBase database.

[0199] Based on the same inventive concept, the embodiment of the present invention further provides a data processing device, see Fig.10 , Fig.10 A structural diagram of a data processing device provided in an embodiment of the present invention, the device comprising:

[0200] The business data reading request acquisition module 1001 is used to receive the business data reading request sent by the client;

[0201] A first primary key query module 1002 is used to query the corresponding primary key in the first database using the data of the condition field carried in the business data read request as index data as the first primary key;

[0202] The non-index query result acquisition module 1003 is used to query the corresponding non-index data in the second database using the first primary key as a row key as a non-index query result;

[0203] The final query result sending module 1004 is used to send the final query result to the client based on the non-index query result.

[0204] Optionally, the device further comprises:

[0205] A first judgment module is used to use the data of the condition field carried in the business data read request as index data in the first database to query the corresponding primary key, and before using it as the first primary key, to judge whether the query field and the condition field carried in the business data read request exist in the metadata; wherein the metadata is used to indicate that a preset index field is stored in the first database, and that a preset non-index field is stored in the second database;

[0206] The first primary key query module 1002 is specifically used to query the corresponding primary key as the first primary key in the first database using the data of the condition field as index data when the query field and the condition field exist in the metadata.

[0207] Optionally, the device further comprises:

[0208] A second judgment module is used to query the corresponding non-index data in the second database using the first primary key as a row key, and before determining whether the query field carried in the business data read request contains a preset non-index field as a non-index query result;

[0209] The non-index query result acquisition module 1003 is specifically used to obtain, in the second database, non-index data corresponding to the preset non-index field contained in the query field using the first primary key as the row key, as a non-index query result when the query field contains a preset non-index field.

[0210] Optionally, the device further comprises:

[0211] A third judgment module is used to detect whether the query field carried in the business data reading request contains a preset index field before sending the final query result to the client based on the non-index query result;

[0212] An index query result acquisition module, configured to, if the query field includes a preset index field, query the first database for data corresponding to the data in the condition field and belonging to the preset index field included in the query field as the index query result;

[0213] The final query result sending module 1004 is specifically configured to send the final query result including the non-index query result and the index query result to the client.

[0214] Optionally, the first database is an ElasticSearch database, and the second database is an HBase database.

[0215] The embodiment of the present invention further provides an electronic device, such as Fig.11 As shown, it includes a processor 1101, a communication interface 1102, a memory 1103 and a communication bus 1104, wherein the processor 1101, the communication interface 1102, and the memory 1103 communicate with each other through the communication bus 1104.

[0216] Memory 1103, used for storing computer programs;

[0217] The processor 1101 is used to implement the following steps when executing the program stored in the memory 1103:

[0218] Acquire business data of a first user to be stored; extract data corresponding to a preset index field in the business data as first index data; acquire a primary key of the business data as a first primary key; store the first primary key and the first index data in a first database in correspondence; use the first primary key as a row key and store the first non-index data belonging to a preset non-index field in the business data in a second database in correspondence;

[0219] or,

[0220] Receive a business data read request sent by a client; in a first database, use the data of the condition field carried in the business data read request as index data, query the corresponding primary key as the first primary key; in a second database, use the first primary key as a row key, query the corresponding non-index data as the non-index query result; based on the non-index query result, send a final query result to the client.

[0221] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0222] The communication interface is used for communication between the above electronic device and other devices.

[0223] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0224] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0225] In another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above data processing methods are implemented.

[0226] In another embodiment of the present invention, a computer program product including instructions is provided. When the computer program product is run on a computer, the computer executes any one of the data processing methods in the above embodiments.

[0227] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0228] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0229] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, electronic device, computer-readable storage medium, and computer program product embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0230] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A data processing method, characterized in that: The method comprises: Acquire the service data of the first user to be stored; Extracting data corresponding to a preset index field in the business data as first index data; Obtaining the primary key of the business data as the first primary key; storing the first primary key and the first index data in a first database correspondingly; The first primary key is used as a row key, and is stored in the second database correspondingly with the first non-index data belonging to the preset non-index field in the business data; The method further comprises: Generate metadata corresponding to the business data; wherein the metadata is used to indicate that preset index fields are stored in the first database, and preset non-index fields are stored in the second database, and the metadata records the identifier of the preset index field and the identifier of the preset non-index field.

2. The method according to claim 1, characterized in that Before storing the first primary key and the first index data in the first database in correspondence, the method further includes: Obtaining a user identifier of the first user in the current sub-service system as a first sub-system user identifier; Based on a preset correspondence between a subsystem user identifier and a system-wide user identifier, determining a system-wide user identifier corresponding to the first subsystem user identifier as a first system-wide user identifier; The storing the first primary key and the first index data in a first database correspondingly includes: The first primary key, the first subsystem user identifier, the first full-system user identifier and the first index data are stored in a first database correspondingly.

3. The method according to claim 2, characterized in that The storing the first primary key as the first non-index data belonging to the preset non-index field in the business data in the second database in correspondence includes: The first primary key is used as a row key and is correspondingly stored in the second database with the first non-index data belonging to a preset non-index field in the business data, the first subsystem user identifier and the first full-system user identifier.

4. The method according to any one of claims 1 to 3, characterized in that: The first database is an ElasticSearch database, and the second database is an HBase database.

5. A data processing method, characterized in that: The method comprises: Receive business data reading requests sent by clients; In the first database, the data of the condition field carried in the business data read request is used as index data, and the corresponding primary key is queried as the first primary key; In the second database, the first primary key is used as a row key to query the corresponding non-index data as a non-index query result; Based on the non-index query result, sending a final query result to the client; In the first database, before querying the corresponding primary key using the data of the condition field carried in the business data read request as the index data and taking it as the first primary key, the method further includes: Determine whether the query field and condition field carried in the business data read request exist in the metadata; wherein the metadata corresponding to each piece of business data is used to indicate that a preset index field of the business data is stored in the first database, and a preset non-index field of the business data is stored in the second database, and the metadata supports recording an identifier of the preset index field and an identifier of the preset non-index field; If it does not exist, it is determined that the query fails; The method of using the data of the condition field carried in the business data read request as index data and querying the corresponding primary key as the first primary key in the first database includes: In the case that the query field and the condition field exist in the metadata, in the first database, the data of the condition field is used as index data to query the corresponding primary key as the first primary key.

6. The method according to claim 5, characterized in that In the second database, before querying the corresponding non-index data using the first primary key as a row key and obtaining the non-index query result, the method further includes: Determine whether the query field carried in the business data reading request includes a preset non-index field; The querying of the corresponding non-index data in the second database using the first primary key as the row key as the non-index query result includes: In the case where the query field includes a preset non-index field, in the second database, the first primary key is used as a row key to obtain non-index data corresponding to the preset non-index field included in the query field as a non-index query result.

7. The method according to claim 5, characterized in that Before sending the final query result to the client based on the non-index query result, the method further includes: Determine whether the query field carried in the business data reading request contains a preset index field; If the query field includes a preset index field, then in the first database, the data corresponding to the data in the condition field and belonging to the preset index field included in the query field are searched as the index query result; The sending a final query result to the client based on the non-index query result includes: Sending to the client a final query result including the non-index query result and the index query result.

8. The method according to any one of claims 5 to 7, characterized in that: The first database is an ElasticSearch database, and the second database is an HBase database.

9. A data processing device, characterized in that: The device comprises: A business data acquisition module, used to acquire the business data of the first user to be stored; A first index data acquisition module, used to extract data corresponding to a preset index field in the business data as first index data; A first primary key acquisition module, used to acquire the primary key of the business data as the first primary key; A first storage module, used for storing the first primary key and the first index data in a first database correspondingly; A second storage module, configured to store the first primary key as a row key and the first non-index data belonging to a preset non-index field in the business data in a second database correspondingly; The device also includes: A metadata generation module is used to generate metadata corresponding to the business data; wherein the metadata is used to indicate that preset index fields are stored in the first database and preset non-index fields are stored in the second database, and the metadata records the identifiers of the preset index fields and the identifiers of the preset non-index fields.

10. The device according to claim 9, characterized in that The device also includes: A first subsystem user identification acquisition module, configured to acquire a user identification of the first user in the current sub-business system as a first subsystem user identification before storing the first primary key and the first index data in the first database in correspondence with each other; A first system-wide user identifier acquisition module, configured to determine, based on a preset correspondence between a subsystem user identifier and a system-wide user identifier, a system-wide user identifier corresponding to the first subsystem user identifier as a first system-wide user identifier; The first storage module is specifically configured to store the first primary key, the first subsystem user identifier, the first full-system user identifier, and the first index data in a first database correspondingly.

11. The device according to claim 10, characterized in that The first storage module is specifically used to use the first primary key as a row key, and store the first non-index data belonging to a preset non-index field in the business data, the first subsystem user identifier and the first full-system user identifier in the second database accordingly.

12. The device according to any one of claims 9 to 11, characterized in that: The first database is an ElasticSearch database, and the second database is an HBase database.

13. A data processing device, characterized in that: The device comprises: A business data reading request acquisition module is used to receive a business data reading request sent by a client; A first primary key query module is used to query the corresponding primary key in the first database using the data of the condition field carried in the business data read request as index data as the first primary key; A non-index query result acquisition module, used to query the corresponding non-index data in the second database using the first primary key as a row key as a non-index query result; A final query result sending module, used for sending a final query result to the client based on the non-index query result; The device also includes: A first judgment module is used to use the data of the condition field carried in the business data read request as index data in the first database to query the corresponding primary key, and before using it as the first primary key, to judge whether the query field and the condition field carried in the business data read request exist in the metadata; wherein the metadata corresponding to each piece of business data is used to indicate that the preset index field of the business data is stored in the first database, and the preset non-index field of the business data is stored in the second database, and the metadata supports recording the identifier of the preset index field and the identifier of the preset non-index field; A determination module, configured to determine that the query fails if the determination result of the first determination module is that the query does not exist; The first primary key query module is specifically used to query the corresponding primary key as the first primary key in the first database using the data of the condition field as index data when the query field and the condition field exist in the metadata.

14. The device according to claim 13, characterized in that The device also includes: A second judgment module is used to query the corresponding non-index data in the second database using the first primary key as a row key, and before determining whether the query field carried in the business data read request contains a preset non-index field as a non-index query result; The non-index query result acquisition module is specifically used to obtain, in the second database, non-index data corresponding to the preset non-index field contained in the query field using the first primary key as the row key, as a non-index query result when the query field contains a preset non-index field.

15. The device according to claim 13, characterized in that The device also includes: A third judgment module is used to detect whether the query field carried in the business data reading request contains a preset index field before sending the final query result to the client based on the non-index query result; An index query result acquisition module, configured to, if the query field includes a preset index field, query the first database for data corresponding to the data in the condition field and belonging to the preset index field included in the query field as the index query result; The final query result sending module is specifically configured to send a final query result including the non-index query result and the index query result to the client.

16. The device according to any one of claims 13 to 15, characterized in that: The first database is an ElasticSearch database, and the second database is an HBase database.

17. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1-4, or 5-8, when executing a program stored in a memory.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1-4, or 5-8 are implemented.

Images