Data Processing Method, Apparatus, Computer-Readable Medium, and Electronic Device
By structured arrangement of the data filtering information of the storage system and generating record encoding, the problem of inefficient data processing of the key-value storage system during high concurrent access is solved, efficient data access and consistent operations are achieved, and data throughput is improved.
Patent Information
- Application Number
- CN202011364836.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-11-27
- Publication Date
- 2025-07-01
- Estimated Expiration
- 2040-11-27
AI Technical Summary
The existing key-value pair storage system has low data processing efficiency when high concurrent access is available, and it is difficult to ensure the consistency of data parallel reading and writing.
By structured arrangement of data filtering information in service data query requests, record encoding is generated, and target service data whose data prefix matches the record encoding is queried in the storage system, high-performance access and multi-dimensional operations are supported, and secondary index components are avoided.
It improves the data processing efficiency and high concurrent data throughput capabilities of the data storage system, maintains an access mode consistent with the relational database, and reduces business development costs.
Smart Images

Figure CN114546982B_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of computer technology, and particularly relates to a data processing method, a data processing device, a computer-readable medium, and an electronic device. Background Art
[0002] With the development of computer and network technologies, a large amount of network data has been generated on the Internet, and most of this data is characterized by being unstructured and small-sized. Data storage systems that provide data support for business systems generally use key-value storage systems (such as DynamoDB, Redis, etc.) to store data to meet the high-concurrency requirements of data access. Key-value storage systems store data in the form of key-value pairs. As the amount of data accumulates, the amount of data stored in the value range corresponding to a single keyword Key will also become larger and larger, which will lead to a decline in the overall data throughput performance of the storage system. For example, when retrieving a certain keyword Key, it is necessary to traverse and scan all the data corresponding to the keyword Key, which not only requires a large amount of computing resources but also has low data processing efficiency. And if the value range data Value corresponding to a single keyword Key is split, it is difficult to ensure the consistency of parallel data reading and writing. Therefore, how to improve the data processing efficiency of data storage systems is an urgent problem to be solved currently. Summary of the Invention
[0003] The purpose of this application is to provide a data processing method, a data processing device, a computer-readable medium, and an electronic device, which can at least overcome to some extent the technical problem of low data processing efficiency existing in data storage systems.
[0004] Other features and advantages of this application will become obvious through the following detailed description, or will be learned partially through the practice of this application.
[0005] According to one aspect of the embodiments of this application, a data processing method is provided. The method includes: obtaining data filtering information for data query from a business data query request based on a business data table, where the data filtering information includes a table identifier of the business data table and a keyword corresponding to the business data to be queried; performing structured arrangement on the data filtering information to obtain a record code corresponding to the business data to be queried; querying target business data in the storage system whose data prefix matches the record code, and returning the target business data to the data requester.
[0006] According to one aspect of the embodiments of the present application, a data processing device is provided. The device includes: an information acquisition module configured to acquire data filtering information for data query from a business data query request based on a business data table, where the data filtering information includes a table identifier of the business data table and a keyword corresponding to the business data to be queried; a record encoding module configured to perform structured arrangement on the data filtering information to obtain a record encoding corresponding to the business data to be queried; and a data query module configured to query target business data in a storage system whose data prefix matches the record encoding, and return the target business data to a data requester.
[0007] In some embodiments of the present application, based on the above technical solution, the record encoding module includes: a template acquisition unit configured to acquire an encoding template obtained by performing structured arrangement on a plurality of identification bits according to a preset encoding format; and a template assignment unit configured to perform assignment processing on each identification bit in the encoding template according to the table identifier and the keyword to obtain a record encoding of the business data.
[0008] In some embodiments of the present application, based on the above technical solution, the template assignment unit includes: a first assignment subunit configured to perform assignment processing on a storage area identification bit in the encoding template according to the table identifier, where the storage area identification bit is used to indicate the business data table where the business data is located; and a second assignment subunit configured to perform assignment processing on a keyword identification bit adjacent to the storage area identification bit in the encoding template according to the keyword, where the keyword identification bit is used to indicate a primary keyword associated with the business data configured when storing the business data.
[0009] In some embodiments of the present application, based on the above technical solution, the template assignment unit further includes: a third assignment subunit configured to perform assignment processing on a storage type identification bit located at the head of the encoding template according to a preset data type identifier, where the storage type identification bit is used to indicate that the storage type of the current business data is table data or non-table data; and a fourth assignment subunit configured to perform assignment processing on a coding type identification bit located at the tail of the encoding template according to a preset record identifier, where the coding type identification bit is used to indicate that the coding type of the current coding is a record coding or an index coding.
[0010] In some embodiments of the present application, based on the above technical solutions, the template assignment unit further includes: a secondary keyword acquisition subunit, configured to acquire secondary keywords associated with service data; a fifth assignment subunit, configured to perform an assignment process on the secondary keyword identification bits in the encoding template according to the secondary keywords, where the secondary keyword identification bits are used to indicate the secondary keywords associated with the service data configured when storing the service data.
[0011] In some embodiments of the present application, based on the above technical solutions, the secondary keyword acquisition subunit includes: a type recognition subunit, configured to perform type recognition on the keywords in the data filtering information to obtain the service type of each keyword; a keyword classification subunit, configured to classify the keywords into primary keywords and secondary keywords associated with service data according to the service type.
[0012] In some embodiments of the present application, based on the above technical solutions, the secondary keyword acquisition subunit includes: an index field acquisition subunit, configured to acquire the index field of the service data to be queried from the data filtering information; a sixth assignment subunit, configured to perform an assignment process on the index field identification bits in the encoding template according to the index field, where the index field identification bits are used to indicate the index field configured when storing the service data; a seventh assignment subunit, configured to perform an assignment process on the encoding type identification bits in the encoding template according to a preset index identifier, where the encoding type identification bits are used to indicate that the current encoding type is record encoding or index encoding; a secondary keyword query subunit, configured to query secondary keywords in the storage system whose index prefixes match the index encoding obtained by the assignment process.
[0013] In some embodiments of the present application, based on the above technical solutions, the information acquisition module includes: a request generation unit, configured to acquire the table operation information executed by a data requester on a service data table in a service system, so as to generate a service data query request according to the table operation information; an interface call unit, configured to call an object-relational mapping interface configured between the service system and the storage system; a mapping conversion unit, configured to perform mapping conversion on the service data query request through the object-relational mapping interface to obtain data filtering information for data query on the storage system.
[0014] In some embodiments of the present application, based on the above technical solutions, the storage system includes a readable and writable file in the memory, a read-only file in the memory, and an ordered string table with a layered structure in the disk; the data query module includes: a first query unit configured to query target service data in the readable and writable file whose data prefix matches the record code; a second query unit configured to, if the query fails in the readable and writable file, query the target service data that matches the record code in the read-only file; a third query unit configured to, if the query fails in the read-only file, query the target service data that matches the record code level by level in the ordered string table.
[0015] In some embodiments of the present application, based on the above technical solutions, the data query module further includes: a data reading unit configured to perform a data reading operation on the queried target service data; a data locking unit configured to perform a locking process on the target service data so that the target service data rejects data access requests from other service data processing threads; a data returning unit configured to return the read target service data to the data requester and perform an unlocking process on the target service data.
[0016] In some embodiments of the present application, based on the above technical solutions, the data locking unit includes: a primary key obtaining subunit configured to obtain a primary key corresponding to the target service data; a primary key locking subunit configured to add a shared lock to a data object corresponding to the primary key in the storage system.
[0017] In some embodiments of the present application, based on the above technical solutions, the data returning unit includes: a record list returning subunit configured to encapsulate the read target service data into a record list composed of data records and return the record list to the data requester.
[0018] According to one aspect of the embodiments of the present application, there is provided a computer-readable medium having a computer program stored thereon, and when the computer program is executed by a processor, it implements the data processing method in the above technical solutions.
[0019] According to one aspect of the embodiments of the present application, there is provided an electronic device, which includes: a processor; and a memory for storing executable instructions of the processor; wherein, the processor is configured to execute the data processing method in the above technical solutions by executing the executable instructions.
[0020] According to one aspect of the embodiments of the present application, a computer program product or a computer program is provided. The computer program product or the computer program includes computer instructions that are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the data processing method in the above technical solution.
[0021] In the technical solution provided by the embodiments of the present application, by structuring and arranging keywords in a key-value pair storage database, it is possible to support high-performance access to a large amount of data stored based on keywords, and support the use of the same table storage access mode as a relational database on a business system, avoiding an increase in business development costs due to a non-tabular customized operation mode, and enabling multi-dimensional operations on data without using an additional configured secondary index component, improving the data processing efficiency of the data storage system and the high-concurrency data throughput capacity.
[0022] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. Brief Description of the Drawings
[0023] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application. Obviously, the drawings in the following description are only some embodiments of the present application, and those of ordinary skill in the art can obtain other drawings based on these drawings without creative efforts.
[0024] Figure 1 Schematically shows an exemplary system architecture block diagram applying the technical solution of the present application.
[0025] Figure 2 Shows a schematic diagram of the principle of the solution in an application scenario of the embodiments of the present application.
[0026] Figure 3 Shows a flowchart of the steps of the data processing method provided by an embodiment of the present application.
[0027] Figure 4 Shows a flowchart of the steps of obtaining data filtering information in an embodiment of the present application.
[0028] Figure 5 Shows a flowchart of the method steps for generating a record code based on a first coding template in an embodiment of the present application.
[0029] Figure 6 Shows a flowchart of the method steps for generating a record code based on a second coding template in an embodiment of the present application.
[0030] Figure 7 The flowchart of the method for generating an index code based on a third coding template in an embodiment of the present application is shown.
[0031] Figure 8 The schematic structural diagram of a storage system in an embodiment of the present application is shown.
[0032] Figure 9 The schematic diagram of the principle of data locking in the dimension of the primary key in an embodiment of the present application is shown.
[0033] Figure 10 The structural block diagram of a data processing device provided by an embodiment of the present application is schematically shown.
[0034] Figure 11 The structural block diagram of a computer system of an electronic device suitable for implementing the embodiments of the present application is schematically shown. Detailed implementation manners
[0035] Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more complete and comprehensive, and will fully convey the concept of the example embodiments to those skilled in the art.
[0036] In addition, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of the embodiments of the present application. However, those skilled in the art will realize that the technical solutions of the present application can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, well-known methods, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the present application.
[0037] The block diagrams shown in the drawings are only functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.
[0038] The flowcharts shown in the drawings are only illustrative and do not necessarily include all the contents and operations / steps, nor do they necessarily need to be executed in the described order. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined. Therefore, the actual execution order may change according to the actual situation.
[0039] Figure 1 Schematically shown is a block diagram of an exemplary system architecture to which the technical solution of the present application is applied.
[0040] As Figure 1 shown, the system architecture 100 may include a terminal device 110, a network 120, and a server 130. The terminal device 110 may include various electronic devices such as a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart wearable device, a smart vehicle-mounted device, etc. The server 130 may be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The network 120 may be a communication medium of various connection types capable of providing a communication link between the terminal device 110 and the server 130. For example, it may be a wired communication link or a wireless communication link.
[0041] According to the implementation requirements, the system architecture in the embodiments of the present application may have any number of terminal devices, networks, and servers. For example, the server 130 may be a server group composed of multiple server devices. In addition, the technical solution provided by the embodiments of the present application may be applied to the terminal device 110, or may be applied to the server 130, or may be jointly implemented by the terminal device 110 and the server 130. The present application does not make special limitations on this.
[0042] For example, a user can access the data stored locally through the client program installed on the terminal device 110, or can perform data communication with the server 130 through the terminal device 110 to access network data. For example, based on a video client, video data can be accessed, and based on a music client, music data can be accessed. For different types of application requirements, corresponding service systems and storage systems can be configured in the system architecture. Among them, the service system is a data interaction system for users, which can generate corresponding data access requests according to the data access information input by users, and can return the corresponding data access results to users based on the data access requests. The storage system is a data processing system running in the background, which is used to maintain the service data stored in the database and can respond to the data access requests generated by the service system to generate corresponding data access results. The data processing operations performed in the storage system mainly include data reading operations and data writing operations. By executing data reading operations and data writing operations, specific service data processing effects such as data addition, deletion, search, and modification can be achieved.
[0043] Figure 2 Shows a schematic diagram of the solution principle in an application scenario of the embodiments of the present application. As Figure 2As shown, in the embodiment of the present application, user 210 can operate the business system 220 through a terminal device. The business system 220 displays the corresponding business data operation results to the user in the form of a business data table. The business data table establishes a one-to-one, one-to-many, or many-to-many relationship model for different types of business data, so as to intuitively display the business relationships between various data to the user. For example, the business data table is used to display the employee information of an enterprise. Each data column (Column) in the table represents a data field, providing different data display dimensions for the employee information; each data row (Row) in the table represents a data record, displaying the specific information of an employee from different dimensions, such as the employee's name, position, business department, contact information, and so on.
[0044] When user 210 processes the business data table, a corresponding business data query request can be generated based on the user's operation information. The business system 220 passes the business data query request to the storage system 230 to query the corresponding business data in the storage system 230. In the storage system 230, the business data is stored in the form of key-value pairs. Each data object can include two parts: the keyword Key used for query and the value Value of the actual stored data content. When the storage system 230 receives a business data query request, it can convert the business data query request into a corresponding keyword Key, and then query the corresponding data content through the keyword Key. After encapsulating the queried data content, the storage system 230 can return the business data query result to the business system 220 to complete a business data processing operation.
[0045] The following will make a detailed description of the technical solutions such as the data processing method, data processing device, computer-readable medium, and electronic device provided by the present application in combination with specific embodiments.
[0046] Figure 3 The flowchart of the steps of the data processing method provided by an embodiment of the present application is shown. This data processing method can be executed by a terminal device, or by a server, or jointly by a terminal device and a server. As Figure 3 shown, this data processing method mainly includes steps S310 to S330.
[0047] In step S310, obtain the data filtering information used for data query from the business data query request based on the business data table. The data filtering information includes the table identifier of the business data table and the keyword corresponding to the business data to be queried.
[0048] In the technical solution provided by the embodiments of the present application, a user can perform data operations on a business system in the access mode of a table system to form a business data query request, and the data operation mode is consistent with the access mode of a traditional relational database. The user can view or edit various business data tables through a client application installed on a terminal device. In an application scenario, the user can perform table operations on business data related to an enterprise on an application program APP that provides enterprise communication applications and office management applications. For example, when the user needs to view or modify the employee information of an enterprise, the user can open a data table about employee information through the business system, and generate a corresponding business data query request by inputting or selecting query information in multiple dimensions such as enterprise name, department name, or employee name. When the storage system performs conversion processing on the business data query request, corresponding data filtering information can be obtained therefrom. For example, the table identifier table_id can be the digital number corresponding to the "employee information table", and the keyword key can be the enterprise name.
[0049] Figure 4 FIG. shows a flowchart of steps for obtaining data filtering information in an embodiment of the present application. As Figure 4 shown, obtaining data filtering information for data query from a business data query request based on a business data table in step S310 may include steps S311 to S313.
[0050] Step S311: Obtain table operation information executed by a data requester on a business data table on a business system, and generate a business data query request according to the table operation information.
[0051] The table operation information may include any operation processing executed by the user on the business data table within the operation authority. For example, after the user selects a certain row or column of data in the table, the user can issue a data operation instruction to view, modify, or delete the corresponding data object, and the business system receives the user's data operation instruction as the table operation information, thereby generating a corresponding business data query request according to the table operation information.
[0052] Step S312: Invoke an object-relational mapping interface configured between the business system and the storage system.
[0053] Object Relational Mapping (ORM) is a technology for performing relational database operations through the syntax of instance objects. By invoking the object-relational mapping interface, the data access mode based on a relational database in the business system can be converted into the data access mode of a key-value pair database used in the storage system.
[0054] Step S313: Perform mapping transformation on the business data query request through the object-relational mapping interface to obtain data filtering information for data query on the storage system.
[0055] In the business system, users can operate on data contents such as rows and columns in the business data table. The object-relational mapping interface provides external mapping functions for mapping the operations of users on the business data table into operations on objects in the key-value pair database. For example, the business data table table can be mapped to the class class, the data row row corresponding to the data record can be mapped to the object object, and the data column column corresponding to the field can be mapped to the attribute of the object.
[0056] By using the object-relational mapping interface to perform transformation processing on the business data query request, when the storage system uses a key-value pair database for data storage, a table system access mode can be provided in the business system, maintaining an access mode consistent with the traditional relational database, and overcoming the problem of increasing the business program development cost due to the non-tabular customized operation mode of the key-value pair database. Correspondingly, when the storage system obtains the data query result, it can also perform mapping transformation through the object-relational mapping interface to obtain a data processing result based on the row / column correspondence relationship adapted to the business data table.
[0057] In step S320, the data filtering information is structurally arranged to obtain the record code of the business data to be queried.
[0058] In an embodiment of the present application, a coding template obtained by structurally arranging multiple identification bits according to a preset coding format can be obtained, and then each identification bit in the coding template is assigned values according to the table identifier and the keyword to obtain the record code of the business data.
[0059] Figures 5 to 7Shows various coding templates based on a preset coding format used in the embodiments of the present application, as well as a method for assigning identification bits to each coding template. Among them, each coding template includes multiple identification bits arranged in sequence from front to back, and different types of coding templates can be used according to different contents of data filtering information. After assigning values to the identification bits in the coding template, a corresponding record code (Record Encode) can be generated. Using this record code as a keyword, the corresponding data object can be queried in the key-value pair database. In some embodiments of the present application, when a secondary index is established for some data objects in the storage system, values can be assigned to the identification bits in the coding template to generate a corresponding index code (Index Encode). Using the index code as a keyword, a secondary keyword used as a secondary index can be queried in the storage system. After generating a record code by assigning values to the identification bits in the coding template using the secondary keyword, the corresponding data object can continue to be queried according to this record code.
[0060] Figure 5 Shows the flowchart of the method steps for generating a record code based on the first coding template in an embodiment of the present application. As Figure 5 shown, the record code generated based on the first coding template can include a storage type identification bit, a storage area identification bit, a keyword identification bit, and a coding type identification bit arranged in sequence from front to back. After the coding type identification bit, the corresponding data object can be stored.
[0061] In an embodiment of the present application, for Figure 5 the method of assigning identification bits to the shown first coding template may include the following steps S510 to step S540.
[0062] Step S510: Assign a value to the storage type identification bit located at the head of the coding template according to a preset data type identifier. The storage type identification bit is used to indicate that the storage type of the current service data is tabular data or non-tabular data.
[0063] Assigning a value to the storage type identification bit according to a preset data type identifier can determine whether the storage type of the current service data is tabular data or non-tabular data when the storage system initially parses the record code, so as to determine the subsequent coding parsing format. For example, if the storage type of the current service data is tabular data, the storage type identification bit can be assigned a value of 1; if the storage type of the current service data is non-tabular data, the storage type identification bit can be assigned a value of 0.
[0064] Step S520: Assign a value to the storage area identification bit in the coding template according to the table identifier. The storage area identification bit is used to indicate the service data table where the service data is located.
[0065] The table identifier obtained from the data filtering information can be used to assign a value to the storage area identifier bit. According to the value of the storage area identifier bit, the business data table where the corresponding data object is located can be determined. The table identifier can be a pre-set digital code table_id that corresponds one-to-one with each business data table. For example, for a certain business data table, the storage area identifier bit can be assigned a value of 1 according to the table identifier.
[0066] Step S530: Assign a value to the keyword identifier bit adjacent to the storage area identifier bit in the coding template according to the keyword. The keyword identifier bit is used to indicate the primary keyword associated with the business data configured when storing the business data.
[0067] The keyword obtained from the data filtering information can be used to assign a value to the keyword identifier bit. According to the value of the keyword identifier bit, the primary keyword associated with the business data configured when the corresponding data object is stored can be determined. In some other embodiments, when storing a data object, a corresponding secondary keyword can also be configured for it to increase the data processing dimension. For example, for business data related to the employee information of an enterprise, the primary keyword key can be the enterprise name ABC, and the secondary keyword can be the employee name XYZ.
[0068] Step S540: Assign a value to the coding type identifier bit located at the end of the coding template according to the pre-set record identifier. The coding type identifier bit is used to indicate that the coding type of the current coding is a record coding or an index coding.
[0069] Assigning a value to the coding type identifier bit according to the pre-set record identifier can be used to determine the coding type of the current coding. For example, the pre-set record identifier is 0, and in the embodiment of the present application, the coding type of the current coding is a record coding, so the coding type identifier bit can be assigned a value of 0. In some other embodiments, if the coding type of the current coding is an index coding, the coding type identifier bit can be assigned a non-zero character according to the pre-set index identifier accordingly.
[0070] In some alternative embodiments, the embodiment of the present application can also solidify the pre-set data type identifier and the pre-set record identifier or index identifier in the coding template, rather than re-assigning values to the storage type identifier bit and the coding type identifier bit every time a record coding or an index coding is generated.
[0071] Figure 6 Shows the method step flow chart for generating a record coding based on the second coding template in an embodiment of the present application. As Figure 6As shown, the record code generated based on the second coding template may include a storage type identification bit, a storage area identification bit, a keyword identification bit, a coding type identification bit, and a secondary keyword identification bit arranged in sequence from front to back. Corresponding data objects may be stored after the secondary keyword identification bit. Compared with Figure 5 the first coding template shown, adding a secondary keyword identification bit can increase the processing dimension of service data and meet the processing requirements of users for complex service data. In some alternative embodiments, more levels of secondary keyword identification bits may also be appended after the coding type identification bit according to system configuration, so as to meet the data processing requirements of more dimensions.
[0072] In an embodiment of the present application, the method for assigning identification bits to the Figure 6 second coding template shown may include the following steps S610 to S650.
[0073] Step S610: Perform an assignment process on the storage type identification bit located at the head of the coding template according to a preset data type identifier. The storage type identification bit is used to indicate that the storage type of the current service data is tabular data or non-tabular data.
[0074] Performing an assignment process on the storage type identification bit according to a preset data type identifier can determine whether the storage type of the current service data is tabular data or non-tabular data when the storage system initially parses the record code, so as to determine the subsequent coding parsing format. For example, if the storage type of the current service data is tabular data, the storage type identification bit may be assigned a value of 1; if the storage type of the current service data is non-tabular data, the storage type identification bit may be assigned a value of 0.
[0075] Step S620: Perform an assignment process on the storage area identification bit in the coding template according to a table identifier. The storage area identification bit is used to indicate the service data table where the service data is located.
[0076] The storage area identification bit can be assigned according to the table identifier obtained from the data filtering information. The service data table where the corresponding data object is located can be determined according to the value of the storage area identification bit. The table identifier may be a pre-set digital code table_id corresponding to each service data table one by one. For example, for a certain service data table, the storage area identification bit may be assigned a value of 1 according to the table identifier.
[0077] Step S630: Perform an assignment process on the keyword identification bit adjacent to the storage area identification bit in the coding template according to a keyword. The keyword identification bit is used to indicate the primary keyword associated with the service data configured when storing the service data.
[0078] Based on the keywords obtained from the data filtering information, the keyword identification bits can be assigned. According to the values of the keyword identification bits, the primary keywords associated with the business data configured during the storage of the corresponding data objects can be determined. For example, for the business data related to the employee information of an enterprise, the primary keyword key can be the enterprise name ABC.
[0079] Step S640: Assign values to the coding type identification bits adjacent to the keyword identification bits according to the preset record identifier. The coding type identification bits are used to indicate that the coding type of the current coding is record coding or index coding.
[0080] Assigning values to the coding type identification bits according to the preset record identifier can be used to determine the coding type of the current coding. For example, the preset record identifier is 0. In the embodiments of the present application, the coding type of the current coding is record coding, so the coding type identification bits can be assigned the value 0. In some other embodiments, if the coding type of the current coding is index coding, the coding type identification bits can be assigned non-zero characters according to the preset index identifier accordingly.
[0081] Step S650: Assign values to the secondary keyword identification bits in the coding template according to the secondary keywords. The secondary keyword identification bits are used to indicate the secondary keywords associated with the business data configured during the storage of the business data.
[0082] In the embodiments of the present application, the secondary keywords associated with the business data can be obtained in advance, and then the secondary keyword identification bits in the coding template can be assigned values according to the secondary keywords to determine the secondary keywords as the second data query dimension.
[0083] In some embodiments of the present application, the types of the keywords in the data filtering information can be identified to obtain the business types of each keyword; then the keywords can be classified into primary keywords and secondary keywords associated with the business data according to the business types. For example, different levels of business types can be divided for the keywords according to the specific business content. The keywords corresponding to the high-level business types can be classified as primary keywords, while the keywords corresponding to the low-level business types can be classified as secondary keywords. For example, for the business data related to the employee information of an enterprise, multiple levels of business sequences such as enterprise name, department name, and employee name can be divided in descending order. When the keywords in the data filtering information include both the enterprise name and the employee name, the enterprise name corresponding to the high-level business type can be determined as the primary keyword, and the employee name corresponding to the low-level business type can be determined as the secondary keyword.
[0084] In some embodiments of the present application, when a user performs an operation on a business data form to generate a business data query request, the user can also determine the primary keyword and secondary keyword specified by the user based on the configuration of the business type in the business system.
[0085] In some embodiments of the present application, when a user performs an operation on a business data form to generate a business data query request, the user can also specify an index field by delimiting a query range. The storage system can obtain the index field of the business data to be queried from the data filtering information, assign values to the identification bits of the coding template according to the index field, generate an index code, use the index code as a keyword to query the secondary keyword used as a secondary index in the storage system, and after generating a record code by assigning values to the identification bits in the coding template using the secondary keyword, continue to query the corresponding data object according to the record code.
[0086] Figure 7 The flowchart of the method steps for generating an index code based on the third coding template in an embodiment of the present application is shown. As Figure 7 shown, the index code generated based on the third coding template may include a storage type identification bit, a storage area identification bit, a keyword identification bit, a coding type identification bit, and an index field identification bit arranged in sequence from front to back. The corresponding secondary keyword may be stored after the index field identification bit.
[0087] In an embodiment of the present application, for Figure 7 the method of assigning values to the identification bits of the third coding template shown may include the following steps S710 to step S750.
[0088] Step S710: Assign values to the storage type identification bit located at the head of the coding template according to a preset data type identifier. The storage type identification bit is used to indicate that the storage type of the current business data is tabular data or non-tabular data.
[0089] Assigning values to the storage type identification bit according to a preset data type identifier can determine whether the storage type of the current business data is tabular data or non-tabular data when the storage system initially parses the record code, so as to determine the subsequent coding parsing format. For example, if the storage type of the current business data is tabular data, the storage type identification bit can be assigned a value of 1; if the storage type of the current business data is non-tabular data, the storage type identification bit can be assigned a value of 0.
[0090] Step S720: Assign values to the storage area identification bit in the coding template according to the table identifier. The storage area identification bit is used to indicate the business data table where the business data is located.
[0091] The storage area identification bit can be assigned according to the table identification obtained from the data filtering information. According to the value of the storage area identification bit, the business data table where the corresponding data object is located can be determined. The table identification can be a pre-set digital code table_id that corresponds one-to-one with each business data table. For example, for a certain business data table, the storage area identification bit can be assigned the value of 1 according to the table identification.
[0092] Step S730: Assign values to the keyword identification bits adjacent to the storage area identification bit in the coding template according to the keyword. The keyword identification bits are used to indicate the primary keyword associated with the business data configured when storing the business data.
[0093] The keyword identification bits can be assigned values according to the keyword obtained from the data filtering information. According to the value of the keyword identification bits, the primary keyword associated with the business data configured when the corresponding data object is stored can be determined. For example, for business data related to the employee information of an enterprise, the primary keyword key can be the enterprise name ABC.
[0094] Step S740: Assign values to the coding type identification bits in the coding template according to the preset index identifier. The coding type identification bits are used to indicate that the coding type of the current coding is record coding or index coding.
[0095] Assigning values to the coding type identification bits according to the preset index identifier can be used to determine the coding type of the current coding. For example, in the embodiment of the present application, the coding type of the current coding is index coding, and the coding type identification bits can be assigned non-zero characters according to the preset index identifier, such as index_id with different coding values.
[0096] Step S750: Assign values to the index field identification bits in the coding template according to the index field. The index field identification bits are used to indicate the index field configured when storing the business data.
[0097] The index field identification bits can be assigned values according to the index field obtained from the data filtering information. According to the value of the index field identification bits, the secondary index configured by the storage system can be used to narrow the query range of the corresponding data object. For example, if the primary keyword is the company name ABC, the index field can be the IT department within the company.
[0098] Based on the index coding generated after the assignment processing in the embodiment of the present application, a secondary keyword whose index prefix matches the index coding obtained from the assignment processing can be queried in the storage system. Based on the primary keyword in the data filtering information and the secondary keyword obtained by querying in the embodiment of the present application, Figure 5Perform an assignment process on the second encoding template shown to generate corresponding record encodings, and then query the corresponding data objects in the storage system according to the record encodings.
[0099] In step S330, query the target business data in the storage system whose data prefix matches the record encoding, and return the target business data to the data requester.
[0100] When the business system queries all the data corresponding to the keyword key through the keyword key, the storage system can convert it into all value data whose query data prefix matches the record encoding "1(table_type):1(table_id):1(key):0(record)", and encapsulate it into the form of a record list RecordList and return it to the business system.
[0101] When the business system hopes to query some internal data of a certain primary keyword key through the secondary keyword sub_key, the storage system only needs to find the data whose data prefix matches the record encoding "1(table_type):1(table_id):1(key):0(record):2(sub_key)".
[0102] If the business system queries the data in a certain keyword key that meets the index field index_field as abc, the storage system only needs to query all the secondary keywords sub_key that meet the data prefix with the index encoding "1(table_type):1(table_id):1(key):1(index_id):abc(index_field)", and then query all the data that meet these conditions through the record encoding "1(table_type):1(table_id):1(key):0(record):3(sub_key)".
[0103] Figure 8 Shows a schematic structural diagram of the storage system in an embodiment of the present application. As Figure 8As shown, the storage system may include a readable and writable file 801 (MemTable) located in the memory, a read-only file 802 (Immutable MemTable) located in the memory, and an ordered string table 803 (SortedString Table, SSTable) with a layered structure located on the disk. The readable and writable file 801 can normally accept data write requests and data read requests. The read-only file 802 only accepts data read requests and does not allow modification of the file content. The ordered string table 803 is a data file sorted in an orderly manner using the sequential read and write functions of the disk. The most recently generated ordered string table is located at the top layer. As new data is continuously written, the original ordered string tables will be merged layer by layer downward in the order of generation time.
[0104] When writing data to the storage system, the file can be first written to the readable and writable file located in the memory. When the data stored in the readable and writable file reaches the preset data volume threshold, the readable and writable file can be converted into a read-only file, and a new readable and writable file can be generated to continue receiving data writes.
[0105] After generating a new readable and writable file, an operation of writing to the disk can be performed on the read-only file, so as to convert the data stored in the file as a whole into an ordered string table located at the top layer on the disk. Each level of the ordered string table has a specified data volume threshold. When the overall data volume of the ordered string table at a certain level reaches the data volume threshold of that level, one of the ordered string tables can be selected for merging downward, and so on.
[0106] Similar to the data writing process, when the storage system receives a data query request, it can first query the target business data in the readable and writable file whose data prefix matches the record encoding. If the query is successful, a data read operation can be performed to return the target business data to the business system. If the query fails in the readable and writable file, the target business data that matches the record encoding is continued to be queried in the read-only file. If the query still fails, the target business data that matches the record encoding is continued to be queried level by level in the ordered string table.
[0107] When the target business data is queried in the storage system, a data read operation can be performed on the queried target business data. The target business data after the read operation is completed is encapsulated into a record list composed of data records, and the record list is returned to the data requester.
[0108] In one embodiment of the present application, when reading data through the current data processing thread, locking processing can be performed on the target business data to reject data access requests from other business data processing threads for the target business data; after the current data processing thread returns the read target business data to the data requester, unlocking processing can be performed on the target business data.
[0109] The database locking mechanism is a rule designed by the database to ensure data consistency and make various shared resources orderly when accessed concurrently. In one embodiment of the present application, the locking processing performed on the target business data can be controlled at the dimension of the primary key. Specifically, when performing a data reading operation on a certain part of the target business data, the primary key corresponding to this part of the target business data can be obtained, and then a shared lock can be added to the data object corresponding to the primary key in the storage system. The data object after adding the shared lock can continue to be read by other data processing threads, but cannot be written and modified. In some other embodiments, if the current data processing thread is performing a data writing operation, then an exclusive lock can be added to the corresponding target business data. The data object after adding the exclusive lock will reject data reading requests or data writing requests from other data processing threads to prevent data inconsistency problems.
[0110] Figure 9 The principle diagram of data locking at the primary key dimension in one embodiment of the present application is shown. As Figure 9 shown, for each data processing thread, the corresponding primary keys Key1, Key2, Key3... can be obtained according to the target business data it actually processes, and the corresponding data locks lock1, lock2, lock3... will be added to the data objects corresponding to the primary keys. For the data objects with data locks added. The embodiment of the present application performs data concurrency control at the granularity of the primary key Key during read and write operations, and ensures isolation among multiple requests when executed concurrently, ensuring the overall data correctness, isolation, and consistency, and finally achieving a serializable isolation level. At the same time, since the data set stored in the business system usually has a large number of keywords and the access to each keyword is scattered, the storage system can maintain high concurrency to serve the business, thus achieving high throughput performance of the overall storage system.
[0111] It should be noted that although the steps of the method in the present application are described in a specific order in the drawings, this does not require or imply that these steps must be executed in this specific order, or that all the steps shown must be executed to achieve the desired result. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step for execution, and / or one step can be decomposed into multiple steps for execution, etc.
[0112] The following introduces the device embodiments of the present application, which can be used to execute the data processing method in the above embodiments of the present application. Figure 10 The structural block diagram of the data processing device provided by the embodiments of the present application is schematically shown. As Figure 10 shown, the data processing device 1000 mainly may include: an information acquisition module 1010, configured to acquire data filtering information for data query from a business data query request based on a business data table, where the data filtering information includes a table identifier of the business data table and a keyword corresponding to the business data to be queried; a record encoding module 1020, configured to perform structured arrangement on the data filtering information to obtain a record encoding corresponding to the business data to be queried; and a data query module 1030, configured to query target business data in a storage system whose data prefix matches the record encoding, and return the target business data to the data requester.
[0113] In some embodiments of the present application, based on the above embodiments, the record encoding module 1020 includes: a template acquisition unit, configured to acquire an encoding template obtained by performing structured arrangement on a plurality of identification bits according to a preset encoding format; and a template assignment unit, configured to perform assignment processing on each identification bit in the encoding template according to the table identifier and the keyword to obtain a record encoding of the business data.
[0114] In some embodiments of the present application, based on the above embodiments, the template assignment unit includes: a first assignment subunit, configured to perform assignment processing on a storage area identification bit in the encoding template according to the table identifier, where the storage area identification bit is used to indicate the business data table where the business data is located; and a second assignment subunit, configured to perform assignment processing on a keyword identification bit adjacent to the storage area identification bit in the encoding template according to the keyword, where the keyword identification bit is used to indicate a primary keyword associated with the business data configured when storing the business data.
[0115] In some embodiments of the present application, based on the above embodiments, the template assignment unit further includes: a third assignment subunit, configured to perform assignment processing on a storage type identification bit located at the head of the encoding template according to a preset data type identifier, where the storage type identification bit is used to indicate that the storage type of the current business data is table data or non-table data; and a fourth assignment subunit, configured to perform assignment processing on a coding type identification bit located at the tail of the encoding template according to a preset record identifier, where the coding type identification bit is used to indicate that the coding type of the current coding is a record coding or an index coding.
[0116] In some embodiments of the present application, based on the above embodiments, the template assignment unit further includes: a secondary keyword acquisition subunit, configured to acquire secondary keywords associated with business data; a fifth assignment subunit, configured to perform an assignment process on the secondary keyword identification bits in the encoding template according to the secondary keywords, where the secondary keyword identification bits are used to indicate the secondary keywords associated with the business data configured when storing the business data.
[0117] In some embodiments of the present application, based on the above embodiments, the secondary keyword acquisition subunit includes: a type recognition subunit, configured to perform type recognition on the keywords in the data filtering information to obtain the business type of each keyword; a keyword classification subunit, configured to classify the keywords into primary keywords and secondary keywords associated with business data according to the business type.
[0118] In some embodiments of the present application, based on the above embodiments, the secondary keyword acquisition subunit includes: an index field acquisition subunit, configured to acquire the index field of the business data to be queried from the data filtering information; a sixth assignment subunit, configured to perform an assignment process on the index field identification bits in the encoding template according to the index field, where the index field identification bits are used to indicate the index field configured when storing the business data; a seventh assignment subunit, configured to perform an assignment process on the encoding type identification bits in the encoding template according to a preset index identifier, where the encoding type identification bits are used to indicate that the current encoding type is record encoding or index encoding; a secondary keyword query subunit, configured to query for secondary keywords in the storage system whose index prefix matches the index encoding obtained by the assignment process.
[0119] In some embodiments of the present application, based on the above embodiments, the information acquisition module 1010 includes: a request generation unit, configured to acquire the table operation information executed by the data requester on the business data table in the business system to generate a business data query request according to the table operation information; an interface call unit, configured to call the object-relational mapping interface configured between the business system and the storage system; a mapping conversion unit, configured to perform mapping conversion on the business data query request through the object-relational mapping interface to obtain data filtering information for data query on the storage system.
[0120] In some embodiments of the present application, based on the above embodiments, the storage system includes a readable and writable file located in the memory, a read-only file located in the memory, and an ordered string table with a layered structure located on the disk; the data query module 1030 includes: a first query unit configured to query target service data in the readable and writable file whose data prefix matches the record code; a second query unit configured to, if the query fails in the readable and writable file, query the target service data that matches the record code in the read-only file; a third query unit configured to, if the query fails in the read-only file, query the target service data that matches the record code in the ordered string table level by level.
[0121] In some embodiments of the present application, based on the above embodiments, the data query module 1030 further includes: a data reading unit configured to perform a data reading operation on the queried target service data; a data locking unit configured to perform a locking process on the target service data so that the target service data rejects data access requests from other service data processing threads; a data returning unit configured to return the read target service data to the data requester and perform an unlocking process on the target service data.
[0122] In some embodiments of the present application, based on the above embodiments, the data locking unit includes: a primary key obtaining subunit configured to obtain a primary key corresponding to the target service data; a primary key locking subunit configured to add a shared lock to the data object corresponding to the primary key in the storage system.
[0123] In some embodiments of the present application, based on the above embodiments, the data returning unit includes: a record list returning subunit configured to encapsulate the read target service data into a record list composed of data records and return the record list to the data requester.
[0124] The specific details of the data processing device provided in each embodiment of the present application have been described in detail in the corresponding method embodiments, and will not be repeated here.
[0125] Figure 11 Schematically shown is a block diagram of a computer system of an electronic device for implementing the embodiments of the present application.
[0126] It should be noted that Figure 11 The computer system 1100 of the electronic device shown is only an example and should not bring any limitations to the functions and usage scope of the embodiments of the present application.
[0127] As Figure 11As shown, the computer system 1100 includes a central processing unit 1101 (CPU), which can perform various appropriate actions and processes according to the program stored in the read-only memory 1102 (ROM) or the program loaded from the storage section 1108 into the random access memory 1103 (RAM). In the random access memory 1103, various programs and data required for system operation are also stored. The central processing unit 1101, the read-only memory 1102, and the random access memory 1103 are connected to each other via a bus 1104. The input / output interface 1105 (Input / Output interface, i.e., I / O interface) is also connected to the bus 1104.
[0128] The following components are connected to the input / output interface 1105: an input section 1106 including a keyboard, a mouse, etc.; an output section 1107 including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section 1108 including a hard disk, etc.; and a communication section 1109 including a network interface card such as a local area network card, a modem, etc. The communication section 1109 performs communication processing via a network such as the Internet. A drive 1110 is also connected to the input / output interface 1105 as needed. A removable medium 1111, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1110 as needed so that a computer program read from it can be installed into the storage section 1108 as needed.
[0129] In particular, according to an embodiment of the present application, the processes described in each method flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from the network via the communication section 1109, and / or installed from the removable medium 1111. When the computer program is executed by the central processing unit 1101, various functions defined in the system of the present application are executed.
[0130] It should be noted that the computer-readable medium shown in the embodiments of the present application can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device. In the present application, the computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, and this computer-readable medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted by any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.
[0131] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of code, and the above module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks can occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.
[0132] It should be noted that although several modules or units of a device for action execution are mentioned in the above detailed description, such a division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of the two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.
[0133] Through the description of the above embodiments, those skilled in the art can easily understand that the example embodiments described herein can be implemented by software or by a combination of software and necessary hardware. Therefore, the technical solutions according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (such as a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiments of the present application.
[0134] After considering the specification and practicing the invention disclosed herein, those skilled in the art will readily conceive of other embodiments of the present application. The present application is intended to cover any variations, uses, or adaptations of the present application, which follow the general principles of the present application and include well-known knowledge or conventional technical means in the technical field not disclosed in the present application.
[0135] It should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present application is only limited by the appended claims.
Claims
1. A data processing method, characterized in that, Including: Obtain data filtering information for data query from a business data query request based on a business data table, where the data filtering information includes a table identifier of the business data table and keywords corresponding to the business data to be queried; Obtain a coding template obtained by structurally arranging multiple identification bits according to a preset coding format; Perform assignment processing on each identification bit in the coding template according to the table identifier and the keywords to obtain a record code of the business data; Query target business data in the storage system whose data prefix matches the record code, and return the target business data to the data requester; Among them, the performing assignment processing on each identification bit in the coding template according to the table identifier and the keywords includes: performing assignment processing on the storage area identification bit in the coding template according to the table identifier, where the storage area identification bit is used to indicate the business data table where the business data is located; performing assignment processing on the keyword identification bit adjacent to the storage area identification bit in the coding template according to the keywords, where the keyword identification bit is used to indicate the primary keyword associated with the business data configured when storing the business data.
2. The data processing method according to claim 1, wherein The performing assignment processing on the identification bits in the coding template according to the table identifier and the keywords further includes: Performing assignment processing on the storage type identification bit located at the head of the coding template according to a preset data type identifier, where the storage type identification bit is used to indicate that the storage type of the current business data is table data or non-table data; Performing assignment processing on the coding type identification bit located at the tail of the coding template according to a preset record identifier, where the coding type identification bit is used to indicate that the coding type of the current coding is a record code or an index code.
3. The data processing method according to claim 1, wherein The performing assignment processing on the identification bits in the coding template according to the table identifier and the keywords further includes: Obtain secondary keywords associated with the business data; Perform assignment processing on the secondary keyword identification bit in the coding template according to the secondary keywords, where the secondary keyword identification bit is used to indicate the secondary keyword associated with the business data configured when storing the business data.
4. The data processing method according to claim 3, wherein The obtaining secondary keywords associated with the business data includes: Perform type identification on the keywords in the data filtering information to obtain the business type of each keyword; Classify the keywords into primary keywords and secondary keywords associated with the business data according to the business type.
5. The data processing method according to claim 3, wherein The obtaining secondary keywords associated with the business data includes: Obtain the index field of the business data to be queried from the data filtering information; Perform assignment processing on the index field identification bit in the coding template according to the index field, where the index field identification bit is used to indicate the index field configured when storing the business data; Perform assignment processing on the coding type identification bit in the coding template according to a preset index identifier, where the coding type identification bit is used to indicate that the coding type of the current coding is a record code or an index code; The storage system is searched for a secondary keyword whose index prefix matches the index code obtained by the assignment process.
6. The data processing method according to claim 1, characterized in that, The step of obtaining data filtering information for performing data query from a business data query request based on a business data table includes: Acquiring table operation information performed by the data requester on the business data table on the business system, so as to generate a business data query request according to the table operation information; Calling an object relational mapping interface configured between the business system and the storage system; The business data query request is mapped and converted through the object relationship mapping interface to obtain data filtering information for performing data query on the storage system.
7. The data processing method according to claim 1, wherein The storage system includes a readable and writable file located in the memory, a read-only file located in the memory, and an ordered string table with a layered structure located in the disk; Querying the storage system for target business data whose data prefix matches the record code includes: Searching the readable and writable file for target business data whose data prefix matches the record code; If the query in the readable and writable file fails, then the target business data matching the record code is searched in the read-only file; If the search in the read-only file fails, the target business data matching the record code is searched level by level in the ordered string table.
8. The data processing method according to claim 1, wherein The returning the target service data to the data requesting party includes: Perform data reading operations on the target business data obtained by the query; Locking the target business data so that the target business data rejects data access requests from other business data processing threads; The read target business data is returned to the data requester, and the target business data is unlocked.
9. The data processing method according to claim 8, wherein The locking process of the target business data includes: Obtaining a primary keyword corresponding to the target business data; A shared lock is added to the data object corresponding to the primary key in the storage system.
10. The data processing method according to claim 8, wherein The step of returning the read target service data to the data requester includes: The read target business data is encapsulated into a record list consisting of data records, and the record list is returned to the data requester.
11. A data processing device, characterized in that, include: An information acquisition module is configured to acquire data filtering information for data query from a business data query request based on a business data table, wherein the data filtering information includes a table identifier of the business data table and keywords corresponding to the business data to be queried; A record encoding module is configured to obtain an encoding template obtained by structurally arranging a plurality of identification bits according to a preset encoding format; Assigning values to each identification bit in the coding template according to the table identification and the keyword to obtain a record code of the business data; A data query module is configured to query the storage system for target business data whose data prefix matches the record code, and return the target business data to the data requester; Among them, the process of assigning values to each identification bit in the encoding template according to the table identification and the keyword includes: assigning a value to the storage area identification bit in the encoding template according to the table identification, where the storage area identification bit is used to indicate the service data table where the service data is located; assigning a value to the keyword identification bit adjacent to the storage area identification bit in the encoding template according to the keyword, where the keyword identification bit is used to indicate the primary keyword associated with the service data configured when storing the service data.
12. A computer-readable medium having a computer program stored thereon, where the computer program, when executed by a processor, implements the data processing method according to any one of claims 1 to 10.
13. An electronic device, characterized in that, Comprising: A processor; And A memory for storing executable instructions of the processor; Among them, the processor is configured to execute the data processing method according to any one of claims 1 to 10 by executing the executable instructions.
14. A computer program product comprising computer instructions, characterized in that, The computer instructions, when executed by the processor, implement the data processing method according to any one of claims 1 to 10.
Citation Information
Patent Citations
Data query method and device based on configuration information
CN110019350A