Data processing method and device
By processing data in the key-value database in parallel, the performance bottleneck of the Jedis client in a high-concurrency environment is resolved, efficient data processing is achieved, and application performance and user experience are improved.
Patent Information
- Application Number
- CN202410829375.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-25
- Publication Date
- 2025-09-09
- Estimated Expiration
- 2044-06-25
AI Technical Summary
The existing Jedis client library has performance bottlenecks in high-concurrency environments, cannot implement non-blocking and asynchronous operations, and requires additional configuration of the connection pool, which increases development complexity.
Provided are a data processing method and device, which processes key-value pair data in a key-value database in parallel by receiving data processing instructions from the client, and utilizes the multi-core performance advantages of the HetuKV key-value database to achieve efficient parallel data processing, improve application performance and reduce operation latency.
It enables efficient client connection to the key-value database, improves data processing efficiency, reduces operation delays and costs, and enhances user experience.
Smart Images

Figure CN118796919B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing technology, and in particular to a data processing method and device. Background Art
[0002] NoSQL databases are a type of database that emerged outside of relational databases, aiming to address challenges such as large-scale data storage and high-concurrency access that traditional relational databases struggle to address. Key-value databases are a major type of NoSQL database. They store data in the form of key-value pairs, suitable for simple data models and fast read and write operations. Redis, in particular, is an in-memory key-value database and is widely used due to its fast read and write operations and rich features such as publish / subscribe, transactions, and pipelines.
[0003] In the Java development world, with the rapid growth of internet applications, the demand for high performance, scalability, and flexibility is growing. Traditional relational databases can face performance bottlenecks when handling large amounts of data and high concurrency, leading developers to seek alternatives. The emergence of key-value databases fills this gap and provides Java developers with more options. Java is a widely used programming language used in enterprise, web, mobile, and embedded systems. Many enterprise applications and microservices architectures are based on the Java technology stack. Developing a Java interface for key-value databases enables these applications to easily access and operate key-value databases, meeting the needs of specific application scenarios.
[0004] Key-value databases are typically characterized by high performance and scalability because they are based on a simple key-value data model that allows for fast read and write operations. Through Java interfaces, developers can leverage the performance advantages of key-value databases for applications requiring caching, session management, real-time data processing, and more. Developing Java interfaces for key-value databases simplifies the development process by abstracting and encapsulating database operations. Developers can interact with key-value databases using familiar Java APIs without having to delve into the database's underlying implementation. This simplification helps improve development efficiency and reduce the likelihood of errors.
[0005] Jedis is a Java client library for Redis that provides comprehensive support and direct mapping of Redis commands. Jedis encapsulates Redis commands, including strings, hashes, lists, lists, ordered lists, transactions, and publish / subscribe. It also provides an API that allows developers to interact with Redis using native commands.
[0006] Jedis runs synchronously, meaning it blocks while waiting for Redis responses. This can lead to performance bottlenecks, especially in highly concurrent environments. Because Jedis is synchronous, it's not suitable for scenarios requiring non-blocking, asynchronous operations. Jedis doesn't include a built-in connection pool, requiring additional configuration to manage connections. This can increase development complexity. Therefore, a method that enables Java clients to efficiently connect to a key-value database in highly concurrent environments is urgently needed. Summary of the Invention
[0007] The present invention provides a data processing method and device, which can use preset data processing instructions to enable a client to efficiently connect to a key-value database, and can perform parallel data processing on key-value data in the key-value database, thereby realizing efficient parallel data processing, thereby improving the performance of the application, improving the performance of the client, and reducing operation delays, thereby improving the efficiency of data processing, reducing data processing costs, and improving the user experience of data processing.
[0008] In a first aspect, the present invention provides a data processing method, which is applied to a solid-state drive corresponding to a key-value database, and includes:
[0009] Receive at least one data processing instruction sent by a client; wherein each data processing instruction includes: target list type data and target list operation type;
[0010] performing parallel data processing on the at least one data processing instruction to obtain data processing results corresponding to each of the at least one data processing instructions; wherein, for each data processing instruction, determining target key-value pair data in the key-value database based on the target list type data, wherein each key-value pair data in the key-value database includes a key and at least one value, each value includes at least one data list, and each data list includes at least one element, wherein each element is a string; performing data processing corresponding to the target list operation type on the target key-value pair data to obtain a data processing result corresponding to the data processing instruction;
[0011] Feedback the data processing results corresponding to each of the at least one data processing instructions to the client.
[0012] In a second aspect, the present invention provides a data processing device, which is applied to a solid-state hard disk corresponding to a key-value database, and includes:
[0013] The first unit is configured to receive at least one data processing instruction sent by a client; wherein each data processing instruction includes: target list type data and target list operation type;
[0014] The second unit is configured to perform parallel data processing on the at least one data processing instruction to obtain data processing results corresponding to each of the at least one data processing instructions; wherein, for each data processing instruction, target key-value pair data in the key-value database is determined according to the target list type data, wherein each key-value pair data in the key-value database includes a key and at least one value, each value includes at least one data list, and each data list includes at least one element, wherein each element is a string; data processing corresponding to the target list operation type is performed on the target key-value pair data to obtain a data processing result corresponding to the data processing instruction;
[0015] The third unit is configured to feed back the data processing results corresponding to each of the at least one data processing instructions to the client.
[0016] In a third aspect, the present invention provides a readable medium comprising an execution instruction. When a processor of an electronic device executes the execution instruction, the electronic device executes any method described in the first aspect.
[0017] In a fourth aspect, the present invention provides an electronic device comprising a processor and a memory storing execution instructions, wherein when the processor executes the execution instructions stored in the memory, the processor executes any method described in the first aspect.
[0018] It can be seen from the above technical solution that the method provided by the present invention can be applied to a solid-state hard disk corresponding to a key-value database, and the method includes: receiving at least one data processing instruction sent by a client; wherein each data processing instruction includes: target list type data, target list operation type; performing parallel data processing on the at least one data processing instruction to obtain a data processing result corresponding to each of the at least one data processing instructions; wherein, for each data processing instruction, the target key-value pair data in the key-value database is determined according to the target list type data, wherein each key-value pair data in the key-value database includes a key and at least one value, each value includes at least one data list, and each data list includes at least one element, wherein each element is a string; performing data processing corresponding to the target list operation type on the target key-value pair data to obtain a data processing result corresponding to the data processing instruction; and feeding back the data processing result corresponding to each of the at least one data processing instructions to the client. Since in the present application, the key-value pair data in the key-value database can be processed in parallel according to at least one data processing instruction, and the data processing results corresponding to each of the at least one data processing instructions can be obtained, and the data processing results corresponding to each of the at least one data processing instructions can be fed back to the client, therefore, the present application can use the preset data processing instructions to enable the client to efficiently connect to the key-value database, and can perform parallel data processing on the key-value pair data in the key-value database, thereby realizing efficient parallel data processing, thereby improving the performance of the application, improving the performance of the client, reducing operation delays, and thus achieving improved data processing efficiency, reduced data processing costs, and thus improved user experience of data processing.
[0019] The further effects of the above-mentioned non-conventional preferred embodiment will be described below in conjunction with specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] In order to more clearly illustrate the embodiments of the present invention or the existing technical solutions, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0021] Figure 1 A flowchart of a data processing method provided by one embodiment of the present invention;
[0022] Figure 2 A flowchart of a data processing method provided by one embodiment of the present invention;
[0023] Figure 3 A flowchart of a data processing method provided by one embodiment of the present invention;
[0024] Figure 4 A flowchart of a data processing method provided by one embodiment of the present invention;
[0025] Figure 5 A flowchart of a data processing method provided by one embodiment of the present invention;
[0026] Figure 6 A flowchart of a data processing method provided by one embodiment of the present invention;
[0027] Figure 7 A flowchart of a data processing method provided by one embodiment of the present invention;
[0028] Figure 8 A flowchart of a data processing method provided by one embodiment of the present invention;
[0029] Figure 9 A schematic structural diagram of a data processing device provided in one embodiment of the present invention;
[0030] Figure 10 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0031] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.
[0032] Various non-limiting embodiments of the present invention are described in detail below with reference to the accompanying drawings.
[0033] See also Figure 1 , shows a data processing method in an embodiment of the present invention, which can be applied to a solid-state drive corresponding to a key-value database. In this embodiment, the method may include the following steps:
[0034] Step 1: Receive at least one data processing instruction sent by a client.
[0035] In this embodiment, when a client needs to process key-value pairs in a key-value database using a solid-state drive, the client can send at least one data processing instruction to the solid-state drive corresponding to the key-value database. In one implementation, the client can be a Java client, and the key-value database can be a HetuKV key-value database.
[0036] Among them, each data processing instruction may include: target list type data, target list operation type. Target list type data can be understood as relevant information about the data list and / or elements in the data list that need to be processed, for example, the key and / or value of the key-value pair to be processed. It should be noted that each key-value pair data in the key-value database includes a key and at least one value, each value includes at least one data list, and each data list includes at least one element, wherein each element is a string. The target list operation type can be understood as the operation method that needs to be performed on the key-value pair to be processed, for example, query, delete, add, etc.
[0037] It should be noted that the key-value data in the key-value database can be relevant data in the fields of artificial intelligence, content recommendation, image rendering, financial risk control, Internet of Things, game development, big data analysis, etc. For example, in the field of artificial intelligence, the solution provided by this application can be used to train and deploy machine learning and deep learning models. The application method is: to read and process large amounts of training data, quickly pre-process data, cache intermediate results, and store trained models in a high-speed cache for real-time reasoning and prediction. In the field of content recommendation systems, content recommendation systems usually need to quickly access and process large amounts of user and content data to generate personalized recommendations. The solution provided by this application can be used to cache user preferences, recommendation history, popular content, etc. to reduce the reading pressure of the database and improve the real-time and accuracy of the recommendation system. In the field of image rendering and processing, fast access and caching of intermediate results are crucial to performance. The solution provided by this application can be used to store and cache intermediate data generated during the rendering process, such as textures, image slices, rendering results, etc., to help speed up the rendering process and reduce computing load. Financial institutions need to quickly process and analyze large amounts of transaction and user data to detect fraud and Managing risks, the solution provided in this application can be used to cache and quickly access transaction records, account information, risk assessment results, etc., thereby improving the real-time response capability and data processing performance of the risk control system; in the field of Internet of Things (IoT), IoT devices generate a large amount of data, and rapid processing and analysis of this data are crucial to system stability and performance. The solution provided in this application can be used to store and cache sensor data, device status, real-time events, etc., to help the IoT system quickly respond to events and instructions and ensure the real-time performance of the system; game development requires processing a large amount of real-time data, including game status, player information and game events. The solution provided in this application can be used to cache game status, player data, rankings, etc. to improve the response speed and performance of the game; big data analysis requires rapid processing and analysis of large amounts of data to obtain valuable insights. The solution provided in this application can be used as a data processing transfer station to help achieve rapid import, processing and caching of data, thereby improving the performance of the analysis system.
[0038] Step 2: Perform parallel data processing on the at least one data processing instruction to obtain data processing results corresponding to each of the at least one data processing instructions.
[0039] In this embodiment, if multiple data processing instructions are received at the same time, the multiple data processing instructions can be processed in parallel to obtain data processing results corresponding to each of the at least one data processing instructions. For example, when the client can be a Java client and the key-value database can be a HetuKV key-value database, the multi-core performance advantage of the HetuKV key-value database can be utilized to achieve efficient parallel data processing, thereby improving the performance of the application. The HetuKV key-value database is multi-core parallel, and the Java client supports and can implement multi-threading, so multiple cores of the CPU can be enabled, and each core processes a data processing instruction separately. For example, when storing a large String file in a multi-core environment, the file can be split into multiple parts, and each core stores a part of the data separately, and can be processed simultaneously without interfering with each other. This multi-core parallel processing method can greatly improve the performance of data processing.
[0040] Specifically, for each data processing instruction, the target key-value pair data in the key-value database can be determined based on the target list type data, that is, the target key-value pair data corresponding to the target list type data can be queried from the key-value database based on the target list type data. Wherein, each key-value pair data in the key-value database includes a key and at least one value, and each value includes at least one data list, and each data list includes at least one element, wherein each element is a string. It can be understood that in this embodiment, the value corresponding to the key in each key-value pair data is at least one data list (i.e., list), that is, using key-value storage to store a hashset (hash data list) can be understood as storing a data list (i.e., list). Wherein, the key is the unique identifier of this data list, and the value is this data list (i.e., list). There are multiple elements in the data list (i.e., set). Simply put, the key is the name of a data list, and the value corresponding to this key is the element (members) in this data list. For example, the data list is [s, e, r, t]. For example, the relationship between keys and values can be compared to an ID number and a person. A list is a specific type of value, such as "woman," and an element can be understood as all the information about that person. In a key-value store like a list, the key is the unique identifier of the list, and the element is the value in the list. For example, a list is [a, b, c, d, e], where a is an element in the list.
[0041] Then, the target key-value pair data can be processed according to the target list operation type to obtain a data processing result corresponding to the data processing instruction. In other words, the target key-value pair data is processed according to the data processing method corresponding to the target list operation type, and the processed data processing result is obtained. For example, the target list operation type is a list operation that performs a key-value operation, such as lpush, rpush, lrange, rpop, etc., which can ensure fast and efficient key-value data storage and retrieval.
[0042] Step 3: Feedback the data processing results corresponding to the at least one data processing instruction to the client.
[0043] After obtaining the data processing results respectively corresponding to the at least one data processing instruction, the data processing results respectively corresponding to the at least one data processing instruction may be fed back to the client.
[0044] It can be seen from the above technical solution that the method provided by the present invention can be applied to a solid-state hard disk corresponding to a key-value database, and the method includes: receiving at least one data processing instruction sent by a client; wherein each data processing instruction includes: target list type data, target list operation type; performing parallel data processing on the at least one data processing instruction to obtain a data processing result corresponding to each of the at least one data processing instructions; wherein, for each data processing instruction, the target key-value pair data in the key-value database is determined according to the target list type data, wherein each key-value pair data in the key-value database includes a key and at least one value, each value includes at least one data list, and each data list includes at least one element, wherein each element is a string; performing data processing corresponding to the target list operation type on the target key-value pair data to obtain a data processing result corresponding to the data processing instruction; and feeding back the data processing result corresponding to each of the at least one data processing instructions to the client. Since in the present application, the key-value pair data in the key-value database can be processed in parallel according to at least one data processing instruction, and the data processing results corresponding to each of the at least one data processing instructions can be obtained, and the data processing results corresponding to each of the at least one data processing instructions can be fed back to the client, therefore, the present application can use the preset data processing instructions to enable the client to efficiently connect to the key-value database, and can perform parallel data processing on the key-value pair data in the key-value database, thereby realizing efficient parallel data processing, thereby improving the performance of the application, improving the performance of the client, reducing operation delays, and thus achieving improved data processing efficiency, reduced data processing costs, and thus improved user experience of data processing.
[0045] Next, embodiments for different target list data types and target list operation types are introduced respectively.
[0046] In one implementation, the target list type data includes a key of a key-value pair to be processed and a target element, and the target list operation type is a first add operation.
[0047] In this embodiment, the specific implementation method of determining the target key-value pair data in the key-value database according to the target list type data may be:
[0048] If the key of the key-value pair to be processed and the target element conform to a preset string type, and the key of the key-value pair to be processed meets a preset threshold condition, a key-value pair class is generated using the key of the key-value pair to be processed and the target element;
[0049] Determine whether there is key-value pair data corresponding to the key-value pair class in the key-value database; wherein the key in the key-value pair data is the same as the key of the key-value pair to be processed;
[0050] If key-value pair data corresponding to the key-value pair class exists in the key-value database, the key-value pair data corresponding to the key-value pair class is used as the target key-value pair data;
[0051] If the key-value pair data corresponding to the key-value pair class does not exist in the key-value database, an empty key-value pair data is created in the key-value database, and the empty key-value pair data is used as the target key-value pair data.
[0052] Accordingly, performing data processing corresponding to the target list operation type on the target key-value pair data to obtain a data processing result corresponding to the data processing instruction includes:
[0053] Convert the value in the target key-value pair data into a data list;
[0054] Inserting the target element into the head of the data list to obtain an updated data list;
[0055] After all target elements are processed, the updated data list is converted into the values in the target key-value pair data;
[0056] Determine that the data processing result corresponding to the data processing instruction is that the element is added successfully, and use the length of the updated data list as the data processing result corresponding to the data processing instruction.
[0057] Next, combine Figure 2 Let’s give an example. Figure 2As shown, the target list type data includes the key of the key-value pair to be processed and the target element members; the target list operation type is the first addition operation lpush, that is, the string value corresponding to the target element members is added to the head (i.e., the first position) of the data list stored corresponding to the key key; if the key key does not exist, an empty list will be created before the append operation. Specifically, the following steps are included:
[0058] S101: Receive parameters.
[0059] For step 101, at least two parameters, key and strings, are accepted, where key represents the key value corresponding to the list, and strings represents the string to be added (ie, the string value corresponding to the target element members).
[0060] S102: Determine whether the parameters are legal.
[0061] For step 102, determine whether the parameter satisfies the data type of String. If so, execute S103; otherwise, execute S111.
[0062] S103: Initialize a HetuItem using parameters.
[0063] For step 103, legal parameters will be passed here to initialize a HetuItem object.
[0064] S104: Determine whether HetuKVItem exists.
[0065] In step 104 , it is checked whether the input key exists in the database. If it exists, S105 is executed; otherwise, S107 is executed.
[0066] S105: Query the value of HetuKVItem.
[0067] In step 105, the value corresponding to the key is queried.
[0068] S106: Convert Value to List type.
[0069] In step 106, the queried value is converted into a list type, that is, the value in the target key-value pair data is converted into a data list.
[0070] S107: Add String to List.
[0071] In step 107, the String is added to the obtained list.
[0072] S108: Convert the List type to ByteStream.
[0073] For step 108, the code converts the list type into the bytestream type corresponding to the value, and converts the updated data list into the value in the target key-value pair data.
[0074] S109: Update to HetuKV.
[0075] For step 109, update the new value to HetuKV and determine whether the update is successful. If so, execute S110, otherwise execute S111.
[0076] S110: Return the length of the List.
[0077] For step 110, after a successful update, the current length of the list will be returned.
[0078] S111: Returns an error code.
[0079] For step 111, if S102 and S109 fail to execute, it means that the operation is wrong and cannot be performed. At this time, the corresponding error code will be returned and the operation will end.
[0080] In one implementation, the target list type data includes a key of a key-value pair to be processed and a target element, and the target list operation type is a second add operation.
[0081] In this embodiment, the specific implementation method of determining the target key-value pair data in the key-value database according to the target list type data may be:
[0082] If the key of the key-value pair to be processed and the target element conform to a preset string type, and the key of the key-value pair to be processed meets a preset threshold condition, a key-value pair class is generated using the key of the key-value pair to be processed and the target element;
[0083] Determine whether there is key-value pair data corresponding to the key-value pair class in the key-value database; wherein the key in the key-value pair data is the same as the key of the key-value pair to be processed;
[0084] If key-value pair data corresponding to the key-value pair class exists in the key-value database, the key-value pair data corresponding to the key-value pair class is used as the target key-value pair data;
[0085] If the key-value pair data corresponding to the key-value pair class does not exist in the key-value database, an empty key-value pair data is created in the key-value database, and the empty key-value pair data is used as the target key-value pair data.
[0086] Accordingly, performing data processing corresponding to the target list operation type on the target key-value pair data to obtain a data processing result corresponding to the data processing instruction includes:
[0087] Convert the value in the target key-value pair data into a data list;
[0088] Inserting the target element into the tail of the data list to obtain an updated data list;
[0089] After all target elements are processed, the updated data list is converted into the values in the target key-value pair data;
[0090] Determine that the data processing result corresponding to the data processing instruction is that the element is added successfully, and use the length of the updated data list as the data processing result corresponding to the data processing instruction.
[0091] Next, combine Figure 3 Let’s give an example. Figure 3 As shown, the target list type data includes the key key of the key-value pair to be processed and the target element members. The target list operation type is the second addition operation rpush, that is, the string value strings corresponding to the target element members is added to the end of the data list stored corresponding to key (i.e., the last element); if the key does not exist, an empty list will be created before the append operation. Specifically, the following steps are included:
[0092] S201: Receive parameters.
[0093] For step 201, at least two parameters, key and strings, are accepted, where key represents the key value corresponding to the list, and strings represents the string to be added.
[0094] S202: Determine whether the parameters are legal.
[0095] For step 202, determine whether the parameter is a string type and the key size cannot exceed the specified maximum value. If so, execute S203; otherwise, execute S211.
[0096] S203: Initialize a HetuItem using parameters.
[0097] For step 203, legal parameters will be passed here to initialize a HetuItem object.
[0098] S204: Determine whether HetuKVItem exists.
[0099] In step 204 , it is checked whether the key-value pair corresponding to the input key exists in the database. If so, S205 is executed; otherwise, S207 is executed.
[0100] S205: Query the value of HetuKVItem.
[0101] For step 205, the value corresponding to the Key is extracted.
[0102] S206: Convert value to List type.
[0103] In step 206, the queried value is converted into a list type.
[0104] S207: Add string to List.
[0105] For step 207, the string is inserted into the list.
[0106] S208: Convert the List type to ByteStream.
[0107] For step 208, the code converts the list type into the bytestream type corresponding to the value.
[0108] S209: Update to HetuKV.
[0109] For step 209, update the new value to HetuKV and determine whether the update is successful. If successful, execute S210; otherwise, execute S211.
[0110] S210: Return the length of List.
[0111] For step 210, after a successful update, the current length of the list will be returned and the operation ends.
[0112] S211: Returns an error code.
[0113] Regarding step 211, if S202 and S209 fail to execute, it is entered, indicating that the operation is wrong and cannot be performed. At this time, the corresponding error code will be returned and the operation will end.
[0114] In one implementation, the target list type data includes the key, element starting position, and element ending position of the key-value pair to be processed, and the target list operation type is a first query operation.
[0115] In this embodiment, determining the target key-value pair data in the key-value database according to the target list type data includes:
[0116] If the key of the key-value pair to be processed, the starting position of the element, and the ending position of the element conform to a preset string type, and the key of the key-value pair to be processed meets a preset threshold condition, a key-value pair class is generated using the key of the key-value pair to be processed, the starting position of the element, and the ending position of the element;
[0117] Determine whether there is key-value pair data corresponding to the key-value pair class in the key-value database; wherein the key in the key-value pair data is the same as the key of the key-value pair to be processed;
[0118] If key-value pair data corresponding to the key-value pair class exists in the key-value database, the key-value pair data corresponding to the key-value pair class is used as the target key-value pair data.
[0119] Accordingly, performing data processing corresponding to the target list operation type on the target key-value pair data to obtain a data processing result corresponding to the data processing instruction includes:
[0120] Convert the value in the target key-value pair data into a data list;
[0121] Determine whether the starting position of the element is less than or equal to the length of the data list;
[0122] If the element starting position is less than or equal to the length of the data list, the element starting position is less than 0, and the sum of the absolute value of the element starting position and the element end position is greater than or equal to the length of the data list, the element starting position is set to 0, and the value of the length of the data list is used as the new element end position; the elements between the element starting position and the new element end position are used as the data processing result corresponding to the data processing instruction;
[0123] If the element starting position is less than or equal to the length of the data list, the element starting position is less than 0, and the sum of the absolute value of the element starting position and the element end position is less than the length of the data list, the sum of the length of the data list and the element starting position is used as the new element starting position; and the elements between the new element starting position and the element end position are used as the data processing result corresponding to the data processing instruction;
[0124] If the element starting position is less than or equal to the length of the data list, the element starting position is greater than or equal to 0, and the element ending position is less than 0, the sum of the length of the data list and the element ending position is used as the new element ending position; the elements between the element starting position and the new element ending position are used as the data processing result corresponding to the data processing instruction;
[0125] If the element starting position is less than or equal to the length of the data list, the element starting position is greater than or equal to 0, and the element ending position is greater than the length of the data list, the length of the data list is used as the new element ending position; the elements between the element starting position and the new element ending position are used as the data processing result corresponding to the data processing instruction;
[0126] If the element starting position is less than or equal to the length of the data list, the element starting position is greater than or equal to 0, and the element ending position is greater than 0 and less than or equal to the length of the data list, the elements between the element starting position and the element ending position are used as the data processing result corresponding to the data processing instruction;
[0127] If the starting position of the element is greater than the length of the data list, a null value is used as the data processing result corresponding to the data processing instruction.
[0128] Next, combine Figure 4 Let’s give an example. Figure 4 As shown, the target list type data includes the key key of the key-value pair to be processed, the starting position start of the element and the ending position stop of the element. The target list operation type is the first query operation lrange: returns the specified element in the list stored with the specified key. The start and end are indexes based on 0, 0 is the first element of the list (list head), 1 is the next element, and so on. The first query operation lrange is to return the elements in the specified range in the list stored in the value corresponding to the specified key. This specification is specified using an index (which can be understood as a subscript). The subscript starts at 0, for example: the element with an index of 0 is the first element of the list (that is, the list head), the element with an index of 1 is the second element of the list, and so on. For example: after using the key to extract the value to obtain the list, this list is [a,b,c,d,e], start is 1, stop is 3, and the final returned result should be [b,c,d]. Specifically, it includes the following steps:
[0129] S301: Receive parameters.
[0130] For step 301, three parameters are accepted: key, start, and stop. Key is the key value corresponding to the list to be queried, start and stop are the starting and ending indexes of the list to be queried, 0 is the first element of the list (list head), 1 is the next element, and so on.
[0131] S302: Determine whether the parameters are legal.
[0132] For step 302, determine whether the parameter key is a string type and whether start and stop are integer types. If so, execute S303; otherwise, execute S321.
[0133] S303: Initialize a HetuItem using parameters.
[0134] For step 303, legal parameters will be passed here to initialize a HetuItem object.
[0135] S304: Determine whether HetuKVItem exists.
[0136] In step 304, it is checked whether the input key exists in the database. If it exists, execute S305; otherwise, execute S321.
[0137] S305: Query the value of HetuKVItem.
[0138] For step 305, the value corresponding to the key is queried.
[0139] S306: Convert the Value to List type.
[0140] In step 306, the queried value is converted into a list type.
[0141] S307: Determine if start<=list.size.
[0142] For step 307, it is determined whether the start is less than or equal to the length of the list. If so, S308 is executed; otherwise, S320 is executed.
[0143] S308: Determine if start<0.
[0144] For step 308 , it is determined whether the start is less than 0. If so, S309 is executed; otherwise, S311 is executed.
[0145] S309:|start|+end>=list.size.
[0146] For step 309, it is determined whether the absolute value of the start plus the end is greater than or equal to the length of the list. If so, execute S310; otherwise, execute S317.
[0147] S310: start=0, end=list.size.
[0148] For step 310, reset the start to 0 and the end to the length of the list.
[0149] S311:end<0.
[0150] For step 311, it is determined whether the end is less than 0. If so, S312 is executed; otherwise, S313 is executed.
[0151] S312: end = list.size + end.
[0152] For step 312, end is reset to list length plus end.
[0153] S313:end>list.size.
[0154] For step 313, it is determined whether the end is greater than the length of the list. If so, S314 is executed; otherwise, S315 is executed.
[0155] S314: end = list.size.
[0156] For step 314, end is set to the length of list.
[0157] S315: Add objects from start to list.size to the local List.
[0158] For step 315, all objects from the current start to the end of the list are added to an empty List.
[0159] S316: Return to List.
[0160] For step 316, the List created in the previous step will be returned.
[0161] S317: start=list.size+start.
[0162] For step 317, the start is reset to the list length plus the start.
[0163] S318: Add objects from start to list.size to the local empty List.
[0164] For step 318, all objects from the current start to the end of the list are added to an empty List.
[0165] S319: start = 0.
[0166] In step 319, start is reset to 0.
[0167] S320: Return an empty List.
[0168] For step 320, when the starting position is greater than the list length, an empty list will be returned.
[0169] S321: Returns an error code.
[0170] For step 321, if S302 and S304 fail to execute, it means that the operation is wrong and cannot be performed. In this case, the corresponding error code will be returned and the operation will end.
[0171] In one implementation, the target list type data includes the key of the key-value pair to be processed and the number of removal operations, and the target list operation type is a first removal operation.
[0172] In this embodiment, determining the target key-value pair data in the key-value database according to the target list type data includes:
[0173] If the key of the key-value pair to be processed and the number of removal operations both conform to the corresponding preset string type, and the key of the key-value pair to be processed meets the preset threshold condition, a key-value pair class is generated using the key of the key-value pair to be processed and the target element;
[0174] Determine whether there is key-value pair data corresponding to the key-value pair class in the key-value database; wherein the key in the key-value pair data is the same as the key of the key-value pair to be processed;
[0175] If key-value pair data corresponding to the key-value pair class exists in the key-value database, the key-value pair data corresponding to the key-value pair class is used as the target key-value pair data.
[0176] Accordingly, performing data processing corresponding to the target list operation type on the target key-value pair data to obtain a data processing result corresponding to the data processing instruction includes:
[0177] Convert the value in the target key-value pair data into a data list;
[0178] Performing the removal operation for the number of removal operations on the data list to obtain an adjusted data list; wherein each removal operation is performed to remove the last element in the data list;
[0179] Convert the adjusted data list into the values in the target key-value pair data, and use all the elements that are successfully removed as the data processing results corresponding to the data processing instruction; wherein, if the number of removal operations is greater than the length of the data list, all the elements that are successfully removed, and the preset null value identifiers of the target number are used as the data processing results corresponding to the data processing instruction, and the target number is the difference between the number of removal operations and the length of the data list.
[0180] Next, combine Figure 5 Let’s give an example. Figure 5 As shown, the target list type data includes the key of the key-value pair to be processed, the number of removal operations count, and the target list operation type is the first removal operation rpop, which returns and removes the last element in the data list. For example, if the data list contains "a", "b", and "c" elements, "c" will be returned and the data list will become "a" and "b". If the key does not exist or the list is empty, the special value "null" will be returned. Specifically, the following steps are included:
[0181] S401: Receive parameters.
[0182] For step 401, two parameters are accepted: key and count, where key represents the key of the operation list and count represents the number of operations.
[0183] S402: Determine whether the parameters are legal.
[0184] For step 402, determine whether the parameter key is a string type and whether count is an integer type. If so, execute S403; otherwise, execute S412.
[0185] S403: Initialize a HetuKVItem using parameters.
[0186] For step 403, legal parameters will be passed here to initialize a HetuItem object.
[0187] S404: Determine whether HetuKVItem exists.
[0188] In step 404, use the key of HetuItem to query the corresponding key-value pair. Determine whether it exists in HetuKV. If so, execute S405; otherwise, execute S411.
[0189] S405: Query the value of HetuKVItem.
[0190] For step 405, if the key-value pair exists, its value is extracted.
[0191] S406: Convert the Value to List type.
[0192] For step 406, convert the value into a list type.
[0193] S407: Delete the element from the List.
[0194] For step 407, the last element is deleted from List, and the number of times is count.
[0195] S408: Convert the List type into ByteStream.
[0196] For step 408 , the List type is converted into a ByteStream for re-storage.
[0197] S409: Update to HetuKV.
[0198] For step 409, use HetuKVItem to update the corresponding key-value pair in HetuKV.
[0199] S410: Return element.
[0200] For step 410, the deleted element is returned.
[0201] S411: Return Null.
[0202] For step 411, when the key-value pair corresponding to the parameter key does not exist, Null is returned.
[0203] S412: Returns an error code.
[0204] Regarding step 412, it is entered when S402 and S409 fail to execute, which means that the operation is wrong and cannot be performed. At this time, the corresponding error code will be returned and the operation will end.
[0205] In one implementation, the target list type data includes the key of the key-value pair to be processed and the number of removal operations, and the target list operation type is a second removal operation.
[0206] In this embodiment, determining the target key-value pair data in the key-value database according to the target list type data includes:
[0207] If the key of the key-value pair to be processed and the number of removal operations both conform to the corresponding preset string type, and the key of the key-value pair to be processed meets the preset threshold condition, a key-value pair class is generated using the key of the key-value pair to be processed and the target element;
[0208] Determine whether there is key-value pair data corresponding to the key-value pair class in the key-value database; wherein the key in the key-value pair data is the same as the key of the key-value pair to be processed;
[0209] If key-value pair data corresponding to the key-value pair class exists in the key-value database, the key-value pair data corresponding to the key-value pair class is used as the target key-value pair data.
[0210] Accordingly, performing data processing corresponding to the target list operation type on the target key-value pair data to obtain a data processing result corresponding to the data processing instruction includes:
[0211] Convert the value in the target key-value pair data into a data list;
[0212] Performing the removal operation for the number of removal operations on the data list to obtain an adjusted data list; wherein each removal operation is performed on the first element in the data list;
[0213] Convert the adjusted data list into the values in the target key-value pair data, and use all the elements that are successfully removed as the data processing results corresponding to the data processing instruction; wherein, if the number of removal operations is greater than the length of the data list, all the elements that are successfully removed, and the preset null value identifiers of the target number are used as the data processing results corresponding to the data processing instruction, and the target number is the difference between the number of removal operations and the length of the data list.
[0214] Next, combine Figure 6 Let’s give an example. Figure 6 As shown, the target list type data includes the key of the key-value pair to be processed, the number of removal operations count, and the target list operation type is the second removal operation lpop, which returns and removes the first element in the list. For example, if the list contains the elements "a", "b", and "c", LPOP will return "a" and the list will become "b" and "c". If the key does not exist or the list is empty, the special value "null" will be returned. Specifically, the following steps are included:
[0215] S501: Receive parameters.
[0216] For step 501, two parameters are accepted: key and count, where key represents the key of the operation list and count represents the number of operations.
[0217] S502: Determine whether the parameters are legal.
[0218] For step 502, determine whether the parameter key is a string type and whether count is an integer type. If so, execute S503; otherwise, execute S512.
[0219] S503: Initialize a HetuKVItem using parameters.
[0220] For step 503, legal parameters will be passed here to initialize a HetuItem object.
[0221] S504: Determine whether HetuKVItem exists.
[0222] For step 504, use the key of HetuItem to check whether the corresponding key-value pair exists in HetuKV. If so, execute S505; otherwise, execute S511.
[0223] S505: Query the value of HetuKVItem.
[0224] For step 505, if the key-value pair exists, its value is extracted.
[0225] S506: Convert the Value to List type.
[0226] For step 506, convert the value into a list type.
[0227] S507: Delete the first element from the List.
[0228] For step 507, the first element is deleted from List, and the number of times is count.
[0229] S508: Convert the List type into ByteStream.
[0230] For step 508 , the List type is converted into a ByteStream for re-storage.
[0231] S509: Update to HetuKV.
[0232] For step 509, use HetuKVItem to update the corresponding key-value pair in HetuKV.
[0233] S510: Return element.
[0234] For step 510, the first element deleted is returned.
[0235] S511: Return Null.
[0236] For step 511, when the key-value pair corresponding to the parameter key does not exist, Null is returned.
[0237] S512: Return an error code.
[0238] Regarding step 512, it is entered when S502 and S509 fail to execute, which means that the operation is wrong and cannot be performed. At this time, the corresponding error code will be returned and the operation will end.
[0239] In one implementation, the target list type data includes keys of key-value pairs to be processed, and the target list operation type is a second query operation.
[0240] In this embodiment, determining the target key-value pair data in the key-value database according to the target list type data includes:
[0241] If the key of the key-value pair to be processed and the number of removal operations both conform to the corresponding preset string type, and the key of the key-value pair to be processed meets the preset threshold condition, a key-value pair class is generated using the key of the key-value pair to be processed and the target element;
[0242] Determine whether there is key-value pair data corresponding to the key-value pair class in the key-value database; wherein the key in the key-value pair data is the same as the key of the key-value pair to be processed;
[0243] If key-value pair data corresponding to the key-value pair class exists in the key-value database, the key-value pair data corresponding to the key-value pair class is used as the target key-value pair data.
[0244] Accordingly, performing data processing corresponding to the target list operation type on the target key-value pair data to obtain a data processing result corresponding to the data processing instruction includes:
[0245] Convert the value in the target key-value pair data into a data list;
[0246] The length of the data list is used as the data processing result corresponding to the data processing instruction.
[0247] Next, combine Figure 7 Let’s give an example. Figure 7 As shown, the target list type data includes the key key of the key-value pair to be processed, and the target list operation type is the second query operation llen, that is, returning the length of the list stored with the specified key. If the key does not exist, 0 is returned (the same behavior as an empty list). Specifically, the following steps are included:
[0248] S601: Receive parameters.
[0249] For step 601, a key parameter is accepted, which represents the key of the list to be queried.
[0250] S602: Determine whether the parameters are legal.
[0251] For step 602, determine whether the parameter key is a string type. If so, execute S603; otherwise, execute S609.
[0252] S603: Initialize a HetuKVItem using parameters.
[0253] For step 603, legal parameters will be passed here to initialize a HetuItem object.
[0254] S604: Determine whether HetuKVItem exists.
[0255] For step 604, use the key of HetuItem to check whether the corresponding key-value pair exists in HetuKV. If so, execute S605; otherwise, execute S608.
[0256] S605: Query the value of HetuKVItem.
[0257] For step 605, if the key-value pair exists, its value is extracted.
[0258] S606: Convert the Value to List type.
[0259] For step 606, convert value into list type.
[0260] S607: Return the length of the list.
[0261] For step 607, the length of the list is returned.
[0262] S608: Return to 0.
[0263] For step 608, when the key-value pair corresponding to the parameter key does not exist, 0 is returned.
[0264] S609: Return an error code.
[0265] Regarding step 609, when S602 fails to execute, it enters, indicating that the operation is wrong and cannot be performed. At this time, the corresponding error code will be returned and the operation will end.
[0266] In one implementation, the target list type data includes a key and a query index of a key-value pair to be processed, and the target list operation type is a third query operation.
[0267] In this embodiment, determining the target key-value pair data in the key-value database according to the target list type data includes:
[0268] If the key of the key-value pair to be processed and the query index both conform to the corresponding preset string type, and the key of the key-value pair to be processed meets the preset threshold condition, a key-value pair class is generated using the key of the key-value pair to be processed and the target element;
[0269] Determine whether there is key-value pair data corresponding to the key-value pair class in the key-value database; wherein the key in the key-value pair data is the same as the key of the key-value pair to be processed;
[0270] If key-value pair data corresponding to the key-value pair class exists in the key-value database, the key-value pair data corresponding to the key-value pair class is used as the target key-value pair data.
[0271] Accordingly, performing data processing corresponding to the target list operation type on the target key-value pair data to obtain a data processing result corresponding to the data processing instruction includes:
[0272] Convert the value in the target key-value pair data into a data list;
[0273] If the query index is less than the length of the data list, and the query index is less than 0, taking the sum of the query index and the length of the data list as a new query index; using the new query index to determine the element in the data list corresponding to the new query index; and taking the element in the data list corresponding to the new query index as the data processing result corresponding to the data processing instruction;
[0274] If the query index is less than or equal to the length of the data list, and the query index is greater than or equal to 0, then taking the element in the data list corresponding to the query index as the data processing result corresponding to the data processing instruction;
[0275] If the query index is greater than the length of the data list, a preset null value identifier is used as the data processing result corresponding to the data processing instruction.
[0276] Next, combine Figure 8 Let’s give an example. Figure 8 As shown, the target list type data includes the key key of the key-value pair to be processed and the query index index. The target list operation type is the third query operation lindex, that is, returning the specified element in the list stored by the specified key. 0 represents the first element, 1 represents the second element, and so on. In addition, negative indexes are supported, such as -1 for the last element, -2, etc. If the index is out of range, a preset null value identifier "null" will be returned. Specifically, the following steps are included:
[0277] S701: Receive parameters.
[0278] For step 701, two parameters, key and index, are accepted, where key represents the key of the list to be queried, index represents the query index, 0 represents the first element, 1 represents the second element, and negative indexes are supported, for example, -1 represents the last element, -2 represents the second to last element, and so on.
[0279] S702: Determine whether the parameters are legal.
[0280] For step 702, determine whether the parameter key is a string type and whether the index is an integer type. If so, execute S703; otherwise, execute S712.
[0281] S703: Initialize a HetuKVItem using parameters.
[0282] For step 703, legal parameters will be passed here to initialize a HetuItem object.
[0283] S704: Determine whether HetuKVItem exists.
[0284] For step 704, use the key of HetuItem to check whether the corresponding key-value pair exists in HetuKV. If so, execute S705; otherwise, execute S711.
[0285] S705: Query the value of HetuKVItem.
[0286] For step 705, if the key-value pair exists, its value is extracted.
[0287] S706: Convert the Value to a list type.
[0288] For step 706, convert value into list type.
[0289] S707: If index>list.size.
[0290] For step 707, determine whether the input index position exceeds the list length. If so, execute S711; otherwise, execute S708.
[0291] S708: If index<0.
[0292] For step 708 , determine whether the input index position is a negative number, if so, execute S709 , otherwise execute S710 .
[0293] S709:index=list.size+index.
[0294] For step 709, if the index is a negative number, the length is added to the index to obtain the sequential position.
[0295] S710: Return the value corresponding to index.
[0296] For step 710, the value is obtained by the index position, the value is returned and the operation ends.
[0297] S711: Return Null.
[0298] For step 711 , when the key-value pair does not exist or the index is out of range, Null is returned.
[0299] S712: Returns an error code.
[0300] Regarding step 712, when S702 fails to execute, it enters, indicating that the operation is wrong and cannot be performed. At this time, the corresponding error code will be returned and the operation will end.
[0301] It can be seen from the above technical solution that the method provided by the present invention can be applied to a solid-state hard disk corresponding to a key-value database, and the method includes: receiving at least one data processing instruction sent by a client; wherein each data processing instruction includes: target list type data, target list operation type; performing parallel data processing on the at least one data processing instruction to obtain a data processing result corresponding to each of the at least one data processing instructions; wherein, for each data processing instruction, the target key-value pair data in the key-value database is determined according to the target list type data, wherein each key-value pair data in the key-value database includes a key and at least one value, each value includes at least one data list, and each data list includes at least one element, wherein each element is a string; performing data processing corresponding to the target list operation type on the target key-value pair data to obtain a data processing result corresponding to the data processing instruction; and feeding back the data processing result corresponding to each of the at least one data processing instructions to the client. Since in the present application, the key-value pair data in the key-value database can be processed in parallel according to at least one data processing instruction, and the data processing results corresponding to each of the at least one data processing instructions can be obtained, and the data processing results corresponding to each of the at least one data processing instructions can be fed back to the client, therefore, the present application can use the preset data processing instructions to enable the client to efficiently connect to the key-value database, and can perform parallel data processing on the key-value pair data in the key-value database, thereby realizing efficient parallel data processing, thereby improving the performance of the application, improving the performance of the client, reducing operation delays, and thus achieving improved data processing efficiency, reduced data processing costs, and thus improved user experience of data processing.
[0302] In one embodiment, before the step of receiving at least one data processing instruction sent by the client, the method further includes:
[0303] Receiving a call instruction sent by the client; wherein the call instruction includes a target solid state drive identifier;
[0304] According to the target solid-state hard disk identifier, the solid-state hard disk corresponding to the target solid-state hard disk identifier is called, and the solid-state hard disk corresponding to the target solid-state hard disk identifier is used as the solid-state hard disk corresponding to the key-value database, and the state of the solid-state hard disk corresponding to the key-value database is adjusted to a locked state.
[0305] Accordingly, after the step of feeding back to the client the data processing results corresponding to each of the at least one data processing instructions, the method further includes:
[0306] The memory of the solid-state hard disk corresponding to the key-value database is released, and the state of the solid-state hard disk corresponding to the key-value database is adjusted to an idle state.
[0307] As an example, when using a Java client to call the HetuKV key-value database, a certain solid-state drive will be specified and locked, and released after the call ends, that is, the garbage collection mechanism of the HetuKV key-value database is used to promptly release the memory occupied by the solid-state drive. During the process of locking the solid-state drive, other processes cannot use the solid-state drive. Moreover, in this embodiment, data calculation is decentralized to the solid-state drive, which can reduce dependence on memory, and optimize resource utilization by utilizing the garbage collection mechanism, and lock the solid-state drive to ensure data security and reliability, thereby achieving resource optimization and security, that is, through the advantages of HetuKV, resource utilization is optimized, data security and reliability are ensured, and memory dependence can be reduced, hardware resource utilization is optimized, memory consumption is reduced, and the system is more flexible. In addition, security and control rights can be enhanced, complete control rights are provided, and data security is strengthened to ensure data security for enterprise-level applications.
[0308] Thus, this embodiment implements the processing of the data processing method in conjunction with a specific application scenario. Of course, it should be understood that the above scenario is merely an example scenario and does not limit the method provided by the present invention. The method provided by the present invention can be extended to other data processing methods based on the same principles.
[0309] like Figure 9 The figure shows a specific embodiment of the data processing device described in this application. The device described in this embodiment is a physical device for executing the method described in the above embodiment. Its technical solution is essentially consistent with the above embodiment, and the corresponding description in the above embodiment is also applicable to this embodiment. In this embodiment, the device is applied to a solid-state hard disk corresponding to a key-value database. The device includes:
[0310] The first unit 901 is configured to receive at least one data processing instruction sent by a client; wherein each data processing instruction includes: target list type data and target list operation type;
[0311] The second unit 902 is configured to perform parallel data processing on the at least one data processing instruction to obtain data processing results corresponding to each of the at least one data processing instructions; wherein, for each data processing instruction, target key-value pair data in the key-value database is determined based on the target list type data, wherein each key-value pair data in the key-value database includes a key and at least one value, each value includes at least one data list, and each data list includes at least one element, wherein each element is a string; data processing corresponding to the target list operation type is performed on the target key-value pair data to obtain a data processing result corresponding to the data processing instruction;
[0312] The third unit 903 is configured to feed back the data processing results corresponding to each of the at least one data processing instructions to the client.
[0313] Optionally, the target list type data includes a key of a key-value pair to be processed and a target element, and the target list operation type is a first add operation.
[0314] Optionally, the second unit 902 is used to:
[0315] If the key of the key-value pair to be processed and the target element conform to a preset string type, and the key of the key-value pair to be processed meets a preset threshold condition, a key-value pair class is generated using the key of the key-value pair to be processed and the target element;
[0316] Determine whether there is key-value pair data corresponding to the key-value pair class in the key-value database; wherein the key in the key-value pair data is the same as the key of the key-value pair to be processed;
[0317] If key-value pair data corresponding to the key-value pair class exists in the key-value database, the key-value pair data corresponding to the key-value pair class is used as the target key-value pair data;
[0318] If the key-value pair data corresponding to the key-value pair class does not exist in the key-value database, an empty key-value pair data is created in the key-value database, and the empty key-value pair data is used as the target key-value pair data.
[0319] Optionally, the second unit 902 is used to:
[0320] Convert the value in the target key-value pair data into a data list;
[0321] Inserting the target element into the head of the data list to obtain an updated data list;
[0322] After all target elements are processed, the updated data list is converted into the values in the target key-value pair data;
[0323] Determine that the data processing result corresponding to the data processing instruction is that the element is added successfully, and use the length of the updated data list as the data processing result corresponding to the data processing instruction.
[0324] Optionally, the target list type data includes the key of the key-value pair to be processed and the target element, and the target list operation type is a second adding operation.
[0325] Optionally, the second unit 902 is used to:
[0326] If the key of the key-value pair to be processed and the target element conform to a preset string type, and the key of the key-value pair to be processed meets a preset threshold condition, a key-value pair class is generated using the key of the key-value pair to be processed and the target element;
[0327] Determine whether there is key-value pair data corresponding to the key-value pair class in the key-value database; wherein the key in the key-value pair data is the same as the key of the key-value pair to be processed;
[0328] If key-value pair data corresponding to the key-value pair class exists in the key-value database, the key-value pair data corresponding to the key-value pair class is used as the target key-value pair data;
[0329] If the key-value pair data corresponding to the key-value pair class does not exist in the key-value database, an empty key-value pair data is created in the key-value database, and the empty key-value pair data is used as the target key-value pair data.
[0330] Optionally, the second unit 902 is used to:
[0331] Convert the value in the target key-value pair data into a data list;
[0332] Inserting the target element into the tail of the data list to obtain an updated data list;
[0333] After all target elements are processed, the updated data list is converted into the values in the target key-value pair data;
[0334] Determine that the data processing result corresponding to the data processing instruction is that the element is added successfully, and use the length of the updated data list as the data processing result corresponding to the data processing instruction.
[0335] Optionally, the target list type data includes the key, element starting position and element ending position of the key-value pair to be processed, and the target list operation type is a first query operation.
[0336] Optionally, the second unit 902 is used to:
[0337] If the key of the key-value pair to be processed, the starting position of the element, and the ending position of the element conform to a preset string type, and the key of the key-value pair to be processed meets a preset threshold condition, a key-value pair class is generated using the key of the key-value pair to be processed, the starting position of the element, and the ending position of the element;
[0338] Determine whether there is key-value pair data corresponding to the key-value pair class in the key-value database; wherein the key in the key-value pair data is the same as the key of the key-value pair to be processed;
[0339] If key-value pair data corresponding to the key-value pair class exists in the key-value database, the key-value pair data corresponding to the key-value pair class is used as the target key-value pair data.
[0340] Optionally, the second unit 902 is used to:
[0341] Convert the value in the target key-value pair data into a data list;
[0342] Determine whether the starting position of the element is less than or equal to the length of the data list;
[0343] If the element starting position is less than or equal to the length of the data list, the element starting position is less than 0, and the sum of the absolute value of the element starting position and the element end position is greater than or equal to the length of the data list, the element starting position is set to 0, and the value of the length of the data list is used as the new element end position; the elements between the element starting position and the new element end position are used as the data processing result corresponding to the data processing instruction;
[0344] If the element starting position is less than or equal to the length of the data list, the element starting position is less than 0, and the sum of the absolute value of the element starting position and the element end position is less than the length of the data list, the sum of the length of the data list and the element starting position is used as the new element starting position; and the elements between the new element starting position and the element end position are used as the data processing result corresponding to the data processing instruction;
[0345] If the element starting position is less than or equal to the length of the data list, the element starting position is greater than or equal to 0, and the element ending position is less than 0, the sum of the length of the data list and the element ending position is used as the new element ending position; the elements between the element starting position and the new element ending position are used as the data processing result corresponding to the data processing instruction;
[0346] If the element starting position is less than or equal to the length of the data list, the element starting position is greater than or equal to 0, and the element ending position is greater than the length of the data list, the length of the data list is used as the new element ending position; the elements between the element starting position and the new element ending position are used as the data processing result corresponding to the data processing instruction;
[0347] If the element starting position is less than or equal to the length of the data list, the element starting position is greater than or equal to 0, and the element ending position is greater than 0 and less than or equal to the length of the data list, the elements between the element starting position and the element ending position are used as the data processing result corresponding to the data processing instruction;
[0348] If the starting position of the element is greater than the length of the data list, a null value is used as the data processing result corresponding to the data processing instruction.
[0349] Optionally, the target list type data includes the key of the key-value pair to be processed and the number of removal operations, and the target list operation type is the first removal operation.
[0350] Optionally, the second unit 902 is used to:
[0351] If the key of the key-value pair to be processed and the number of removal operations both conform to the corresponding preset string type, and the key of the key-value pair to be processed meets the preset threshold condition, a key-value pair class is generated using the key of the key-value pair to be processed and the target element;
[0352] Determine whether there is key-value pair data corresponding to the key-value pair class in the key-value database; wherein the key in the key-value pair data is the same as the key of the key-value pair to be processed;
[0353] If key-value pair data corresponding to the key-value pair class exists in the key-value database, the key-value pair data corresponding to the key-value pair class is used as the target key-value pair data.
[0354] Optionally, the second unit 902 is used to:
[0355] Convert the value in the target key-value pair data into a data list;
[0356] Performing the removal operation for the number of removal operations on the data list to obtain an adjusted data list; wherein each removal operation is performed to remove the last element in the data list;
[0357] Convert the adjusted data list into the values in the target key-value pair data, and use all the elements that are successfully removed as the data processing results corresponding to the data processing instruction; wherein, if the number of removal operations is greater than the length of the data list, all the elements that are successfully removed, and the preset null value identifiers of the target number are used as the data processing results corresponding to the data processing instruction, and the target number is the difference between the number of removal operations and the length of the data list.
[0358] Optionally, the target list type data includes the key of the key-value pair to be processed and the number of removal operations, and the target list operation type is a second removal operation.
[0359] Optionally, the second unit 902 is used to:
[0360] If the key of the key-value pair to be processed and the number of removal operations both conform to the corresponding preset string type, and the key of the key-value pair to be processed meets the preset threshold condition, a key-value pair class is generated using the key of the key-value pair to be processed and the target element;
[0361] Determine whether there is key-value pair data corresponding to the key-value pair class in the key-value database; wherein the key in the key-value pair data is the same as the key of the key-value pair to be processed;
[0362] If key-value pair data corresponding to the key-value pair class exists in the key-value database, the key-value pair data corresponding to the key-value pair class is used as the target key-value pair data.
[0363] Optionally, the second unit 902 is used to:
[0364] Convert the value in the target key-value pair data into a data list;
[0365] Performing the removal operation for the number of removal operations on the data list to obtain an adjusted data list; wherein each removal operation is performed on the first element in the data list;
[0366] Convert the adjusted data list into the values in the target key-value pair data, and use all the elements that are successfully removed as the data processing results corresponding to the data processing instruction; wherein, if the number of removal operations is greater than the length of the data list, all the elements that are successfully removed, and the preset null value identifiers of the target number are used as the data processing results corresponding to the data processing instruction, and the target number is the difference between the number of removal operations and the length of the data list.
[0367] Optionally, the target list type data includes the key of the key-value pair to be processed, and the target list operation type is a second query operation.
[0368] Optionally, the second unit 902 is used to:
[0369] If the key of the key-value pair to be processed and the number of removal operations both conform to the corresponding preset string type, and the key of the key-value pair to be processed meets the preset threshold condition, a key-value pair class is generated using the key of the key-value pair to be processed and the target element;
[0370] Determine whether there is key-value pair data corresponding to the key-value pair class in the key-value database; wherein the key in the key-value pair data is the same as the key of the key-value pair to be processed;
[0371] If key-value pair data corresponding to the key-value pair class exists in the key-value database, the key-value pair data corresponding to the key-value pair class is used as the target key-value pair data.
[0372] Optionally, the second unit 902 is used to:
[0373] Convert the value in the target key-value pair data into a data list;
[0374] The length of the data list is used as the data processing result corresponding to the data processing instruction.
[0375] Optionally, the target list type data includes a key and a query index of a key-value pair to be processed, and the target list operation type is a third query operation.
[0376] Optionally, the second unit 902 is used to:
[0377] If the key of the key-value pair to be processed and the query index both conform to the corresponding preset string type, and the key of the key-value pair to be processed meets the preset threshold condition, a key-value pair class is generated using the key of the key-value pair to be processed and the target element;
[0378] Determine whether there is key-value pair data corresponding to the key-value pair class in the key-value database; wherein the key in the key-value pair data is the same as the key of the key-value pair to be processed;
[0379] If key-value pair data corresponding to the key-value pair class exists in the key-value database, the key-value pair data corresponding to the key-value pair class is used as the target key-value pair data.
[0380] Optionally, the second unit 902 is used to:
[0381] Convert the value in the target key-value pair data into a data list;
[0382] If the query index is less than the length of the data list, and the query index is less than 0, taking the sum of the query index and the length of the data list as a new query index; using the new query index to determine the element in the data list corresponding to the new query index; and taking the element in the data list corresponding to the new query index as the data processing result corresponding to the data processing instruction;
[0383] If the query index is less than or equal to the length of the data list, and the query index is greater than or equal to 0, then taking the element in the data list corresponding to the query index as the data processing result corresponding to the data processing instruction;
[0384] If the query index is greater than the length of the data list, a preset null value identifier is used as the data processing result corresponding to the data processing instruction.
[0385] Optionally, the client is a Java client, and the key-value database is a co-located key-value database.
[0386] Optionally, the device further includes a fourth unit, configured to:
[0387] Receiving a call instruction sent by the client; wherein the call instruction includes a target solid state drive identifier;
[0388] According to the target solid-state hard disk identifier, the solid-state hard disk corresponding to the target solid-state hard disk identifier is called, and the solid-state hard disk corresponding to the target solid-state hard disk identifier is used as the solid-state hard disk corresponding to the key-value database, and the state of the solid-state hard disk corresponding to the key-value database is adjusted to a locked state.
[0389] Optionally, the fourth unit is configured to, after the step of feeding back to the client the data processing results corresponding to each of the at least one data processing instructions, further comprising:
[0390] The memory of the solid-state hard disk corresponding to the key-value database is released, and the state of the solid-state hard disk corresponding to the key-value database is adjusted to an idle state.
[0391] Figure 10 : This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. Among them, the memory may include internal memory, such as high-speed random access memory (RAM), and may also include non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.
[0392] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 10 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or one type of bus.
[0393] Memory is used to store execution instructions. Specifically, execution instructions are computer programs that can be executed. Memory can include internal memory and non-volatile memory, and provides execution instructions and data to the processor.
[0394] In one possible implementation, a processor reads corresponding execution instructions from a non-volatile memory into a memory and then executes them. Alternatively, the processor may obtain corresponding execution instructions from other devices to form a data processing device at a logical level. The processor executes the execution instructions stored in the memory to implement the data processing method provided in any embodiment of the present invention through the execution of the execution instructions.
[0395] The present invention Figure 1 The method performed by the data processing device provided in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or instructions in the form of software. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The various methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.
[0396] The steps of the method disclosed in conjunction with the embodiments of the present invention can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules within the decoding processor. The software modules can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the method described above.
[0397] An embodiment of the present invention also proposes a readable storage medium, which stores execution instructions. When the stored execution instructions are executed by the processor of an electronic device, the electronic device can execute the data processing method provided in any embodiment of the present invention, and is specifically used to execute the method described above for data query.
[0398] The electronic device described in each of the aforementioned embodiments may be a computer.
[0399] Those skilled in the art will appreciate that the embodiments of the present invention may be provided as methods or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or a combination of software and hardware.
[0400] The various embodiments of the present invention are described in a progressive manner. Similar portions between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the device embodiments are generally similar to the method embodiments, so their description is relatively simple. For relevant portions, refer to the description of the method embodiments.
[0401] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.
[0402] The foregoing is merely an embodiment of the present invention and is not intended to limit the present invention. It will be apparent to those skilled in the art that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention are intended to be included within the scope of the claims of the present invention.
Claims
1. A data processing method, characterized in that: The method is applied to a solid-state drive corresponding to a key-value database, where the key-value database is a co-located key-value database, and includes: Receive at least one data processing instruction sent by a client; wherein each data processing instruction includes: target list type data and target list operation type; the client is a Java client; performing parallel data processing on the at least one data processing instruction to obtain data processing results corresponding to each of the at least one data processing instructions; wherein, for each data processing instruction, determining target key-value pair data in the key-value database based on the target list type data, wherein each key-value pair data in the key-value database includes a key and at least one value, each value includes at least one data list, and each data list includes at least one element, wherein each element is a string; performing data processing corresponding to the target list operation type on the target key-value pair data to obtain a data processing result corresponding to the data processing instruction; Feedback to the client a data processing result corresponding to each of the at least one data processing instructions; The key-value database is multi-core parallel and can enable multiple cores of the CPU, with each core independently processing a data processing instruction; The target list type data includes the key, element starting position and element ending position of the key-value pair to be processed, and the target list operation type is the first query operation; The determining, according to the target list type data, target key-value pair data in the key-value database includes: If the key of the key-value pair to be processed, the starting position of the element, and the ending position of the element conform to a preset string type, and the key of the key-value pair to be processed meets a preset threshold condition, a key-value pair class is generated using the key of the key-value pair to be processed, the starting position of the element, and the ending position of the element; Determine whether there is key-value pair data corresponding to the key-value pair class in the key-value database; wherein the key in the key-value pair data is the same as the key of the key-value pair to be processed; If key-value pair data corresponding to the key-value pair class exists in the key-value database, the key-value pair data corresponding to the key-value pair class is used as the target key-value pair data; The performing data processing corresponding to the target list operation type on the target key-value pair data to obtain a data processing result corresponding to the data processing instruction includes: Convert the value in the target key-value pair data into a data list; Determine whether the starting position of the element is less than or equal to the length of the data list; If the element starting position is less than or equal to the length of the data list, the element starting position is less than 0, and the sum of the absolute value of the element starting position and the element end position is greater than or equal to the length of the data list, the element starting position is set to 0, and the value of the length of the data list is used as the new element end position; the elements between the element starting position and the new element end position are used as the data processing result corresponding to the data processing instruction; If the element starting position is less than or equal to the length of the data list, the element starting position is less than 0, and the sum of the absolute value of the element starting position and the element end position is less than the length of the data list, the sum of the length of the data list and the element starting position is used as the new element starting position; and the elements between the new element starting position and the element end position are used as the data processing result corresponding to the data processing instruction; If the element starting position is less than or equal to the length of the data list, the element starting position is greater than or equal to 0, and the element ending position is less than 0, the sum of the length of the data list and the element ending position is used as the new element ending position; the elements between the element starting position and the new element ending position are used as the data processing result corresponding to the data processing instruction; If the element starting position is less than or equal to the length of the data list, the element starting position is greater than or equal to 0, and the element ending position is greater than the length of the data list, the length of the data list is used as the new element ending position; the elements between the element starting position and the new element ending position are used as the data processing result corresponding to the data processing instruction; If the element starting position is less than or equal to the length of the data list, the element starting position is greater than or equal to 0, and the element ending position is greater than 0 and less than or equal to the length of the data list, the elements between the element starting position and the element ending position are used as the data processing result corresponding to the data processing instruction; If the starting position of the element is greater than the length of the data list, a null value is used as the data processing result corresponding to the data processing instruction.
2. The method according to claim 1, characterized in that The target list type data includes a key of a key-value pair to be processed and a target element, and the target list operation type is a first adding operation.
3. The method according to claim 2, characterized in that The determining, according to the target list type data, target key-value pair data in the key-value database includes: If the key of the key-value pair to be processed and the target element conform to a preset string type, and the key of the key-value pair to be processed meets a preset threshold condition, a key-value pair class is generated using the key of the key-value pair to be processed and the target element; Determine whether there is key-value pair data corresponding to the key-value pair class in the key-value database; wherein the key in the key-value pair data is the same as the key of the key-value pair to be processed; If key-value pair data corresponding to the key-value pair class exists in the key-value database, the key-value pair data corresponding to the key-value pair class is used as the target key-value pair data; If the key-value pair data corresponding to the key-value pair class does not exist in the key-value database, an empty key-value pair data is created in the key-value database, and the empty key-value pair data is used as the target key-value pair data.
4. The method according to claim 3, characterized in that The performing data processing corresponding to the target list operation type on the target key-value pair data to obtain a data processing result corresponding to the data processing instruction includes: Convert the value in the target key-value pair data into a data list; Inserting the target element into the head of the data list to obtain an updated data list; After all target elements are processed, the updated data list is converted into the values in the target key-value pair data; Determine that the data processing result corresponding to the data processing instruction is that the element is added successfully, and use the length of the updated data list as the data processing result corresponding to the data processing instruction.
5. The method according to claim 1, wherein The target list type data includes the key of the key-value pair to be processed and the target element, and the target list operation type is a second adding operation.
6. The method according to claim 5, characterized in that The determining, according to the target list type data, target key-value pair data in the key-value database includes: If the key of the key-value pair to be processed and the target element conform to a preset string type, and the key of the key-value pair to be processed meets a preset threshold condition, a key-value pair class is generated using the key of the key-value pair to be processed and the target element; Determine whether there is key-value pair data corresponding to the key-value pair class in the key-value database; wherein the key in the key-value pair data is the same as the key of the key-value pair to be processed; If key-value pair data corresponding to the key-value pair class exists in the key-value database, the key-value pair data corresponding to the key-value pair class is used as the target key-value pair data; If the key-value pair data corresponding to the key-value pair class does not exist in the key-value database, an empty key-value pair data is created in the key-value database, and the empty key-value pair data is used as the target key-value pair data.
7. The method according to claim 5, characterized in that The performing data processing corresponding to the target list operation type on the target key-value pair data to obtain a data processing result corresponding to the data processing instruction includes: Convert the value in the target key-value pair data into a data list; Inserting the target element into the tail of the data list to obtain an updated data list; After all target elements are processed, the updated data list is converted into the values in the target key-value pair data; Determine that the data processing result corresponding to the data processing instruction is that the element is added successfully, and use the length of the updated data list as the data processing result corresponding to the data processing instruction.
8. The method according to claim 1, characterized in that The target list type data includes the key of the key-value pair to be processed and the number of removal operations, and the target list operation type is the first removal operation.
9. The method according to claim 8, characterized in that The determining, according to the target list type data, target key-value pair data in the key-value database includes: If the key of the key-value pair to be processed and the number of removal operations both conform to the corresponding preset string type, and the key of the key-value pair to be processed meets the preset threshold condition, a key-value pair class is generated using the key of the key-value pair to be processed and the target element; Determine whether there is key-value pair data corresponding to the key-value pair class in the key-value database; wherein the key in the key-value pair data is the same as the key of the key-value pair to be processed; If key-value pair data corresponding to the key-value pair class exists in the key-value database, the key-value pair data corresponding to the key-value pair class is used as the target key-value pair data.
10. The method according to claim 9, characterized in that The performing data processing corresponding to the target list operation type on the target key-value pair data to obtain a data processing result corresponding to the data processing instruction includes: Convert the value in the target key-value pair data into a data list; Performing the removal operation for the number of removal operations on the data list to obtain an adjusted data list; wherein each removal operation is performed to remove the last element in the data list; Convert the adjusted data list into the values in the target key-value pair data, and use all the elements that are successfully removed as the data processing results corresponding to the data processing instruction; wherein, if the number of removal operations is greater than the length of the data list, all the elements that are successfully removed, and the preset null value identifiers of the target number are used as the data processing results corresponding to the data processing instruction, and the target number is the difference between the number of removal operations and the length of the data list.
11. The method according to claim 1, characterized in that The target list type data includes the key of the key-value pair to be processed and the number of removal operations, and the target list operation type is the second removal operation.
12. The method according to claim 11, characterized in that The determining, according to the target list type data, target key-value pair data in the key-value database includes: If the key of the key-value pair to be processed and the number of removal operations both conform to the corresponding preset string type, and the key of the key-value pair to be processed meets the preset threshold condition, a key-value pair class is generated using the key of the key-value pair to be processed and the target element; Determine whether there is key-value pair data corresponding to the key-value pair class in the key-value database; wherein the key in the key-value pair data is the same as the key of the key-value pair to be processed; If key-value pair data corresponding to the key-value pair class exists in the key-value database, the key-value pair data corresponding to the key-value pair class is used as the target key-value pair data.
13. The method according to claim 12, characterized in that The performing data processing corresponding to the target list operation type on the target key-value pair data to obtain a data processing result corresponding to the data processing instruction includes: Convert the value in the target key-value pair data into a data list; Performing the removal operation for the number of removal operations on the data list to obtain an adjusted data list; wherein each removal operation is performed on the first element in the data list; Convert the adjusted data list into the values in the target key-value pair data, and use all the elements that are successfully removed as the data processing results corresponding to the data processing instruction; wherein, if the number of removal operations is greater than the length of the data list, all the elements that are successfully removed, and the preset null value identifiers of the target number are used as the data processing results corresponding to the data processing instruction, and the target number is the difference between the number of removal operations and the length of the data list.
14. The method according to claim 1, wherein The target list type data includes the key of the key-value pair to be processed, and the target list operation type is a second query operation.
15. The method according to claim 14, characterized in that The determining, according to the target list type data, target key-value pair data in the key-value database includes: If the key and the number of removal operations of the key-value pair to be processed both conform to the corresponding preset string type, and the key of the key-value pair to be processed meets the preset threshold condition, a key-value pair class is generated using the key of the key-value pair to be processed and the target element; Determine whether there is key-value pair data corresponding to the key-value pair class in the key-value database; wherein the key in the key-value pair data is the same as the key of the key-value pair to be processed; If key-value pair data corresponding to the key-value pair class exists in the key-value database, the key-value pair data corresponding to the key-value pair class is used as the target key-value pair data.
16. The method according to claim 15, characterized in that The performing data processing corresponding to the target list operation type on the target key-value pair data to obtain a data processing result corresponding to the data processing instruction includes: Convert the value in the target key-value pair data into a data list; The length of the data list is used as the data processing result corresponding to the data processing instruction.
17. The method according to claim 1, wherein The target list type data includes the key and query index of the key-value pair to be processed, and the target list operation type is the third query operation.
18. The method according to claim 17, characterized in that The determining, according to the target list type data, target key-value pair data in the key-value database includes: If the key of the key-value pair to be processed and the query index both conform to the corresponding preset string type, and the key of the key-value pair to be processed meets the preset threshold condition, a key-value pair class is generated using the key of the key-value pair to be processed and the target element; Determine whether there is key-value pair data corresponding to the key-value pair class in the key-value database; wherein the key in the key-value pair data is the same as the key of the key-value pair to be processed; If key-value pair data corresponding to the key-value pair class exists in the key-value database, the key-value pair data corresponding to the key-value pair class is used as the target key-value pair data.
19. The method according to claim 18, characterized in that The performing data processing corresponding to the target list operation type on the target key-value pair data to obtain a data processing result corresponding to the data processing instruction includes: Convert the value in the target key-value pair data into a data list; If the query index is less than the length of the data list, and the query index is less than 0, taking the sum of the query index and the length of the data list as a new query index; using the new query index to determine the element in the data list corresponding to the new query index; and taking the element in the data list corresponding to the new query index as the data processing result corresponding to the data processing instruction; If the query index is less than or equal to the length of the data list, and the query index is greater than or equal to 0, then taking the element in the data list corresponding to the query index as the data processing result corresponding to the data processing instruction; If the query index is greater than the length of the data list, a preset null value identifier is used as the data processing result corresponding to the data processing instruction.
20. The method according to any one of claims 1 to 19, characterized in that: Before the step of receiving at least one data processing instruction sent by the client, the method further includes: Receiving a call instruction sent by the client; wherein the call instruction includes a target solid state drive identifier; According to the target solid-state hard disk identifier, the solid-state hard disk corresponding to the target solid-state hard disk identifier is called, and the solid-state hard disk corresponding to the target solid-state hard disk identifier is used as the solid-state hard disk corresponding to the key-value database, and the state of the solid-state hard disk corresponding to the key-value database is adjusted to a locked state.
21. The method according to claim 20, characterized in that After the step of feeding back to the client the data processing results corresponding to each of the at least one data processing instructions, the method further includes: The memory of the solid-state hard disk corresponding to the key-value database is released, and the state of the solid-state hard disk corresponding to the key-value database is adjusted to an idle state.
22. A data processing device, characterized in that: The device is applied to a solid-state drive corresponding to a key-value database, wherein the key-value database is a co-located key-value database, and the key-value database is multi-core parallel and can enable multiple cores of a CPU, each core independently processing a data processing instruction. The device includes: The first unit is configured to receive at least one data processing instruction sent by a client; wherein each data processing instruction includes: target list type data and target list operation type; the client is a Java client; The second unit is configured to perform parallel data processing on the at least one data processing instruction to obtain data processing results corresponding to each of the at least one data processing instructions; wherein, for each data processing instruction, target key-value pair data in the key-value database is determined according to the target list type data, wherein each key-value pair data in the key-value database includes a key and at least one value, each value includes at least one data list, and each data list includes at least one element, wherein each element is a string; data processing corresponding to the target list operation type is performed on the target key-value pair data to obtain a data processing result corresponding to the data processing instruction; A third unit is configured to feed back to the client a data processing result corresponding to each of the at least one data processing instructions; The second unit is further configured to: The target list type data includes the key, element starting position and element ending position of the key-value pair to be processed, and the target list operation type is the first query operation; The determining, according to the target list type data, target key-value pair data in the key-value database includes: If the key of the key-value pair to be processed, the starting position of the element, and the ending position of the element conform to a preset string type, and the key of the key-value pair to be processed meets a preset threshold condition, a key-value pair class is generated using the key of the key-value pair to be processed, the starting position of the element, and the ending position of the element; Determine whether there is key-value pair data corresponding to the key-value pair class in the key-value database; wherein the key in the key-value pair data is the same as the key of the key-value pair to be processed; If key-value pair data corresponding to the key-value pair class exists in the key-value database, the key-value pair data corresponding to the key-value pair class is used as the target key-value pair data; The performing data processing corresponding to the target list operation type on the target key-value pair data to obtain a data processing result corresponding to the data processing instruction includes: Convert the value in the target key-value pair data into a data list; Determine whether the starting position of the element is less than or equal to the length of the data list; If the element starting position is less than or equal to the length of the data list, the element starting position is less than 0, and the sum of the absolute value of the element starting position and the element end position is greater than or equal to the length of the data list, the element starting position is set to 0, and the value of the length of the data list is used as the new element end position; the elements between the element starting position and the new element end position are used as the data processing result corresponding to the data processing instruction; If the element starting position is less than or equal to the length of the data list, the element starting position is less than 0, and the sum of the absolute value of the element starting position and the element end position is less than the length of the data list, the sum of the length of the data list and the element starting position is used as the new element starting position; and the elements between the new element starting position and the element end position are used as the data processing result corresponding to the data processing instruction; If the element starting position is less than or equal to the length of the data list, the element starting position is greater than or equal to 0, and the element ending position is less than 0, the sum of the length of the data list and the element ending position is used as the new element ending position; the elements between the element starting position and the new element ending position are used as the data processing result corresponding to the data processing instruction; If the element starting position is less than or equal to the length of the data list, the element starting position is greater than or equal to 0, and the element ending position is greater than the length of the data list, the length of the data list is used as the new element ending position; the elements between the element starting position and the new element ending position are used as the data processing result corresponding to the data processing instruction; If the element starting position is less than or equal to the length of the data list, the element starting position is greater than or equal to 0, and the element ending position is greater than 0 and less than or equal to the length of the data list, the elements between the element starting position and the element ending position are used as the data processing result corresponding to the data processing instruction; If the starting position of the element is greater than the length of the data list, a null value is used as the data processing result corresponding to the data processing instruction.
23. A readable medium comprising an execution instruction, wherein when a processor of an electronic device executes the execution instruction, the electronic device executes the method according to any one of claims 1 to 21.
24. An electronic device comprising a processor and a memory storing execution instructions, wherein when the processor executes the execution instructions stored in the memory, the processor executes the method according to any one of claims 1 to 21.
Citation Information
Patent Citations
KV database data management method and device, computing equipment and storage medium
CN114328453A
Request processing method and device, data acquisition method and device and electronic equipment
CN116860172A