High-concurrency data processing method and device, electronic equipment and storage medium

Abstract

The invention discloses a high-concurrency data processing method and device, electronic equipment and a storage medium. The method comprises the following steps: constructing request mapping relationships corresponding to different requests, and storing the request mapping relationships in a preset cache region; the request mapping relation is a mapping relation among the first data, the database table and the second data; the second data is related data extracted from a database table associated with the first data; in response to a request operation of the first request, obtaining first request content for the first request, and performing hash processing on the first request content to obtain a first hash value; and if the first data matched with the first hash value exists in the preset cache region, taking the request mapping relationship corresponding to the first data matched with the first hash value as a reference request mapping relationship, and returning second data in the reference request mapping relationship or a state code of the reference request mapping relationship. According to the scheme, the high-concurrency data processing efficiency of the system is improved, and the system stability is improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a high-concurrency data processing method, device, electronic device and storage medium. Background Art

[0002] Currently, an increasing number of systems are required to handle real-time updates, historical data tracing, and multi-dimensional statistical analysis. These processes involve a large number of highly concurrent data processing processes and complex data query processes. These processes require responding to numerous user requests simultaneously. Typically, data is frequently called and queried based on user requests and returned to the user. However, different users may request the same or different data. This results in frequent system data calls, consuming a large amount of cache, leading to choppy system operation, reduced data processing efficiency, and poor system stability. Therefore, improving the system's efficiency in handling highly concurrent data and enhancing system stability is crucial. Summary of the Invention

[0003] The present invention provides a high-concurrency data processing method, device, electronic device and storage medium to improve the efficiency of the system in processing high-concurrency data and enhance the system stability.

[0004] According to one aspect of the present invention, a high-concurrency data processing method is provided, the method comprising:

[0005] Constructing request mapping relationships corresponding to different requests and storing the request mapping relationships in a preset cache area; the request mapping relationships are mapping relationships between first data, a database table, and second data; the first data is a hash value obtained by hashing third data, and the third data is the request content corresponding to the request; the second data is relevant data extracted from the database table associated with the first data;

[0006] In response to the request operation of the first request, obtaining first request content for the first request, and performing hash processing on the first request content to obtain a first hash value;

[0007] If the preset cache area contains first data matching the first hash value, the request mapping relationship corresponding to the first data matching the first hash value is used as the reference request mapping relationship, and the second data in the reference request mapping relationship or the status code of the reference request mapping relationship is returned.

[0008] According to another aspect of the present invention, a high-concurrency data processing device is provided, the device comprising:

[0009] a mapping relationship determination module, configured to construct request mapping relationships corresponding to different requests and store the request mapping relationships in a preset cache area; the request mapping relationships being mapping relationships between first data, a database table, and second data; the first data being a hash value obtained by hashing third data, the third data being request content corresponding to the request; and the second data being relevant data extracted from the database table associated with the first data;

[0010] a data determination module, configured to obtain, in response to a request operation of the first request, first request content for the first request, and hash the first request content to obtain a first hash value;

[0011] A data processing module is used to, if there is first data matching the first hash value in the preset cache area, use the request mapping relationship corresponding to the first data matching the first hash value as a reference request mapping relationship, and return the second data in the reference request mapping relationship or the status code of the reference request mapping relationship.

[0012] According to another aspect of the present invention, an electronic device is provided, comprising:

[0013] at least one processor; and

[0014] a memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the high-concurrency data processing method described in any embodiment of the present invention.

[0016] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the high-concurrency data processing method described in any embodiment of the present invention when executed.

[0017] The technical solution of the embodiment of the present invention is to construct a request mapping relationship corresponding to different requests, and the request mapping relationship is a mapping relationship between the first data, the database table and the second data; the first data is a hash value after hashing the third data, and the third data is the request content corresponding to the request; the second data is the relevant data extracted from the database table associated with the first data; and the request mapping relationship is stored in a preset cache area, so that the subsequent process can directly call data from the preset cache area, which can speed up the data query processing process. Further, in response to the request operation of the first request, the first request content for the first request is obtained, the first request content is hashed to obtain a first hash value; the first hash value is compared with the first data of each request mapping relationship in the preset cache area, if the preset cache area has a first data matching the first hash value, the request mapping relationship corresponding to the first data matching the first hash value is used as a reference request mapping relationship, and the second data in the reference request mapping relationship or the status code of the reference request mapping relationship is returned, that is, the data can be directly called from the preset cache area, avoiding the need to call the database again, and querying the data required to call the first request from the database, which greatly reduces the query process, improves the efficiency of the system in processing high-concurrency data, and improves the stability of the system.

[0018] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 This is a flowchart of a high-concurrency data processing method provided by an embodiment of the present invention;

[0021] Figure 2 is a flowchart of another high-concurrency data processing method provided according to an embodiment of the present invention;

[0022] Figure 3 This is a flow chart of updating a request mapping relationship in a preset cache area provided by an embodiment of the present invention;

[0023] Figure 4 2 is a schematic structural diagram of a high-concurrency data processing device provided according to an embodiment of the present invention;

[0024] Figure 5 3 is a schematic diagram of the structure of an electronic device for implementing the high-concurrency data processing method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0026] It should be noted that the terms "first", "second", "third", "fourth", "reference", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or inherent to these processes, methods, products or apparatus.

[0027] Example 1

[0028] Figure 1 This is a flowchart of a high-concurrency data processing method provided by an embodiment of the present invention. This embodiment is applicable to processing high-concurrency data in a system, and is applicable to systems with high-concurrency data processing, especially distributed systems in integrated energy scenarios. The method can be executed by a high-concurrency data processing device, which can be implemented in the form of hardware and / or software. The high-concurrency data processing device can be configured in any electronic device with network communication capabilities. Figure 1 As shown, the high-concurrency data processing method of the present invention includes:

[0029] S110. Construct a request mapping relationship corresponding to different requests, and store the request mapping relationship in a preset cache area; the request mapping relationship is a mapping relationship between the first data, the database table and the second data; the first data is a hash value after hashing the third data, and the third data is the request content corresponding to the request; the second data is related data extracted from the database table associated with the first data.

[0030] The request mapping relationship can be understood as the mapping relationship between the first data, the database table, and the second data. The preset cache area stores the request mapping relationship, and also stores the first data and the second data. After the preset cache area stores the above content, when querying data, the data can be called directly from the preset cache area without having to call the data from the database, which greatly reduces the data query process. The request content can be data that is strongly associated with the business data, ensuring that the hash value of the third data of the same request is the same after hashing.

[0031] In addition, the requests corresponding to the request mapping relationships stored in the preset cache area are all requests that access the system a preset number of times within the preset time; this can ensure that the request mapping relationships of requests with a large number of access times are saved in the preset cache area as much as possible, avoiding waste of preset cache area resources.

[0032] Specifically, historical requests processed by the system can be obtained, and requests in the historical requests that have accessed the system a preset number of times can be used as target requests for which a request mapping relationship needs to be established. The request content corresponding to the target request and the database table associated with the target request can be obtained, and the first data of the target request can be called from the database table associated with the target request according to the request content corresponding to the target request. Furthermore, the request content of the target request is hashed to obtain the first data of the target request; thereby establishing a correspondence between the first data, the second data and the database table associated with the target request, that is, a request mapping relationship corresponding to the target request, and storing the request mapping relationship in a preset cache area.

[0033] Optionally, the request mapping relationships in the preset cache area can be updated based on the number of times the request mapping relationships are called within a preset time period. Specifically, during system operation, if the number of times a request mapping relationship in the preset cache area is called is less than a preset number of times, the request mapping relationship that has been called less than the preset number of times can be deleted to free up resources in the preset cache area and avoid wasting resources.

[0034] S120 . In response to the request operation of the first request, obtain first request content for the first request, perform hash processing on the first request content, and obtain a first hash value.

[0035] Specifically, all data for the first request is obtained, all data of the first request is parameter-normalized to obtain normalized data, data in the normalized data that is strongly related to the business data is used as the first request content, and the first request content is hashed to obtain a first hash value. Further hashing the first request content to obtain the first hash value may include: hashing the first request content using an MD5 algorithm to obtain the first hash value.

[0036] S130. If the preset cache area contains first data that matches the first hash value, the request mapping relationship corresponding to the first data that matches the first hash value is used as the reference request mapping relationship, and the second data in the reference request mapping relationship or the status code of the reference request mapping relationship is returned.

[0037] The status code can be understood as identification information that prompts the user that the second data has been displayed on the user terminal, thereby avoiding multiple calls for displaying the data and wasting system resources.

[0038] Specifically, the preset cache area contains first data that matches the first hash value, indicating that the preset cache area has already stored the content to be mobilized for the first request. Therefore, the corresponding data can be called directly from the preset cache area without having to query the data from the database, which greatly shortens the data query process and improves the query efficiency of high-concurrency data.

[0039] In an embodiment of the present invention, optionally, after obtaining the first hash value, the method further includes: if the preset cache does not contain first data matching the first hash value, obtaining a first database table associated with the first request content from the database based on the first request content. Extracting data associated with the first request content from the first database table as second data for the first request, and returning the second data for the first request. Establishing a request mapping relationship for the first request based on the first hash value, the first database table, and the second data for the first request, and saving the request mapping relationship for the first request to the preset cache.

[0040] Specifically, if the preset cache does not contain the first data that matches the first hash value, it means that the preset cache does not contain the request mapping relationship corresponding to the first request. It is necessary to call the database to query the first database table associated with the first request content, so as to subsequently call the second data of the first request from the first database table. This process is to avoid the situation where the preset cache does not cache the request mapping relationship, but can still continue to query the data to be called by the request, without affecting the user experience. At the same time, a request mapping relationship is established for the first request based on the first hash value, the first database table and the second data of the first request, and the request mapping relationship of the first request is saved to the preset cache, so that the next time the request is approved, the corresponding second data can be directly called from the preset cache.

[0041] In an embodiment of the present invention, when the first data matching the first hash value does not exist in the preset cache area, the first database table associated with the first request content is obtained from the database according to the first request content, the data associated with the first request content is extracted from the first database table as the second data of the first request, and the second data of the first request is returned. This avoids the situation where the preset cache area does not cache the request mapping relationship, and the data to be called by the request can still be queried, which does not affect the user experience and ensures the stability of the system data processing.

[0042] As an optional embodiment, returning the second data in the reference request mapping relationship or the status code of the reference request mapping relationship includes: if it is detected that the first request carries instruction information, then returning the status code of the reference request mapping relationship; the instruction information is used to reflect that the user terminal has displayed the second data in the reference request mapping relationship. If it is detected that the first request does not carry instruction information, then returning the second data in the reference request mapping relationship and displaying the second data in the reference request mapping relationship. In this embodiment, the instruction information can be information that the system receives the same request again within a preset time interval; the preset time interval can be the maximum time interval between the two requests received by the system that are the same request. For example, if the time interval between the first request and the last time the system received the same request as the first request is less than the preset time interval, then the first request will carry instruction information. Carrying instruction information can ensure that the same data is returned frequently in the system, that is, to ensure the effective use of system resources.

[0043] In this embodiment, the request mapping relationship also includes a data correspondence between the first data and the fourth data, and the fourth data is a hash value after hashing the second data. Accordingly, returning the status code of the reference request mapping relationship includes: determining the fourth data of the reference request mapping relationship based on the data correspondence and the first hash value in the reference request mapping relationship; determining the status code of the reference request mapping relationship based on the fourth data of the reference request mapping relationship, and returning the status code of the reference request mapping relationship. In this embodiment, the request mapping relationship also includes a data correspondence between the first hash value and the fourth data. When the first request carries instruction information, it is not necessary to call the second data from the preset cache area first, and the fourth data corresponding to the first request can be directly obtained. Generally, the second data occupies more resources. By reducing the call of large data, the utilization of system resources is further optimized.

[0044] The technical solution of the embodiment of the present invention is to construct a request mapping relationship corresponding to different requests, and the request mapping relationship is a mapping relationship between the first data, the database table and the second data; the first data is a hash value after hashing the third data, and the third data is the request content corresponding to the request; the second data is the relevant data extracted from the database table associated with the first data; and the request mapping relationship is stored in a preset cache area, so that the subsequent process can directly call data from the preset cache area, which can speed up the data query processing process. Further, in response to the request operation of the first request, the first request content for the first request is obtained, the first request content is hashed to obtain a first hash value; the first hash value is compared with the first data of each request mapping relationship in the preset cache area, if the preset cache area has a first data matching the first hash value, the request mapping relationship corresponding to the first data matching the first hash value is used as a reference request mapping relationship, and the second data in the reference request mapping relationship or the status code of the reference request mapping relationship is returned, that is, the data can be directly called from the preset cache area, avoiding the need to call the database again, and querying the data required to call the first request from the database, which greatly reduces the query process, improves the efficiency of the system in processing high-concurrency data, and improves the stability of the system.

[0045] Example 2

[0046] Figure 2 This is a flowchart of another high-concurrency data processing method provided by an embodiment of the present invention. The technical solution of this embodiment further optimizes the process of constructing the request mapping relationship corresponding to different requests in S110 in the above embodiment on the basis of the above embodiment. This embodiment can be combined with various optional solutions in one or more of the above embodiments. Figure 2 As shown, the method includes:

[0047] S210. Determine a reference request based on the request content corresponding to the first candidate request, and use the second hash value as the first data corresponding to the reference request; the first candidate request is a preset number of requests accumulated by the system; the second hash value is a hash value obtained by hashing the request content corresponding to the reference request, and each reference request is a different request.

[0048] Specifically, determining the reference request according to the request content corresponding to the first candidate request includes steps A1-A4:

[0049] Step A1: Obtain a preset number of first candidate requests accumulated by the system and request contents corresponding to the first candidate requests; the request contents are data that are strongly associated with business data.

[0050] Step A2: Hash the request content corresponding to the first candidate request to obtain the candidate hash value corresponding to the first candidate request, group the first candidate requests with the same candidate hash value into a group and record them as a candidate request set, and determine the number of first candidate requests in each candidate request set.

[0051] Specifically, the MD5 algorithm may be used to hash the request content corresponding to the first candidate request to obtain a candidate hash value corresponding to the first candidate request. First candidate requests with the same candidate hash value are grouped together and recorded as a candidate request set to filter out the same request and avoid storing multiple request mappings for the same request in the preset cache.

[0052] Step A3: Select a first candidate request from each set of candidate requests as a second candidate request, and determine the request attribute information of the second candidate request according to the number of requests corresponding to the second candidate request; the request attribute information is used to reflect the access priority of the request.

[0053] The number of requests corresponding to the second candidate request is the number of first candidate requests in the candidate request set corresponding to the second candidate request. Access priority can be understood as reflecting whether a request frequently accesses the system; that is, the higher the access priority, the more frequently the system receives a request.

[0054] Specifically, the request attribute information of the second candidate request is determined by comparing the request quantity corresponding to the second candidate request with a preset value range.

[0055] For example, determining the request attribute information of the second candidate request based on the number of requests corresponding to the second candidate request includes: when the number of requests corresponding to the second candidate request is greater than a first preset value, the request attribute information of the second candidate request is a first priority; when the number of requests corresponding to the second candidate request is greater than a second preset value and the number of requests corresponding to the second candidate request is less than the first preset value, the request attribute information of the second candidate request is a second priority; when the number of requests corresponding to the second candidate request is less than the second preset value, the request attribute information of the second candidate request is a third priority. The first priority is higher than the second priority, and the second priority is higher than the third priority.

[0056] Step A4: The second candidate request corresponding to the request attribute information greater than the preset attribute information in the request attribute information is used as a reference request; the preset attribute information is used to limit the lower limit of the access priority of the requests stored in the preset cache area.

[0057] The access priority levels included in the request attribute information are at least three priority levels, and the preset attribute information can be determined according to the remaining storage space and access priority levels of the preset cache area.

[0058] For example, if the request attribute information includes three levels, namely the first priority, the second priority and the third priority, if the preset attribute information is the second priority, the second candidate request corresponding to the request attribute information in the request attribute information that is greater than the second priority will be used as the reference request, thereby accurately determining the reference request based on the request content corresponding to the first candidate request.

[0059] The technical solution of this embodiment is to obtain a preset number of first candidate requests accumulated by the system and the request contents corresponding to the first candidate requests, so as to obtain a sufficient number of requests to ensure the diversity of request mapping relationships that can be stored in the preset cache area. The request contents corresponding to the first candidate requests are further hashed to obtain the candidate hash values corresponding to the first candidate requests, and the first candidate requests with the same candidate hash values are grouped together and recorded as a candidate request set. The number of requests of the first candidate requests in each group of candidate request sets is determined, so as to classify the same requests in the first candidate requests, thereby facilitating the selection of a first candidate request as the second candidate request from each group of candidate request sets, avoiding duplication of request mapping relationships stored in the preset cache area and wasting resources of the preset cache area. The request attribute information of the second candidate request is determined based on the number of requests corresponding to the second candidate request; the request attribute information is used to reflect the access priority of the request, so that the second candidate request corresponding to the request attribute information in the request attribute information that is greater than the preset attribute information is used as a reference request, thereby accurately determining the requests that frequently initiate access to the system.

[0060] S220. If field information exists in the request content corresponding to the reference request, extract preset field data from the database table associated with the reference request, and use the preset field data as the second data corresponding to the reference request; the field information is identification information of the preset field data in the database table associated with the reference request.

[0061] Specifically, a reference request may request all field data for a specified date in a database table in the database, or it may request preset field data for a specified date. If a reference request may request preset field data for a specified date in a database table in the database, the reference request will contain field information in the request content. If a reference request may request all field data for a specified date in a database table in the database, the reference request will not contain field information in the request content. This allows accurate differentiation between requests for the same data table, such as whether to extract all data for a specified date in the data table or to extract specified data for a specified date in the data table, and avoids incorrect determination of request mapping relationships due to association with the same database table.

[0062] Furthermore, if there is field information in the request content corresponding to the reference request, the first data of the reference request includes a hash value obtained by hashing the field information.

[0063] S230: If there is no field information in the request content corresponding to the reference request, all data associated with the request content corresponding to the reference request are extracted from the database table associated with the reference request as second data corresponding to the reference request.

[0064] S240: Construct a request mapping relationship corresponding to the reference request based on the first data corresponding to the reference request, identification information of the database table associated with the reference request, and the second data corresponding to the reference request, and store the request mapping relationship in a preset cache area.

[0065] Specifically, if the request content corresponding to the reference request contains field information, a first request mapping relationship corresponding to the reference request is constructed based on the first data corresponding to the reference request, the identification information of the database table associated with the reference request, the identification information of the corresponding field information in the database table, and the second data corresponding to the reference request. For example, the first request mapping relationship can be expressed as: first data → {table 1: [field 1, field 2, ...], table 2: [field 3, field 4, ...], ...} → second data, i.e., the first request mapping relationship is a field-level mapping relationship.

[0066] If the request content corresponding to the reference request does not contain field information, a second request mapping relationship corresponding to the reference request is constructed based on the first data corresponding to the reference request, the identification information of the database table associated with the reference request, and the second data corresponding to the reference request. The second request mapping relationship can be expressed as: first data → {Table 1, Table 2, ...} → second data, i.e., the second request mapping relationship is a full-scale mapping relationship.

[0067] S250 : In response to the request operation of the first request, obtain first request content for the first request, perform hash processing on the first request content, and obtain a first hash value.

[0068] S260. If the preset cache area contains first data that matches the first hash value, the request mapping relationship corresponding to the first data that matches the first hash value is used as the reference request mapping relationship, and the second data in the reference request mapping relationship or the status code of the reference request mapping relationship is returned.

[0069] The technical solution of the embodiment of the present invention is to determine the reference request according to the request content corresponding to the first candidate request, and use the second hash value as the first data corresponding to the reference request; the first candidate request is a preset number of requests accumulated by the system; the second hash value is a hash value obtained by hashing the request content corresponding to the reference request, and each reference request is a different request. By judging whether there is field information in the request content corresponding to the reference request, the second data corresponding to the reference request is accurately determined. When different requests are associated with the same database table, they can be distinguished as different requests according to the field information, thereby accurately constructing a request mapping relationship, and storing the request mapping relationship in a preset cache area, so as to respond to the request operation of the first request, obtain the first request content for the first request, hash the first request content, and obtain the first hash value. If the preset cache area contains first data that matches the first hash value, the request mapping relationship corresponding to the first data that matches the first hash value is used as the reference request mapping relationship, and the second data in the reference request mapping relationship or the status code of the reference request mapping relationship is returned, thereby improving the efficiency of the system in processing high-concurrency data and improving system stability.

[0070] Example 3

[0071] Figure 3 This is a flowchart of an embodiment of the present invention providing a method for updating a request mapping relationship in a preset cache area. The technical solution of this embodiment provides a detailed description of updating the request mapping relationship in the preset cache area in the above embodiment based on the above embodiment. This embodiment can be combined with various optional solutions in one or more of the above embodiments. Figure 3 As shown, the method includes:

[0072] S310. Determine cache information of a preset cache area and obtain request attribute information of each request mapping relationship in the preset cache area; the cache information is used to reflect the size of resources occupied by the request mapping relationship; the request attribute information is used to reflect the access priority of the request corresponding to the request mapping relationship.

[0073] The cache information may include memory usage and / or CPU usage.

[0074] Specifically, the request attribute information can be determined based on the time interval between the requested node time and the current moment; the node time can be the time when the request was last initiated. Optionally, if the time interval between the requested node time and the current moment is less than a first preset time interval, the request attribute information is of the first priority; if the time interval between the requested node time and the current moment is greater than the first preset time interval, and the time interval between the requested node time and the current moment is less than a second preset time interval, the request attribute information is of the second priority; if the time interval between the requested node time and the current moment is greater than the second preset time interval, the request attribute information is of the third priority. The first priority is higher than the second priority, and the second priority is higher than the third priority.

[0075] For example, the first preset time interval is 3 days, and the second preset time interval is 7 days. The priority levels can be: First priority (highest): requests 0-3 days old are frequently accepted by the system, and the data in the database table associated with requests 0-3 days old is frequently accessed data. Second priority: requests 3-7 days old are less frequently accepted by the system, and the data in the database table associated with requests 3-7 days old is less frequently accessed data. Third priority (lowest): data in the database table associated with requests greater than 7 days old is historically infrequently accessed data.

[0076] In addition, the cache information of the preset cache area in the next stage can also be predicted based on the operating status of the system preset cache area in the current stage. Specifically, the cache information of the preset cache area in the next stage = α × current observation value + (1-α) × cache information of the preset cache area in the stage. The current observation value can be the memory usage and / or CPU usage of the preset cache area in the current stage; α is the optimal weight factor, which can be determined through historical data analysis. Furthermore, based on the cache information of the preset cache area in the next stage, the update strategy for the request mapping relationship in the preset cache area in the next stage is determined.

[0077] S320: Update the request mapping relationship in the preset cache area according to the cache information and the request attribute information.

[0078] Specifically, updating the request mapping relationship in the preset cache area according to the cache information and the request attribute information may include steps B1-B3:

[0079] Step B1. If the cache information is less than the first cache value, determine to use the first update strategy to update the request mapping relationship in the preset cache area; the first update strategy is to add a new request mapping relationship according to a preset ratio, and the preset ratio of the request mapping relationship for different request attribute information is different, and the higher the priority reflected by the request attribute information, the larger the preset ratio.

[0080] Specifically, the sum of the preset ratios of all request attribute information is 1. For example, if the request attribute information includes a first priority, a second priority, and a third priority; the first priority is higher than the second priority, and the second priority is higher than the third priority. The preset ratio of the first priority is higher than the preset ratio of the second priority, and the preset ratio of the second priority is higher than the preset ratio of the third priority.

[0081] Furthermore, if the cache information is less than the first cache value, it indicates that the current preset cache area is in a low-load state and new request mapping relationships need to be stored. Specifically, the target quantity can be determined based on the cache information. The target quantity is the number of request mapping relationships that can be stored in the preset cache area. Based on the preset ratios of the request attribute information and the target quantity, the number of request mapping relationships corresponding to each priority request is determined, thereby updating the request mapping relationships in the preset cache area. For example, the low-load state can correspond to: memory usage < 50% and / or CPU usage < 30%.

[0082] Step B2: If the cache information is greater than the first cache value and less than the second cache value, the request mapping relationships in the preset cache area are not updated.

[0083] Specifically, if the cache information is greater than the first cache value and less than the second cache value, it indicates that the current preset cache area is in a medium-load state, and the current state can be maintained. For example, the medium-load state can correspond to: 50% < memory usage < 80% and / or 30% < CPU usage < 70%.

[0084] Step B3: If the cache information is greater than the second cache value, it is determined to update the request mapping relationships in the preset cache area using a second update strategy. The second update strategy is to delete request mapping relationships according to the request attribute information until the cache information meets the preset conditions.

[0085] Specifically, if the cache information is greater than the second cache value, it indicates that the current preset cache area is in a high-load state and request mapping relationships need to be deleted, that is, request mapping relationships with lower priorities are deleted first. For example, if the request attribute information includes a first priority, a second priority, and a third priority; the first priority is higher than the second priority, and the second priority is higher than the third priority, then the request mapping relationships of the third priority are deleted first, and then the request mapping relationships of the second priority are deleted. The high-load state can correspond to: memory usage > 80% and / or CPU usage > 70%.

[0086] In this embodiment, optionally, the request mapping relationships stored in the preset cache area include a time validity period. The time validity period can be understood as the time of existence in the preset cache area. When the request mapping relationship reaches the time validity period, the request mapping relationship becomes invalid and is deleted from the preset cache area. The higher the priority of the request mapping relationship, the higher the time validity period.

[0087] The technical solution of an embodiment of the present invention determines the cache information of a preset cache area and obtains the request attribute information of each request mapping relationship in the preset cache area; the cache information is used to reflect the size of resources occupied by the request mapping relationship; the request attribute information is used to reflect the access priority of the request corresponding to the request mapping relationship, so as to accurately update the request mapping relationship in the preset cache area according to the cache information and the request attribute information, thereby ensuring the efficient allocation of resources in the preset cache area.

[0088] Example 4

[0089] Figure 4 This is a structural diagram of a high-concurrency data processing device provided by an embodiment of the present invention. This embodiment is applicable to processing high-concurrency data in a system. The high-concurrency data processing device can be implemented in the form of hardware and / or software. The high-concurrency data processing device can be configured in any electronic device with network communication function. Figure 4 As shown, the high-concurrency data processing device includes:

[0090] A mapping relationship determination module 410 is configured to construct request mapping relationships corresponding to different requests and store the request mapping relationships in a preset cache area; the request mapping relationships are mapping relationships between first data, a database table, and second data; the first data is a hash value obtained by hashing third data, and the third data is the request content corresponding to the request; the second data is related data extracted from the database table associated with the first data;

[0091] The data determination module 420 is configured to obtain, in response to the request operation of the first request, first request content for the first request, and hash the first request content to obtain a first hash value;

[0092] The data processing module 430 is used to use the request mapping relationship corresponding to the first data matching the first hash value as the reference request mapping relationship if there is first data matching the first hash value in the preset cache area, and return the second data in the reference request mapping relationship or the status code of the reference request mapping relationship.

[0093] Based on the above embodiments, optionally, the high-concurrency data processing device also includes a reference data processing module, which is used to: if the preset cache area does not have first data matching the first hash value, obtain the first database table associated with the first request content from the database according to the first request content; extract data associated with the first request content from the first database table as the second data of the first request, and return the second data of the first request; establish a request mapping relationship for the first request based on the first hash value, the first database table and the second data of the first request, and save the request mapping relationship of the first request to the preset cache area.

[0094] On the basis of the above embodiment, optionally, the mapping relationship determination module includes a data determination unit, a first judgment unit and a first mapping relationship construction unit;

[0095] a data determining unit, configured to determine a reference request based on request content corresponding to a first candidate request, and use a second hash value as first data corresponding to the reference request; the first candidate request being a preset number of requests accumulated by the system; the second hash value being a hash value obtained by hashing the request content corresponding to the reference request, each of the reference requests being a different request;

[0096] A first judgment unit is configured to extract, if field information exists in the request content corresponding to the reference request, preset field data from a database table associated with the reference request, and use the preset field data as second data corresponding to the reference request; the field information being identification information of the preset field data in the database table associated with the reference request;

[0097] The first mapping relationship construction unit constructs a first request mapping relationship corresponding to the reference request according to the first data corresponding to the reference request, identification information of a database table associated with the reference request, and the second data corresponding to the reference request.

[0098] On the basis of the above embodiment, optionally, the mapping relationship determination module further includes a second judgment unit and a second mapping relationship construction unit;

[0099] a second determining unit configured to extract, if no field information exists in the request content corresponding to the reference request, all data associated with the request content corresponding to the reference request from a database table associated with the reference request as second data corresponding to the reference request;

[0100] The second mapping relationship construction unit is configured to construct a second request mapping relationship corresponding to the reference request according to the first data corresponding to the reference request, identification information of a database table associated with the reference request, and the second data corresponding to the reference request.

[0101] Based on the above embodiment, optionally, the data determination unit is configured to:

[0102] Obtaining a preset number of first candidate requests accumulated by the system and request contents corresponding to the first candidate requests; the request contents are data strongly associated with business data;

[0103] Hash the request content corresponding to the first candidate request to obtain a candidate hash value corresponding to the first candidate request, group the first candidate requests with the same candidate hash value into a group and record them as a candidate request set, and determine the number of first candidate requests in each candidate request set;

[0104] Selecting a first candidate request from each set of candidate requests as a second candidate request, and determining request attribute information of the second candidate request according to the number of requests corresponding to the second candidate request; the request attribute information is used to reflect the access priority of the request;

[0105] The second candidate request corresponding to the request attribute information greater than the preset attribute information in the request attribute information is used as a reference request; the preset attribute information is used to limit the lower limit of the access priority of the request stored in the preset cache area.

[0106] Based on the above embodiment, optionally, each of the request mapping relationships in the preset cache area includes request attribute information, and the request attribute information is used to reflect the access priority of the request corresponding to the request mapping relationship. The high-concurrency data processing device also includes an update module, and the update module is used to determine the cache information of the preset cache area, and update the request mapping relationship in the preset cache area according to the cache information and the request attribute information; the cache information is used to reflect the size of the resources occupied by the request mapping relationship.

[0107] Based on the above embodiment, optionally, the update module is further configured to:

[0108] If the cache information is less than the first cache value, determining to adopt a first update strategy to update the request mapping relationship in the preset cache area; the first update strategy is to add a new request mapping relationship according to a preset ratio, the preset ratio of the request mapping relationship for different request attribute information is different, and the higher the priority reflected by the request attribute information, the larger the preset ratio;

[0109] If the cache information is greater than the first cache value and the cache information is less than the second cache value, the request mapping relationship in the preset cache area is not updated;

[0110] If the cache information is greater than the second cache value, it is determined to adopt a second update strategy to update the request mapping relationship in the preset cache area, and the second update strategy is to delete the request mapping relationship according to the request attribute information until the cache information meets the preset condition.

[0111] Based on the above embodiment, optionally, the data processing module includes a first data processing unit and a second data processing unit;

[0112] The first data processing unit is configured to return a status code of the reference request mapping relationship if it is detected that the first request carries instruction information; the instruction information is used to reflect that the user terminal has displayed the second data in the reference request mapping relationship,

[0113] The second data processing unit is configured to return the second data in the reference request mapping relationship and display the second data in the reference request mapping relationship if it is detected that the first request does not carry instruction information.

[0114] Based on the above embodiment, optionally, the request mapping relationship further includes a data correspondence relationship between the first data and the fourth data, and the fourth data is a hash value obtained by hashing the second data;

[0115] Correspondingly, the data processing module includes a status code return unit, which is used to: determine the fourth data of the reference request mapping relationship based on the data correspondence relationship and the first hash value in the reference request mapping relationship; determine the status code of the reference request mapping relationship based on the fourth data of the reference request mapping relationship, and return the status code of the reference request mapping relationship.

[0116] The high-concurrency data processing device provided in the embodiment of the present invention can execute the high-concurrency data processing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0117] Example 5

[0118] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0119] Figure 5A schematic diagram of the structure of an electronic device that can be used to implement the high-concurrency data processing method of an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0120] like Figure 5 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11. The memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. Various programs and data required for the operation of the electronic device 10 can also be stored in the RAM 13. The processor 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0121] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0122] The processor 11 can be various general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the high-concurrency data processing method.

[0123] In some embodiments, the high-concurrency data processing method can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the high-concurrency data processing method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to execute the high-concurrency data processing method in any other appropriate manner (for example, by means of firmware).

[0124] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0125] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0126] In the context of the present invention, computer-readable storage media can be tangible media that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Computer-readable storage media can include but are not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage media can be machine-readable signal media. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0127] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0128] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0129] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0130] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0131] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A high-concurrency data processing method, characterized in that: The method comprises: Constructing request mapping relationships corresponding to different requests and storing the request mapping relationships in a preset cache area; the request mapping relationships are mapping relationships between first data, a database table, and second data; the first data is a hash value obtained by hashing third data, and the third data is the request content corresponding to the request; the second data is relevant data extracted from the database table associated with the first data; In response to the request operation of the first request, obtaining first request content for the first request, and performing hash processing on the first request content to obtain a first hash value; If the preset cache area contains first data matching the first hash value, the request mapping relationship corresponding to the first data matching the first hash value is used as the reference request mapping relationship, and the second data in the reference request mapping relationship or the status code of the reference request mapping relationship is returned.

2. The method according to claim 1, characterized in that After obtaining the first hash value, the method further includes: If the preset cache does not contain first data matching the first hash value, obtaining a first database table associated with the first request content from a database according to the first request content; Extracting data associated with the first request content from the first database table as second data of the first request, and returning the second data of the first request; A request mapping relationship for the first request is established according to the first hash value, the first database table, and the second data of the first request, and the request mapping relationship of the first request is saved in a preset cache area.

3. The method according to claim 1, characterized in that Build request mapping relationships corresponding to different requests, including: Determine a reference request based on the request content corresponding to the first candidate request, and use the second hash value as the first data corresponding to the reference request; the first candidate request is a preset number of requests accumulated by the system; the second hash value is a hash value obtained by hashing the request content corresponding to the reference request, and each reference request is a different request; If field information exists in the request content corresponding to the reference request, extracting preset field data from the database table associated with the reference request, and using the preset field data as the second data corresponding to the reference request; the field information is identification information of the preset field data in the database table associated with the reference request; A first request mapping relationship corresponding to the reference request is constructed according to the first data corresponding to the reference request, identification information of a database table associated with the reference request, and the second data corresponding to the reference request.

4. The method according to claim 3, characterized in that After using the second hash value as the first data corresponding to the reference request, the method further includes: If the request content corresponding to the reference request does not contain field information, all data associated with the request content corresponding to the reference request will be extracted from the database table associated with the reference request as the second data corresponding to the reference request; A second request mapping relationship corresponding to the reference request is constructed according to the first data corresponding to the reference request, identification information of a database table associated with the reference request, and the second data corresponding to the reference request.

5. The method according to claim 3, characterized in that Determining a reference request according to the request content corresponding to the first candidate request includes: Obtaining a preset number of first candidate requests accumulated by the system and request contents corresponding to the first candidate requests; the request contents are data strongly associated with business data; Hash the request content corresponding to the first candidate request to obtain a candidate hash value corresponding to the first candidate request, group the first candidate requests with the same candidate hash value into a group and record them as a candidate request set, and determine the number of first candidate requests in each candidate request set; Selecting a first candidate request from each set of candidate requests as a second candidate request, and determining request attribute information of the second candidate request according to the number of requests corresponding to the second candidate request; the request attribute information is used to reflect the access priority of the request; The second candidate request corresponding to the request attribute information greater than the preset attribute information in the request attribute information is used as a reference request; the preset attribute information is used to limit the lower limit of the access priority of the request stored in the preset cache area.

6. The method according to claim 1, characterized in that Each of the request mapping relationships in the preset cache area includes request attribute information, where the request attribute information is used to reflect the access priority of the request corresponding to the request mapping relationship. The method further includes: Determine cache information of the preset cache area, and update the request mapping relationship in the preset cache area according to the cache information and the request attribute information; the cache information is used to reflect the size of resources occupied by the request mapping relationship.

7. The method according to claim 6, characterized in that Updating the request mapping relationship in the preset cache area according to the cache information and the request attribute information includes: If the cache information is less than the first cache value, determining to adopt a first update strategy to update the request mapping relationship in the preset cache area; the first update strategy is to add a new request mapping relationship according to a preset ratio, the preset ratio of the request mapping relationship for different request attribute information is different, and the higher the priority reflected by the request attribute information, the larger the preset ratio; If the cache information is greater than the first cache value and the cache information is less than the second cache value, the request mapping relationship in the preset cache area is not updated; If the cache information is greater than the second cache value, it is determined to adopt a second update strategy to update the request mapping relationship in the preset cache area, and the second update strategy is to delete the request mapping relationship according to the request attribute information until the cache information meets the preset condition.

8. The method according to claim 1, characterized in that Returns the second data in the reference request mapping relationship or the status code of the reference request mapping relationship, including: If it is detected that the first request carries instruction information, a status code of the reference request mapping relationship is returned; the instruction information is used to reflect that the user terminal has displayed the second data in the reference request mapping relationship; If it is detected that the first request does not carry instruction information, the second data in the reference request mapping relationship is returned and the second data in the reference request mapping relationship is displayed.

9. The method according to claim 8, characterized in that The request mapping relationship also includes a data correspondence between the first data and the fourth data, where the fourth data is a hash value of the second data after hash processing; Accordingly, the status code of the reference request mapping relationship is returned, including: Determining fourth data of the reference request mapping relationship according to the data correspondence relationship in the reference request mapping relationship and the first hash value; A status code of the reference request mapping relationship is determined according to the fourth data of the reference request mapping relationship, and the status code of the reference request mapping relationship is returned.

10. A high-concurrency data processing device, characterized in that: The device comprises: a mapping relationship determination module, configured to construct request mapping relationships corresponding to different requests and store the request mapping relationships in a preset cache area; the request mapping relationships being mapping relationships between first data, a database table, and second data; the first data being a hash value obtained by hashing third data, the third data being request content corresponding to the request; and the second data being relevant data extracted from the database table associated with the first data; a data determination module, configured to obtain, in response to a request operation of the first request, first request content for the first request, and hash the first request content to obtain a first hash value; A data processing module is used to, if there is first data matching the first hash value in the preset cache area, use the request mapping relationship corresponding to the first data matching the first hash value as a reference request mapping relationship, and return the second data in the reference request mapping relationship or the status code of the reference request mapping relationship.

11. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to execute the high-concurrency data processing method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the high-concurrency data processing method according to any one of claims 1 to 9 when executed.

Images