Data processing method, device, electronic device and storage medium

Abstract

The present disclosure relates to a data processing method, device, electronic device and storage medium, the method comprising: receiving a data query request; the data query request includes an object identifier to be queried; obtaining an accessible object identifier interval; the accessible object identifier interval includes a first accessible endpoint value and a second accessible endpoint value; the first accessible endpoint value is greater than the second accessible endpoint value; the first accessible endpoint value is determined according to the object identifier and a first estimated deviation value; the object identifier represents the current maximum object identifier obtained from a preset cache area; the first estimated deviation value is updated according to a preset period; when the object identifier to be queried is within the accessible object identifier interval, a query operation is performed according to the object identifier to be queried, and a data query result is returned. The data processing method of the present disclosure embodiment can solve the problem of database performance degradation and security reduction caused by malicious access in the related art, can effectively prevent the waste of server resources, and ensure the security and performance stability of the database.

Description

Technical Field

[0001] The present disclosure relates to the field of Internet technology, and in particular to a data processing method, device, electronic device and storage medium. Background Art

[0002] The object database that stores business object data is the core library of each business. For example, the live broadcast database that stores live broadcast room data is the core library of the live broadcast business. In actual business scenarios, operations to access the object database are very frequent. For example, many operations triggered by the client or applications using the business will generate logic to call and query the object database. When the object database fails, it will affect its core business. Therefore, it is necessary to screen requests to access the core object database to prevent malicious access from causing database performance degradation and security reduction. Summary of the invention

[0003] The present disclosure provides a data processing method, device, electronic device and storage medium. The technical solution of the present disclosure is as follows:

[0004] According to a first aspect of an embodiment of the present disclosure, there is provided a data processing method, including:

[0005] Receiving a data query request; the data query request includes an identifier of an object to be queried;

[0006] Obtain an accessible object identification interval; the accessible object identification interval includes a first accessible endpoint value and a second accessible endpoint value; the first accessible endpoint value is greater than the second accessible endpoint value; the first accessible endpoint value is determined according to the object identification and the first estimated deviation value; the object identification represents the current maximum object identification obtained from the preset cache area; the first estimated deviation value is updated according to a preset period;

[0007] When the identifier of the object to be queried is within the accessible object identifier range, the query operation is performed according to the identifier of the object to be queried, and the data query result is returned.

[0008] In some possible embodiments, the second accessible endpoint value is determined according to the object identifier and the second estimated deviation value; obtaining the accessible object identifier interval includes:

[0009] Calling the local cache of the application end to perform a local acquisition operation; the local acquisition operation is used to obtain the first accessible endpoint value and the second accessible endpoint value;

[0010] When the local acquisition operation is successfully executed, the first accessible endpoint value and the second accessible endpoint value are obtained; when the local acquisition operation fails, the remote cache area is called to obtain the first estimated deviation value and the second estimated deviation value from the remote cache area; the preset cache area is called to obtain the object identifier from the preset cache area; the first accessible endpoint value is determined based on the object identifier and the first estimated deviation value; the second accessible endpoint value is determined based on the object identifier and the second estimated deviation value.

[0011] In some possible embodiments, the data processing method further includes:

[0012] Acquire multiple object increments corresponding to multiple preset cycles; the object increment is determined according to the maximum object identifier corresponding to each preset cycle and the maximum object identifier corresponding to the previous cycle of each preset cycle; the multiple preset cycles include the current cycle;

[0013] The object increment with the largest value among the multiple object increments is used as the target object increment;

[0014] A first estimated deviation value corresponding to a current period is determined based on the target object increment.

[0015] In some possible embodiments, determining a first estimated deviation value corresponding to a current period based on a target object increment includes:

[0016] The object increment with the largest value is multiplied by the preset parameter to obtain the first estimated deviation value corresponding to the current period.

[0017] In some possible embodiments, before obtaining a plurality of object increments corresponding to a plurality of preset periods, the method further includes:

[0018] In each preset period, the object increment corresponding to the current preset period is determined according to the maximum object identifier corresponding to the current preset period and the maximum object identifier corresponding to the previous preset period before the current preset period;

[0019] The object increment corresponding to the current preset period and the timestamp corresponding to the current preset period are stored in the remote cache area so as to be obtained when the first estimated deviation value is updated according to the preset period.

[0020] In some possible embodiments, the remote cache area includes a plurality of slices, and each slice stores the first estimated deviation value and the second estimated deviation value;

[0021] Obtaining a first estimated deviation value and a second estimated deviation value from a remote buffer area includes:

[0022] A first estimated deviation value and a second estimated deviation value are obtained from any shard among the plurality of shards.

[0023] In some possible embodiments, performing a query operation according to the identifier of the object to be queried includes:

[0024] In the preset cache area, the corresponding object data is searched according to the object identifier to be searched, and a preliminary search result is obtained;

[0025] When the preliminary query result indicates success, the data query result is obtained; when the preliminary query result indicates failure, the object database is called to obtain the object data corresponding to the object identifier to be queried from the object database to obtain the data query result.

[0026] According to a second aspect of an embodiment of the present disclosure, there is provided a data processing device, including:

[0027] A receiving module is configured to receive a data query request; the data query request includes an identifier of an object to be queried;

[0028] The acquisition module is configured to execute acquisition of an accessible object identification interval; the accessible object identification interval includes a first accessible endpoint value and a second accessible endpoint value; the first accessible endpoint value is greater than the second accessible endpoint value; the first accessible endpoint value is determined according to the object identification and the first estimated deviation value; the object identification represents the current maximum object identification obtained from the preset cache area; the first estimated deviation value is updated according to a preset period;

[0029] The query module is configured to execute a query operation according to the object identifier to be queried when the object identifier to be queried is within the accessible object identifier interval, and return a data query result.

[0030] In some possible embodiments, the second accessible endpoint value is determined based on the object identifier and the second estimated deviation value;

[0031] The acquisition module is further configured to execute the local cache of the calling application end and perform a local acquisition operation; the local acquisition operation is used to obtain the first accessible endpoint value and the second accessible endpoint value;

[0032] The acquisition module is also configured to execute, when the local acquisition operation is successfully executed, to obtain the first accessible endpoint value and the second accessible endpoint value; when the local acquisition operation fails to execute, to call the remote cache area and obtain the first estimated deviation value and the second estimated deviation value from the remote cache area; to call the preset cache area and obtain the object identifier from the preset cache area; to determine the first accessible endpoint value based on the object identifier and the first estimated deviation value; and to determine the second accessible endpoint value based on the object identifier and the second estimated deviation value.

[0033] In some possible embodiments, the data processing device further includes:

[0034] An update module is configured to execute acquisition of multiple object increments corresponding to multiple preset periods; the object increment is determined based on the maximum object identifier corresponding to each preset period and the maximum object identifier corresponding to the previous period of each preset period; the multiple preset periods include the current period; the object increment with the largest value among the multiple object increments is used as the target object increment; and the first estimated deviation value corresponding to the current period is determined based on the target object increment.

[0035] In some possible embodiments, the updating module is further configured to execute multiplication of the object increment with the largest value by a preset parameter to obtain a first estimated deviation value corresponding to the current period.

[0036] In some possible embodiments, the update module is further configured to execute within each preset period, determine the object increment corresponding to the current preset period based on the maximum object identifier corresponding to the current preset period and the maximum object identifier corresponding to the previous preset period of the current preset period; store the object increment corresponding to the current preset period and the timestamp corresponding to the current preset period in a remote cache area, so as to obtain them when updating the first estimated deviation value according to the preset period.

[0037] In some possible embodiments, the remote cache area includes a plurality of slices, and each slice stores the first estimated deviation value and the second estimated deviation value;

[0038] The acquisition module is also configured to execute acquisition of a first estimated deviation value and a second estimated deviation value from any one of the multiple slices.

[0039] In some possible embodiments, the query module is also configured to execute in a preset cache area, query the object data corresponding to the object identifier to be queried, and obtain a preliminary query result; when the preliminary query result indicates success, obtain the data query result; when the preliminary query result indicates failure, call the object database, obtain the object data corresponding to the object identifier to be queried from the object database, and obtain the data query result.

[0040] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0041] processor;

[0042] a memory for storing processor-executable instructions;

[0043] Among them, the processor is configured to execute instructions to implement the data processing method provided in the first aspect of the embodiment of the present disclosure.

[0044] According to the fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the data processing method provided by the first aspect of the embodiment of the present disclosure.

[0045] According to the fifth aspect of an embodiment of the present disclosure, a computer program product is provided, the computer program product comprising a computer program, the computer program being stored in a readable storage medium, at least one processor of a computer device reading and executing the computer program from the readable storage medium, so that the computer device executes the data processing method provided in the first aspect of an embodiment of the present disclosure.

[0046] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0047] By receiving a data query request; the data query request includes the identifier of the object to be queried; obtaining an accessible object identifier interval; the accessible object identifier interval includes a first accessible endpoint value and a second accessible endpoint value; the first accessible endpoint value is greater than the second accessible endpoint value; the first accessible endpoint value is determined according to the object identifier and the first estimated deviation value; the object identifier represents the current maximum object identifier obtained from the preset cache area; the first estimated deviation value is updated according to a preset period; when the object identifier to be queried is within the accessible object identifier interval, performing a query operation according to the object identifier to be queried, and returning a data query result. In this way, the data processing method of the disclosed embodiment can solve the problem of database performance degradation and security reduction caused by malicious access in the related technology, and by identifying a large number of illegal data query requests, it can effectively prevent the waste of server resources and ensure the security and performance stability of the database.

[0048] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0050] Figure 1 is a schematic diagram of an application environment according to an exemplary embodiment;

[0051] Figure 2 is a flow chart of a data processing method according to an exemplary embodiment;

[0052] Figure 3 is a flow chart showing a data processing according to an exemplary embodiment;

[0053] Figure 4 is a schematic diagram showing an application scenario according to an exemplary embodiment;

[0054] Figure 5 is a flow chart showing a data processing according to an exemplary embodiment;

[0055] Figure 6 is a flow chart showing a data processing according to an exemplary embodiment;

[0056] Figure 7 is a block diagram of a data processing device according to an exemplary embodiment;

[0057] Figure 8 It is a block diagram of an electronic device for data processing according to an exemplary embodiment. DETAILED DESCRIPTION

[0058] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0059] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar first objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0060] It should be noted that the user information involved in this disclosure (including but not limited to user device information, user personal information, etc.) is all information authorized by the user or fully authorized by all parties.

[0061] In the relevant application scenario, the server providing a certain business service includes an object database, which is used to store the object data of each object involved in the business, so the object database usually faces a large number of accesses, such as query operations on target objects. In the related art, the server receives a large number of data query requests sent by different access terminals, and the data query request carries the identifier of the target object requested by the access terminal. The server responds to each data query request and performs a database query operation, that is, calls the object database to obtain the object data of the identifier of the target object from the object database. Under normal circumstances, the identifier of the target object carried by the data query request is real, such as the data query request issued by the legitimate client, and the server can normally execute the above-mentioned query database operation, obtain the corresponding object data and return it to the legitimate client; in the case of program abnormality or malicious attack by hackers, some abnormal data query requests will be triggered, that is, the identifier of the target object carried in the data query request does not actually exist, and the server will not obtain any valid data after executing the database query operation, but will cause the database's query rate per second (Queries PerSecond, QPS) to increase and the query hit rate to decrease, which affects the performance and security of the database.

[0062] Based on this, the present disclosure provides a data processing method to solve the problem of database performance degradation and security reduction caused by malicious access existing in the related art.

[0063] See also Figure 1 , Figure 1 It is a schematic diagram of an application environment of a data processing method according to an exemplary embodiment. The application environment may include a server 01 and a client 02. The server 01 may provide data query services to the client 02. The client 02 and the server 01 may be connected via a wired network or a wireless network.

[0064] In some possible embodiments, the server 01 receives a data query request from the client 02; the data query request includes an object identifier to be queried; the server 01 obtains an accessible object identifier interval; the accessible object identifier interval includes a first accessible endpoint value and a second accessible endpoint value; the first accessible endpoint value is greater than the second accessible endpoint value; the first accessible endpoint value is determined based on the object identifier and a first estimated deviation value; the object identifier represents the current maximum object identifier obtained from a preset cache area; the first estimated deviation value is updated according to a preset period; when the object identifier to be queried is within the accessible object identifier interval, the server 01 performs a query operation based on the object identifier to be queried, and returns the data query result to the client 02.

[0065] The server 01 may include an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The operating system running on the server may include but is not limited to Android, IOS, Linux, Windows, Unix, etc.

[0066] The client 02 may include but is not limited to a smartphone, a desktop computer, a tablet computer, a laptop computer, a smart speaker, a digital assistant, an augmented reality (AR) / virtual reality (VR) device, a smart wearable device, etc. It may also be software running on the client, such as an application, a small program, etc. Optionally, the operating system running on the client may include but is not limited to Android, IOS, Linux, Windows, Unix, etc.

[0067] In addition, it should be noted that Figure 1 What is shown is only one application environment of the data processing method provided by the present disclosure. In actual application, other application environments may also be included.

[0068] Figure 2 is a flow chart of a data processing method according to an exemplary embodiment. Figure 2 As shown, the data processing method can be applied to Figure 1 The server in the application scenario includes the following steps:

[0069] In step S201, a data query request is received; the data query request includes an identifier of an object to be queried.

[0070] In the embodiment of the present disclosure, the server includes a data query business service module, which executes steps S201 to S205 of the embodiment of the present disclosure; the server also includes an object database, which stores data of different objects, and the specific content of the data is related to the actual application scenario and the object. The object identifier to be queried is the identifier of the object to be queried, and the identifier is a feature that characterizes the uniqueness of the object, and the identifier may include a serial number or an ID number.

[0071] In some possible embodiments, the object identifier is unique characteristic information generated by the server using certain rules to distinguish the object from other objects when the object is generated. For example, an ID number is determined for the generated object according to an incremental rule based on the object generation time.

[0072] In actual application scenarios, such as live broadcast application scenarios, objects may include live broadcast rooms. Correspondingly, the object database stores live broadcast data of different live broadcast rooms, and the live broadcast data includes live broadcast room identification, anchor information, viewing user information, playback screen information, etc. The object identifier to be queried may be the ID number of the live broadcast room.

[0073] In the related technology, when providing services, the server needs to call the object database to obtain relevant object data, thereby triggering corresponding database operations; for example, in the live broadcast application scenario, the live broadcast application background server responds to each user entering the live broadcast room, calls the object database, and queries the live broadcast data corresponding to the live broadcast room.

[0074] In some possible embodiments, the demand for calling the object database to obtain relevant object data may come from the operation of the client user, that is, the data query request may be generated based on the operation of the client user.

[0075] Correspondingly, in this embodiment, the client determines the identifier of the object to be queried according to the client user operation, and then the client sends a data query request carrying the identifier of the object to be queried to the server; or, the client sends the user operation information to the server, and the preprocessing module of the server first determines the identifier of the object to be queried according to the user operation information, and generates a corresponding data query request based on the identifier of the object to be queried; then the preprocessing module sends the generated data query request to the data query business service module of the server, and the data query business service module continues to execute steps S201 to S205 of the embodiment of the present disclosure.

[0076] In some possible embodiments, the demand for calling the object database to obtain relevant object data may come from the server itself, that is, the server itself includes multiple different business service modules, and the above-mentioned object database is only the core database corresponding to one business service module among the multiple different business service modules (such as the data query business service module mentioned above). In actual applications, there is a demand for other business service modules of the server itself to call the data query business service module. Therefore, the data query request is generated by other business service modules of the server itself; of course, the demand for calling the object database to obtain relevant object data may also come from other servers, that is, other servers generate data query requests carrying the identifier of the object to be queried, and then send them to this server.

[0077] In the two possible embodiments mentioned above, under normal circumstances, the identifier of the object to be queried carried in the data query request is real, but there are some abnormal situations in actual applications, and the identifier of the object to be queried carried in the data query request does not exist at all. For example, the client user triggers the operation of entering a live broadcast room by clicking on the image of a live broadcast room displayed on the specified page. The client sends the user operation information to the server, and the server parses it according to the user operation information. During the parsing process, there may be a program error that causes the parsing to obtain an incorrect live broadcast room ID number, that is, the live broadcast room ID number does not exist at all, and the data query business service module will still use the live broadcast room ID number to query in the live broadcast room database; for example, hackers maliciously generate some data query requests, and the identifiers of the objects to be queried carried by these data query requests are compiled by hackers, with the purpose of dragging down the database performance of this server, and the data query business service module is currently unable to effectively distinguish which data query requests come from hackers. In the above abnormal situations, the limited resources of the server object database will be occupied by meaningless requests, resulting in a decrease in the performance of the object database and affecting the security of the object database.

[0078] Therefore, in the disclosed embodiment, after receiving the data query request, the data query business service module of the server identifies the identifier of the object to be queried in the data query request, that is, through the following steps S203 to S205, after the identifier of the object to be queried is determined to be an accessible object identifier, the object database is called to obtain the relevant object data. In this way, by performing a preliminary screening of the data query request, a large number of illegal data query requests can be identified, and the response to their requests can be cut off, which can effectively prevent the waste of server resources, reduce the database QPS, and improve the database query hit rate.

[0079] In step S203, an accessible object identification interval is obtained; the accessible object identification interval includes a first accessible endpoint value and a second accessible endpoint value; the first accessible endpoint value is greater than the second accessible endpoint value; the first accessible endpoint value is determined based on the object identification and the first estimated deviation value; the object identification represents the current maximum object identification obtained from a preset cache area; the first estimated deviation value is updated according to a preset period.

[0080] In the disclosed embodiment, the server is provided with an accessible object identification interval, and the accessible object identification interval is used to identify the object identification to be queried in the data query request; the accessible object identification interval includes a first accessible endpoint value and a second accessible endpoint value, and the first accessible endpoint value is greater than the second accessible endpoint value, that is, when the object identification to be queried is in the range of [second accessible endpoint value, first accessible endpoint value], it is considered that the corresponding data query request is legal, and the subsequent step S205 can be executed, otherwise the subsequent steps are stopped. Among them, the accessible object identification interval can be an open interval, a closed interval or a half-open interval, which is determined according to actual needs.

[0081] In the disclosed embodiment, the first accessible endpoint value serves as the upper bound of the accessible object identification interval, and the first accessible endpoint value is determined based on the object identification and the first estimated deviation value; wherein, the object identification represents the current maximum object identification obtained from the preset cache area, that is, the object identification is equal to the current maximum object identification. Since objects are continuously generated in actual applications, the current maximum object identification is also continuously increasing; as mentioned above, the object identification is generated by the server based on the generation time of the object and according to the incremental rule. Each time the server generates an object identification, it stores the object identification in the preset cache area to update the identification of the largest object currently stored in the preset cache area. In addition, the present disclosure takes into account that the frequency of object generation is very high, and new objects may also be generated in the process of storing the identifier of the current object in the preset cache area. Therefore, the current maximum object identifier obtained from the preset cache area is not actually the actual maximum object identifier. Therefore, the present disclosure does not directly set the current maximum object identifier as the first accessible endpoint value, but sets a first estimated deviation value. The first estimated deviation value represents the difference between the actual maximum object identifier and the current maximum object identifier. The first estimated deviation value is updated according to a preset period, and the specific update method will be described in detail below, which will not be repeated here. In addition, since the object identifier and the first estimated deviation value are dynamically changing, the first accessible endpoint value is also dynamically changing.

[0082] In the disclosed embodiment, the second accessible endpoint value serves as the lower bound of the accessible object identification interval, and the second accessible endpoint value is determined based on the object identification and the second estimated deviation value; wherein, the object identification is defined as described above; considering that in actual application scenarios, the demand for access to historical objects is not large, therefore, the server can obtain the second estimated deviation value based on experience summary, and the second estimated deviation value represents the difference between the current maximum object identification and the identification of the oldest object allowed to be accessed, and the object identification minus the second estimated deviation value is the second accessible endpoint value, that is, the identification of the oldest object allowed to be accessed. The second estimated deviation value can be a fixed value obtained based on experience summary, and thus the second accessible endpoint value can be fixed.

[0083] In some possible embodiments, the above step S203 of obtaining the accessible object identification interval may include: Figure 3 The following steps are shown:

[0084] In step 301, the local cache of the application is called to perform a local acquisition operation; the local acquisition operation is used to obtain the first accessible endpoint value and the second accessible endpoint value.

[0085] In step 303, it is determined whether the local acquisition operation is successfully executed.

[0086] If the local acquisition operation is successfully executed, the first accessible endpoint value and the second accessible endpoint value are obtained. If the local acquisition operation fails, steps S305 to S311 are executed.

[0087] The above-mentioned application end represents different objects in different application scenarios. The application end may refer to the present server or other servers.

[0088] like Figure 4 As shown, Figure 4 is a schematic diagram of an application scenario according to an exemplary embodiment, including multiple application terminals 401, a server terminal 402 and a remote cache area 403;

[0089] The local cache of the application end 401 stores the first accessible endpoint value and the second accessible endpoint value that were read from the remote cache area 403 in the past. The first accessible endpoint value and the second accessible endpoint value stored in the local cache of the application end 401 have a storage period, which is relatively short and can be 1 second. When the storage period is exceeded, the local cache of the application end 401 can automatically delete the record. In this way, it is possible to avoid multiple accesses to the remote cache area in a short period of time, directly read the first accessible endpoint value and the second accessible endpoint value with higher timeliness from the local cache, and realize fast reading of the accessible object identification interval, thereby shortening the identification time of the data query request and improving data processing efficiency.

[0090] In step S305, the remote cache area is called to obtain the first estimated deviation value and the second estimated deviation value from the remote cache area.

[0091] The remote cache area may include a Redis database. Figure 4 As shown, the server 402 stores the first estimated deviation value and the preset second estimated deviation value updated according to the preset period in the Redis database 403, so that different application terminals 401 can obtain them from them.

[0092] In step S307, the preset cache area is called to obtain the object identifier from the preset cache area.

[0093] In step S309, a first accessible endpoint value is determined according to the object identifier and the first estimated deviation value.

[0094] Specifically, the object identifier and the first estimated deviation value are added to obtain the first accessible endpoint value.

[0095] In step S311, a second accessible endpoint value is determined according to the object identifier and the second estimated deviation value.

[0096] Specifically, the second estimated deviation value is subtracted from the object identifier to obtain the second accessible endpoint value.

[0097] The above-mentioned preset cache area refers to the preset cache area of ​​this server. As mentioned above, each time the server generates an object identifier, it will store the object identifier in the preset cache area. Therefore, any application end that calls the preset cache area can obtain the current maximum object identifier, that is, the object identifier, from the preset cache area; then add the object identifier and the first estimated deviation value to obtain the first accessible endpoint value, and subtract the second estimated deviation value from the object identifier to obtain the second accessible endpoint value.

[0098] In the above embodiment, by setting a dynamically updated first accessible endpoint value and a fixed second accessible endpoint value as the upper and lower bounds of the accessible object identification interval, the accuracy of the judgment of the object identification to be queried can be improved. The dynamically updated first accessible endpoint value can avoid the misjudgment of the legitimate object identification to be queried, and the fixed second accessible endpoint value can intercept the access to the data of the long-ago object, so that the limited resources can be used to implement the current more important business logic, thereby realizing the rational and effective use of the server resources.

[0099] In the embodiments of the present disclosure, considering that in actual scenarios, different application terminals may have the need to obtain the first estimated deviation value and the second estimated deviation value at the same time, a single Key will be generated in the case of a large demand, which will cause a hot spot problem. Based on this, in some possible embodiments, the above-mentioned remote cache area can be set with multiple shards, and each shard stores the first estimated deviation value and the second estimated deviation value;

[0100] Then, correspondingly, the obtaining of the first estimated deviation value and the second estimated deviation value from the remote cache area in the above step S305 may specifically include: obtaining the first estimated deviation value and the second estimated deviation value from any slice among the multiple slices.

[0101] In a specific embodiment, taking any application end as an example, a shard information is randomly selected for the application end from the multiple currently optional shard information, the application end assembles a rediskey based on the randomly selected shard information, and uses the assembled rediskey to obtain the first estimated deviation value and the second estimated deviation value from the corresponding shard.

[0102] In this way, the first estimated deviation value and the second estimated deviation value are redundantly cached through multiple shards set in the remote cache area to cope with the problem of redis performance degradation caused by hot spots in sudden situations.

[0103] In some possible embodiments, the data processing method of the embodiment of the present disclosure further includes: updating the first estimated deviation value according to a preset period; wherein the preset period may be 1 minute, that is, calculating and obtaining a new first estimated deviation value every 1 minute;

[0104] The above-mentioned updating of the first estimated deviation value according to the preset period may include: Figure 5 The following steps are shown:

[0105] In step S501, a plurality of object increments corresponding to a plurality of preset periods are obtained; the object increment is determined according to the maximum object identifier corresponding to each preset period and the maximum object identifier corresponding to the previous period of each preset period; the plurality of preset periods include the current period.

[0106] When the preset period is 1 minute, steps S501 to S505 are performed every 1 minute to obtain an updated first estimated deviation value. Taking the first estimated deviation value corresponding to the current period as an example, first obtain multiple object increments corresponding to multiple preset periods including the current period, and the object increment is determined according to the maximum object identifier corresponding to each preset period and the maximum object identifier corresponding to the previous period of each preset period. Specifically, the object increment can be the difference between the maximum object identifier corresponding to each preset period and the maximum object identifier corresponding to the previous period of each preset period.

[0107] In step S503, the object increment with the largest value among the multiple object increments is used as the target object increment.

[0108] In step S505, a first estimated deviation value corresponding to the current period is determined based on the target object increment.

[0109] Among them, the specific number of multiple object increments can be determined according to actual needs; for example, if the object increments in the last hour are filtered out, then according to the duration of the preset cycle, it can be calculated that 1 hour corresponds to 60 object increments, and the object increment with the largest value is selected from the 60 object increments as the target object increment, and then the first estimated deviation value corresponding to the current cycle is determined based on the target object increment; thus, when subsequently determining the first accessible endpoint value of the accessible object identification interval, the first estimated deviation value corresponding to the current cycle is added to the current maximum object identification to obtain the latest first accessible endpoint value.

[0110] In a specific embodiment, the above-mentioned determination of the first estimated deviation value corresponding to the current period based on the target object increment may include: multiplying the object increment with the largest value by a preset parameter to obtain the first estimated deviation value corresponding to the current period.

[0111] In the above embodiment, considering that the growth rate of object increment is different in different time periods, in actual application, in a specific time period, the object increment per minute is larger than that in other time periods, so by counting multiple object increments corresponding to multiple periods, the target object increment is selected from the multiple object increments, and the first estimated deviation value corresponding to the current period is determined based on the target object increment. In this way, by dynamically adjusting the first estimated deviation value to obtain a more accurate first accessible endpoint value that fits the actual situation, the judgment accuracy of the object identifier to be queried can be improved to avoid the misjudgment of the legitimate object identifier to be queried.

[0112] In some possible embodiments, before step S501 obtains multiple object increments corresponding to multiple preset periods, the data processing method of the embodiment of the present disclosure may further include the following steps:

[0113] In step S5001, in each preset period, the object increment corresponding to the current preset period is determined according to the maximum object identifier corresponding to the current preset period and the maximum object identifier corresponding to the previous preset period before the current preset period;

[0114] In step S5003, the object increment corresponding to the current preset period and the timestamp corresponding to the current preset period are stored in the remote cache area so as to be obtained when the first estimated deviation value is updated according to the preset period.

[0115] Specifically, the object increment corresponding to the current cycle is obtained by subtracting the maximum object identifier corresponding to the previous cycle from the maximum object identifier corresponding to the current cycle; then the object increment corresponding to the current cycle and the timestamp corresponding to the current cycle are stored in the remote cache area in the zset data format, where the object increment corresponding to the current cycle and the maximum object identifier corresponding to the current cycle are the value values ​​in the zset data format, and the timestamp corresponding to the current cycle is the score value in the zset data format.

[0116] In each preset period, the object increment and timestamp corresponding to each preset period are stored in the remote cache in zset data format according to the above steps S5001 to S5003, so that in the subsequent step S503, the corresponding object increment can be selected based on the timestamp corresponding to each preset period.

[0117] The above embodiment is explained below through a specific example. Assume that the maximum object identifier corresponding to the current cycle is 10000, and the maximum object identifier corresponding to the previous cycle of the current cycle is 9000; then, the object increment corresponding to the current cycle is 1000, and accordingly, the value in the zset data corresponding to the current cycle can be expressed as 1000_10000, and the timestamp is assumed to be 10:00:00; below, the zset data corresponding to multiple cycles stored in the remote cache area are given as an example:

[0118] "value": "1000_10000", "score": "10:00:00";

[0119] "value": "2000_9000", "score": "9:59:00";

[0120] "value": "500_7000", "score": "9:58:00";

[0121] …

[0122] "value": "1500_3000", "score": "9:01:00"

[0123] For the convenience of explanation, the above example only shows the zset data of 4 periods within 1 hour in detail, and the remaining 56 are not shown; assuming that the largest object increment in the 60 periods within 1 hour is the object increment of 2000 corresponding to the previous period of the current period, then 2000 is taken as the target object increment, and 2000 is multiplied by the preset parameter to obtain the first estimated deviation value corresponding to the current period.

[0124] In step S205, when the object identifier to be queried is within the accessible object identifier interval, a query operation is performed according to the object identifier to be queried, and a data query result is returned.

[0125] In the disclosed embodiment, the data query business service module of the server determines whether the identifier of the object to be queried is within the accessible object identifier interval; if the identifier of the object to be queried is within the accessible object identifier interval, it indicates that the identifier of the object to be queried actually exists, that is, the data query request is legal, then the query operation is performed according to the identifier of the object to be queried, and then the data query result is returned to the access end (client / other business service modules of this server / other server); alternatively, if the identifier of the object to be queried is not within the accessible object identifier interval, it indicates that the identifier of the object to be queried does not actually exist, that is, the data query request is illegal, then the subsequent steps are not executed, and the data processing is terminated.

[0126] In some possible embodiments, it is determined whether the object identifier to be queried is within the accessible object identifier interval, that is, it is determined whether the object identifier to be queried is greater than or equal to the second accessible endpoint value and less than or equal to the first accessible endpoint value.

[0127] In some possible embodiments, the query operation performed according to the identifier of the object to be queried may include: Figure 6 The following steps are shown:

[0128] In step S601, in a preset cache area, corresponding object data is searched according to the object identifier to be searched, and a preliminary search result is obtained.

[0129] The preset cache area of ​​the server stores the object identifiers to be queried and the corresponding object data that have been successfully queried in the past. Therefore, the data query business service module can first perform a preliminary query in the preset cache area, that is, query whether the object identifier to be queried already exists and obtain preliminary query results.

[0130] In step S603, it is determined whether the preliminary query result indicates success.

[0131] If the preliminary query result indicates success, the data query result is obtained; if the preliminary query result indicates failure, step S605 is executed.

[0132] Specifically, if the current object identifier query to be queried already exists, the preliminary query result includes the object data corresponding to the current object identifier query to be queried, indicating that the query is successful, and the preliminary query result is the final data query result. If the current object identifier query to be queried does not exist, the preliminary query result includes indication information that the query failed, and step S605 is executed.

[0133] In step S605, the object database is called to obtain object data corresponding to the object identifier to be queried from the object database to obtain a data query result.

[0134] If the preset cache does not contain the current object identifier to be queried, the data query business service module returns to the source object database, calls the object database, obtains the corresponding object data from the object database according to the current object identifier to be queried, and obtains the data query result.

[0135] In the above embodiment, the identifiers of the objects to be queried and the corresponding object data that have been successfully queried in the past are stored in the preset cache area, which can quickly respond to data query requests for the same object to be queried, that is, the server directly reads the data query results from the preset cache area without having to repeatedly perform multiple query operations, which can reduce the access frequency of the object database. In this way, the data reading efficiency can be improved, thereby improving the response speed of the server to the request, and at the same time ensuring the performance stability of the object database. In a further optional embodiment, the preset cache area can set a storage time for the identifiers of the objects to be queried and the corresponding object data that have been successfully queried in the past. After the storage time is exceeded, the preset cache area automatically deletes the records of the corresponding object data, thereby releasing the cache to avoid resource occupation.

[0136] In summary, in the disclosed embodiment, the data query business service module of the server side identifies the identifier of the object to be queried in the data query request through the above steps S201 to S205, and only after the identifier of the object to be queried is determined to be an accessible object identifier, it executes the call to the object database to obtain the relevant object data, thereby realizing the preliminary screening of the data query request. In this way, a large number of illegal data query requests can be identified and the response to their requests can be cut off, thereby effectively preventing the waste of server resources, ensuring the security of the database, reducing the database QPS, and improving the database query hit rate.

[0137] Figure 7 is a block diagram of a data processing device according to an exemplary embodiment. Figure 7 , the device includes a receiving module 701, an acquiring module 702 and a query module 703;

[0138] The receiving module 701 is configured to receive a data query request; the data query request includes an identifier of an object to be queried;

[0139] The acquisition module 702 is configured to execute acquisition of an accessible object identification interval; the accessible object identification interval includes a first accessible endpoint value and a second accessible endpoint value; the first accessible endpoint value is greater than the second accessible endpoint value; the first accessible endpoint value is determined according to the object identification and the first estimated deviation value; the object identification represents the current maximum object identification obtained from the preset cache area; the first estimated deviation value is updated according to a preset period;

[0140] The query module 703 is configured to execute a query operation according to the object identifier to be queried when the object identifier to be queried is within the accessible object identifier interval, and return a data query result.

[0141] In some possible embodiments, the second accessible endpoint value is determined based on the object identifier and the second estimated deviation value;

[0142] The acquisition module 702 is further configured to execute the local cache of the calling application end and perform a local acquisition operation; the local acquisition operation is used to obtain the first accessible endpoint value and the second accessible endpoint value;

[0143] The acquisition module 702 is also configured to execute, when the local acquisition operation is successfully executed, to obtain the first accessible endpoint value and the second accessible endpoint value; when the local acquisition operation fails to execute, to call the remote cache area and obtain the first estimated deviation value and the second estimated deviation value from the remote cache area; to call the preset cache area and obtain the object identifier from the preset cache area; to determine the first accessible endpoint value based on the object identifier and the first estimated deviation value; and to determine the second accessible endpoint value based on the object identifier and the second estimated deviation value.

[0144] In some possible embodiments, the data processing device further includes:

[0145] An update module is configured to execute acquisition of multiple object increments corresponding to multiple preset periods; the object increment is determined based on the maximum object identifier corresponding to each preset period and the maximum object identifier corresponding to the previous period of each preset period; the multiple preset periods include the current period; the object increment with the largest value among the multiple object increments is used as the target object increment; and the first estimated deviation value corresponding to the current period is determined based on the target object increment.

[0146] In some possible embodiments, the updating module is further configured to execute multiplication of the object increment with the largest value by a preset parameter to obtain a first estimated deviation value corresponding to the current period.

[0147] In some possible embodiments, the update module is further configured to execute within each preset period, determine the object increment corresponding to the current preset period based on the maximum object identifier corresponding to the current preset period and the maximum object identifier corresponding to the previous preset period of the current preset period; store the object increment corresponding to the current preset period and the timestamp corresponding to the current preset period in a remote cache area, so as to obtain them when updating the first estimated deviation value according to the preset period.

[0148] In some possible embodiments, the remote cache area includes a plurality of slices, and each slice stores the first estimated deviation value and the second estimated deviation value;

[0149] The acquisition module 702 is further configured to acquire a first estimated deviation value and a second estimated deviation value from any slice among the multiple slices.

[0150] In some possible embodiments, the query module 703 is also configured to execute in a preset cache area, query the object data corresponding to the object identifier to be queried, and obtain a preliminary query result; when the preliminary query result indicates success, obtain the data query result; when the preliminary query result indicates failure, call the object database, obtain the object data corresponding to the object identifier to be queried from the object database, and obtain the data query result.

[0151] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0152] Figure 8 It is a block diagram of an electronic device for data processing according to an exemplary embodiment.

[0153] The electronic device may be a server or a terminal device, and its internal structure diagram may be as follows: Figure 8 As shown. The electronic device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a data processing method is implemented.

[0154] Those skilled in the art will understand that Figure 8 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present disclosure, and does not constitute a limitation on the electronic device to which the scheme of the present disclosure is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0155] In an exemplary embodiment, an electronic device is also provided, including: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the data processing method in the embodiment of the present disclosure.

[0156] In an exemplary embodiment, a computer-readable storage medium is also provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the data processing method in the embodiment of the present disclosure.

[0157] In an exemplary embodiment, a computer program product is also provided, which includes a computer program. The computer program is stored in a readable storage medium. At least one processor of a computer device reads and executes the computer program from the readable storage medium, so that the computer device executes the data processing method of the embodiment of the present disclosure.

[0158] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0159] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0160] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A data processing method, characterized in that: include: Receiving a data query request; the data query request includes an identifier of an object to be queried; Get the accessible object identification interval; The accessible object identification interval includes a first accessible endpoint value and a second accessible endpoint value; the first accessible endpoint value is greater than the second accessible endpoint value; the first accessible endpoint value is determined according to the object identification and the first estimated deviation value; the object identification represents the current maximum object identification obtained from the preset cache area; the first estimated deviation value is updated according to a preset period; When the to-be-queried object identifier is within the accessible object identifier interval, performing a query operation according to the to-be-queried object identifier and returning a data query result; The first estimated deviation value is obtained in the following manner: Acquire multiple object increments corresponding to multiple preset periods; the multiple object increments are determined according to the maximum object identifier corresponding to each preset period and the maximum object identifier corresponding to the previous period of each preset period; the multiple preset periods include the current period; Taking the object increment with the largest value among the multiple object increments as the target object increment; The target object increment is multiplied by a preset parameter to obtain the first estimated deviation value corresponding to the current period.

2. The data processing method according to claim 1, characterized in that: The second accessible endpoint value is determined according to the object identifier and the second estimated deviation value; and obtaining the accessible object identifier interval includes: Calling the local cache of the application end to perform a local acquisition operation; the local acquisition operation is used to obtain the first accessible endpoint value and the second accessible endpoint value; In the case where the local acquisition operation is successfully executed, obtaining the first accessible endpoint value and the second accessible endpoint value; In the event that the local acquisition operation fails, the remote cache area is called to obtain the first estimated deviation value and the second estimated deviation value from the remote cache area; the preset cache area is called to obtain the object identifier from the preset cache area; the first accessible endpoint value is determined based on the object identifier and the first estimated deviation value; the second accessible endpoint value is determined based on the object identifier and the second estimated deviation value.

3. The data processing method according to claim 1, characterized in that: Before obtaining a plurality of object increments corresponding to a plurality of preset periods, the method further includes: In each preset period, determining the object increment corresponding to the current preset period according to the acquired maximum object identifier corresponding to the current preset period and the maximum object identifier corresponding to the previous preset period before the current preset period; The object increment corresponding to the current preset period and the timestamp corresponding to the current preset period are stored in a remote cache area so as to be obtained when the first estimated deviation value is updated according to the preset period.

4. The data processing method according to claim 2, characterized in that: The remote cache area includes a plurality of slices, and each slice stores the first estimated deviation value and the second estimated deviation value; The acquiring the first estimated deviation value and the second estimated deviation value from the remote buffer area includes: The first estimated deviation value and the second estimated deviation value are obtained from any slice among the multiple slices.

5. The data processing method according to claim 1, characterized in that: The performing a query operation according to the identifier of the object to be queried includes: In the preset cache area, the corresponding object data is searched according to the to-be-queried object identifier to obtain a preliminary search result; If the preliminary query result indicates success, obtaining the data query result; In the case where the preliminary query result indicates a failure, the object database is called to obtain object data corresponding to the object identifier to be queried from the object database to obtain the data query result.

6. A data processing device, characterized in that: include: A receiving module is configured to execute and receive a data query request; The data query request includes an identifier of an object to be queried; An acquisition module is configured to execute acquisition of an accessible object identification interval; the accessible object identification interval includes a first accessible endpoint value and a second accessible endpoint value; the first accessible endpoint value is greater than the second accessible endpoint value; the first accessible endpoint value is determined according to an object identification and a first estimated deviation value; the object identification represents a current maximum object identification obtained from a preset cache area; the first estimated deviation value is updated according to a preset period; A query module, configured to execute a query operation according to the to-be-queried object identifier when the to-be-queried object identifier is within the accessible object identifier interval, and return a data query result; The first estimated deviation value is obtained by using the following modules: The update module is configured to execute the acquisition of multiple object increments corresponding to multiple preset periods; the multiple object increments are determined according to the maximum object identifier corresponding to each preset period and the maximum object identifier corresponding to the previous period of each preset period; the multiple preset periods include the current period; and the object increment with the largest value among the multiple object increments is used as the target object increment; The target object increment is multiplied by a preset parameter to obtain the first estimated deviation value corresponding to the current period.

7. The data processing device according to claim 6, characterized in that: The second accessible endpoint value is determined according to the object identifier and the second estimated deviation value; The acquisition module is further configured to execute a call to a local cache of the application end and perform a local acquisition operation; the local acquisition operation is used to obtain the first accessible endpoint value and the second accessible endpoint value; The acquisition module is further configured to obtain the first accessible endpoint value and the second accessible endpoint value when the local acquisition operation is successfully executed; to call the remote cache area to obtain the first estimated deviation value and the second estimated deviation value from the remote cache area when the local acquisition operation fails; and to call the preset cache area to obtain the object identifier from the preset cache area. Determining the first accessible endpoint value according to the object identifier and the first estimated deviation value; The second accessible endpoint value is determined according to the object identifier and the second estimated deviation value.

8. The data processing device according to claim 6, characterized in that: The updating module is further configured to determine, within each preset period, an object increment corresponding to the current preset period according to the acquired maximum object identifier corresponding to the current preset period and the maximum object identifier corresponding to the previous preset period of the current preset period; The object increment corresponding to the current preset period and the timestamp corresponding to the current preset period are stored in a remote cache area so as to be obtained when the first estimated deviation value is updated according to the preset period.

9. The data processing device according to claim 7, characterized in that: The remote cache area includes a plurality of slices, and each slice stores the first estimated deviation value and the second estimated deviation value; The acquisition module is further configured to acquire the first estimated deviation value and the second estimated deviation value from any slice among the multiple slices.

10. The data processing device according to claim 6, characterized in that: The query module is further configured to execute in the preset cache area, query the corresponding object data according to the to-be-queried object identifier, and obtain a preliminary query result; if the preliminary query result indicates success, obtain the data query result; In the case where the preliminary query result indicates a failure, the object database is called to obtain object data corresponding to the object identifier to be queried from the object database to obtain the data query result.

11. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the data processing method according to any one of claims 1 to 5.

12. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the data processing method as claimed in any one of claims 1 to 5.

13. A computer program product, characterized in that The computer program product comprises a computer program, which is stored in a readable storage medium. At least one processor of a computer device reads and executes the computer program from the readable storage medium, so that the computer device performs the data processing method according to any one of claims 1 to 5.

Images