44 results about "Queries per second" patented technology

The embodiment of the invention provides a server system expansion method and apparatus and an expansion processing device. The method comprises the following steps: for a non access layer server, determining a server expansion number of each service cluster; after the server expansion number of each service cluster of the non access layer server is determined, and the non access layer server is expanded, simulating a business request in an access layer service cluster; in a process of simulating the service request, if the query rate per second QPS corresponding to the access layer service cluster cannot be improved by increasing the flow of the business request, after the number of the access servers of the access layer service cluster is increased according to a preset number, judging whether the QPS is increased; and if so, determining the number of the access servers of the access layer service cluster, which needs to be increased according to the set number again, until the QPS reaches a set QPS value. The expansion manner provided by the invention is relatively rational, the condition of resource waste of the server of the expanded server system can be reduced.

The embodiment of the invention discloses a data searching method and device. The method comprises the steps that the query per second is calculated by a target server receiving search requests; when the query per second is smaller than an upper threshold value of the query per second, the target server searches for target data meeting the preset requirements in a local database; when the target data is found, the target data is returned to a target client; when the target data is not found, a second server cluster is made to search for the target data in a local cluster database; the second server cluster feeds the found target data back to the target server, and then the target data is returned back to the target client; under the situation that the second server cluster does not find the target data, and the query per second is smaller than a lower threshold value of the query per second, a third server cluster searches for the target data in the local cluster database, and the target data is fed back to the target server; the target server returns the target data to the target client. By means of the data searching method and device, the stability and effectiveness of search service can be improved.

The invention discloses a method and a device for template rendering. The method includes the following steps: acquiring a corresponding template and generating a corresponding Java-type file according to a web page browsing request of a user, and calling a confirmed variable type and / or a confirmed variable operation method in the template by using a Java direct calling method; and then, executing the Java-type file to generate a corresponding web page, and returning the wed page to the user. Thus, code execution efficiency in template switching can be effectively improved, the number of query per second of a service platform can be improved, response time is shortened, and therefore garbage recovery of a Java virtual machine of a system can be reduced, then performance of a whole application system can be improved, and performance bottlenecks caused by the template rendering can be avoided.

The invention discloses a bullet screen forbidding system and a bullet screen forbidding method for live video websites, and relates to the field of bullet screen type live videos. The bullet screen forbidding system comprises a forbidden-post recording module, a main remote service module, an authority management writing module, a plurality of authority reading modules and a plurality of auxiliary remote service modules. The bullet screen forbidding method includes that a user posts an illegal bullet screen, an administrator sends a post forbidding request to the authority management writing module, and the authority management writing module records a corresponding post forbidding state in the main remote service module according to an operation type value in the post forbidding request and synchronizes the post forbidding state to all the auxiliary remote service modules; the user logins in a room to send a query request to an authority unit in each authority reading module, selects the second auxiliary remote service module to send the query request and acquires the queried post forbidding state, wherein users who are forbidden to post bullet screens cannot post the bullet screens at corresponding positions within post-forbidding periods. The bullet screen forbidding system and the bullet screen forbidding method have the advantages that server workload is reduced, servers are added directly through transverse extension to meet QPS (query per second), and new server development is not needed.

Embodiments of the invention disclose a QPS control method and device for a server and electronic equipment, relate to the technical field of computers and networks, and solve a problem of QPS controlunder a large-scale server cluster. The QPS control method for the server is used for the servers in a server cluster. The method comprises the following steps of receiving the total number of servercluster requests in designated time counted in advance; computing a ratio of the number of requests received locally within the designated time to the total number of server cluster requests; and computing a first QPS control value according to the ratio and a cluster query per second (QPS) limit value required by a demand side platform. According to the method, the device and the electronic equipment, a network usage rate can be improved, and the method, the device and the electronic equipment are suitable for QPS control of various server clusters.

The embodiment of the invention provides a data splitting method and device for a distributed storage system. The method comprises the steps that whether query per second (QPS) of all data fragments is greater than a first preset threshold value is monitored; the data fragment with the QPS greater than the first preset threshold value is determined as a target data fragment, access requests of the target data fragment are sampled, and a sampled data stream of index key values corresponding to the sampled access requests is formed; whether the QPS of the target data fragment is greater than a second preset threshold value is judged; if yes, a median of the index key values in the sampled data stream in each preset time period is determined; a median of the index key values in a next preset time period corresponding to the current moment is obtained through prediction according to the determined median, and the median obtained through prediction is determined as a splitting point; and the target data fragment is split according to the splitting point. When the data fragments on access request hotspots are split through the scheme, load balance of the access requests can be realized.

The invention discloses a resource downloading system and method, and a crawler downloading system. The resource downloading system limits a capture frequency of a crawler to a site through a token bucket flow limiting method; a value range of a timestamp increase value of a token generated by a random token generator is determined according to the rate of query per second of the crawler to the site; and the timestamp of the token is determined according to a timestamp at a current moment and the timestamp increase value, so that the timestamp increase value of the token is limited to be a random value, the capture frequency of the crawler to the site is limited to be a random frequency, an anti-crawler program is prevented from identifying the crawler according to the characteristic, namely, a fixed capture frequency, the probability of identifying the crawler by the anti-crawler program is reduced, and the functional stability of the crawler is improved.

The disclosed embodiments provide a system for processing data. During operation, the system determines a current incoming queries per second (QPS) to one or more components for processing queries of a graph database, wherein the graph database is replicated across multiple clusters and distributed among a set of storage nodes in each of the clusters. Next, the system uses the current incoming QPS to estimate, for the one or more components, an expected QPS associated with fanning out of the queries to the clusters. The system then selects a number of clusters in the multiple clusters for fanning out of a query based on the expected QPS and one or more throughput limits for the one or more components. Finally, the system transmits the query to one or more of the storage nodes in the selected number of clusters.

The invention discloses a domain name system (DNS) capacity measurement method, device and equipment and a medium. The method comprises the following steps: acquiring historical log information of a target DNS; extracting at least one domain name and a timestamp corresponding to the domain name in the historical log information; generating a test script according to at least one domain name and the timestamp corresponding to the domain name; sending request information to the target DNS by using the test script, so that the target DNS analyzes the request information; receiving an analysis result of the target DNS, and calculating a first preset index according to the analysis result; when the first preset index meets the first preset condition, taking the query per second QPS corresponding to the target DNS as the actual capacity of the target DNS. According to the embodiment of the invention, the actual capacity of the target DNS can be measured in a simulated current network environment as real as possible.

The invention discloses a realization system and method for video subscription dynamic states and relates to the field of Internet video-on-demand. The method comprises a plurality of clients, a plurality of slave remote dictionary servers, a master remote dictionary server and a subscription dynamic state service cluster. The subscription dynamic state service cluster comprises at least one service unit. Video authors upload video request data comprising video uploading time stamps of videos through the clients. Users query the subscription dynamic states through the clients; query request data is sent to one service unit; one slave remote dictionary server for providing query is selected; each video author ID subscribed by the user is traversed; the corresponding latest video uploading time stamps are obtained; if the video uploading time stamps are greater than the last access time stamps of the users, it indicates that the users have the subscription dynamic states. According to the system and the method, the update of the subscription dynamic states of partial users can be reduced, the workloads of the servers is reduced, the speeds of the servers are improved, and the QPS (Query Per Second) can be satisfied by directly increasing new servers in a transverse expansion mode.

The invention provides a method for acquiring Mysql database monitoring data by using Python language. The method comprises the following steps: acquiring a machine name of a host through Python; putting the connection information of a plurality of Mysql databases into a character string array; connecting each Mysql database, inquiring each index of the database, and putting into a Python dictionary; obtaining Mysql monitoring indexes, and extracting values of a plurality of important monitoring indexes needing to be collected from the Mysql monitoring index, mainly comprising the query rate per second, the system throughput, the slow query number per second, the total number of established connections, the number of connections in operation, the number of rejected connections and the buffer pool index; calculating required monitoring index values; and arranging the calculated values of the monitoring items into a storage format of a time sequence database, and forwarding the values toa monitoring system.

The invention provides an online state data processing method, device and system for an online system. The method comprises the following steps that a data diffusion layer receives online state change messages of a client; the data diffusion layer distributes the online state change messages to a data distribution layer of each subarea according to a preset redundancy strategy; the data distribution layers forward the online state change messages to data storage layer groups of data storage layers of the subareas corresponding to the data distribution layers according to a preset distribution strategy, wherein each data storage layer comprises more than two data storage layer groups; and the data storage layer groups store the online state change messages. According to the method, parallel expansion can be realized conveniently; the bottleneck problem of query per second QPS and system storage capacity is solved; full-dose redundancy storage is carried out on data among the subareas; disaster recovery backup is realized; and the system reliability is improved.

The invention relates to an access request processing method and device, a server and a storage medium. The method comprises the following steps: if a preset condition is satisfied, determining a first global query per second rate in the current current limiting period, the preset condition including that the total number of at least one access server reporting the query per second rate accordingto a preset period reaches a preset number, or the total number of the query per second rates reported by the at least one access server according to the preset period reaches a first threshold; determining a second global per-second query rate in a next current limiting period according to the first global per-second query rate, wherein the next current limiting period is located after the current current limiting period in time sequence and is adjacent to the current current limiting period; determining a target overload parameter matched with the second global query rate per second; and feeding back the target overload parameter to the at least one access server, wherein the target overload parameter is used for the at least one access server to perform flow limiting processing on the access request from the user terminal. According to the method of the invention, effective access flow limiting can be realized.

The embodiment of the invention discloses a request control method, which is applied to a working node in a cluster, the working node is used for processing a request, and the method comprises the following steps: obtaining a query rate per second of the working node for processing the request; reporting the per-second query rate of the working node to a management server, so that the management server determines whether to trigger a control instruction or not according to the per-second query rate of the working node; and receving the control instruction issued by the management server, processing the current to-be-processed request according to the control instruction, wherein the control instruction is an instruction determined to be triggered by the management server according to the query rate per second of the working node. The method in the embodiment of the invention can give full play to the cluster performance, and does not cause resource waste.

The invention proposes a network asynchronous grasping system and method. The network asynchronous grasping system comprises a task queue manager used for providing at least one task queue; a scheduler used for reading a uniform resource locator URL of a network content to be grasped from each task queue, and triggering a driver to schedule the URL according to the environment type of a back end where a task to which the URL belongs locates; the driver used for reading task information of the task of the URL after being triggered by the scheduler, injecting the URL into a grasping pool based on the task information, and controlling the frequency of injecting the URL into the grasping pool according to the task information, wherein the task information comprises the query per second and a concurrency value; and an actuator used for reading the URL from the grasping pool, and grasping the URL. The invention can ensures the stability of the grasping system during high concurrency, effectively save system resources and improve the grasping performance.

The invention discloses a cache elimination method, and the method comprises the steps: setting a first queue and a second queue, wherein the first queue is used for maintaining cache data and a unique code corresponding to each piece of data, and the second queue is used for maintaining the code and the query rate per second of each piece of data; receiving the code of the data to be queried; and initiating a query operation to the first queue according to the code, updating the first queue and the second queue when the code is not queried, and eliminating the cold data in the first queue. The invention further discloses a cache elimination system, an electronic device and a computer readable storage medium. Therefore, the hot data set, the warm data set and the cold data set in the queue can be effectively distinguished, the elimination accuracy is improved, and cache data of historical access records can be eliminated without a large amount of data access.

The disclosed embodiments provide a system for processing data. During operation, the system determines a current incoming queries per second (QPS) to one or more components for processing queries of a graph database, wherein the graph database is replicated across multiple clusters and distributed among a set of storage nodes in each of the clusters. Next, the system uses the current incoming QPS to estimate, for the one or more components, an expected QPS associated with fanning out of the queries to the clusters. The system then selects a number of clusters in the multiple clusters for fanning out of a query based on the expected QPS and one or more throughput limits for the one or more components. Finally, the system transmits the query to one or more of the storage nodes in the selected number of clusters.

The embodiment of the invention discloses a data processing method and related equipment, which are used for adjusting the maximum concurrency of services to ensure that hardware resources are utilized as much as possible while a server is not in fault due to overload. The method comprises the following steps: acquiring a sampling sample in a current time window; determining a first query per second QPS of the current time window according to the sampling sample in the current time window; updating the limit QPS of the target time window according to the first QPS and a preset QPS updating strategy; determining a first average time delay of the sampling samples in the current time window according to the sampling samples in the current time window; according to the first average time delayand a preset average time delay updating strategy, updating a currently stored target average time delay at a low load; determining the target maximum concurrency according to the limit QPS of the target time window and the updated target average time delay; and adjusting the maximum concurrency of the target time window based on the target maximum concurrency.

The invention discloses a method and a device for template rendering. The method includes the following steps: acquiring a corresponding template and generating a corresponding Java-type file according to a web page browsing request of a user, and calling a confirmed variable type and / or a confirmed variable operation method in the template by using a Java direct calling method; and then, executing the Java-type file to generate a corresponding web page, and returning the wed page to the user. Thus, code execution efficiency in template switching can be effectively improved, the number of query per second of a service platform can be improved, response time is shortened, and therefore garbage recovery of a Java virtual machine of a system can be reduced, then performance of a whole application system can be improved, and performance bottlenecks caused by the template rendering can be avoided.

The invention provides a hotspot detection method and device of a distributed storage system, a monitoring server and a storage medium, and belongs to the technical field of distributed storage. The hotspot detection method comprises the following steps: obtaining a per-second query rate of a data fragment in a storage node, wherein the per-second query rate comprises a current per-second query rate and a historical per-second query rate; determining the state of the current per-second query rate based on the historical per-second query rate; and in response to the state of the current query rate per second as a target state, determining that the storage node corresponding to the data fragment is a hotspot. Therefore, the hot spot can be found automatically, quickly and accurately, and the problems of complexity, time consumption and low accuracy caused by speculating the position of the hot spot in a manual observation mode are effectively solved.

The invention relates to the technical field of software development, in particular to a partitioning / sharding persistent scheme for high QPS (Query Per Second) data. The partitioning / sharding persistent scheme is characterized by adopting the following method comprising the following steps: firstly, saving and processing system data, and periodically performing sharding processing on generation time of the data, wherein each datum generated in a specified time is only subjected to service logic processing in a corresponding time slice database; secondly, dividing the service logic processing process in the first step into three core stages: a saving stage, a stage of unloading into other persistent systems and a stage of cleaning up current time slice accumulating data. The partitioning / sharding persistent scheme has the advantages that a flow is simple; the problems except the problems related to data and time; the partitioning / sharding persistent scheme is irrelevant to other demands of the system; high QPS persistent data are kept into a database in the long-term operation process of the system; the working efficiency of the system is improved, and the like.

The invention relates to a data management method and device, relates to the field of computer technologies and software, and aims to solve the problem that high-concurrency inventory quantity management with the query rate of more than 100,000 per second cannot be stably supported in related technologies. The method comprises the following steps: receiving a plurality of request messages, whereinthe plurality of request messages are used for requesting to obtain virtual resources; if the number of the current request messages is greater than or equal to a first threshold, wherein the sum issmaller than or equal to the sum of the first storage value and the second storage value. If yes, a first storage value is applied to the local cache, the first storage value is updated, the first storage value is a storage value corresponding to the virtual resources of the local cache, the second storage value is a storage value of the virtual resources stored in the remote dictionary service Redis, and the sum of the first storage value and the second storage value is the remaining number of the virtual resources; If the number of the current request messages is smaller than the first threshold value or larger than the sum of the first storage value and the second storage value, applying for the second storage value from the Redis, and updating the second storage value.

The embodiment of the invention provides a data splitting method and device for a distributed storage system. The method comprises the steps that whether query per second (QPS) of all data fragments is greater than a first preset threshold value is monitored; the data fragment with the QPS greater than the first preset threshold value is determined as a target data fragment, access requests of the target data fragment are sampled, and a sampled data stream of index key values corresponding to the sampled access requests is formed; whether the QPS of the target data fragment is greater than a second preset threshold value is judged; if yes, a median of the index key values in the sampled data stream in each preset time period is determined; a median of the index key values in a next preset time period corresponding to the current moment is obtained through prediction according to the determined median, and the median obtained through prediction is determined as a splitting point; and the target data fragment is split according to the splitting point. When the data fragments on access request hotspots are split through the scheme, load balance of the access requests can be realized.

The present application discloses a cache elimination method, which includes: setting a first queue and a second queue, wherein the first queue is used to maintain cached data and a unique code corresponding to each of the data, the second queue The queue is used to maintain the code and the query rate per second of each of the data; receive the code of the data to be queried; initiate a query operation to the first queue according to the code, and if the code is not queried The first queue and the second queue are updated during encoding, and the cold data in the first queue is eliminated. The present application also discloses a cache elimination system, an electronic device and a computer-readable storage medium. In this way, hot data sets, warm data sets and cold data sets in the queue can be effectively distinguished, the accuracy of elimination can be improved, and cached data of historical access records can be eliminated without a large number of data accesses.

The invention discloses a domain name system DNS capacity measurement method, device, equipment and medium. The method includes: obtaining historical log information of the target domain name system DNS; extracting at least one domain name and its corresponding time stamp in the historical log information; generating a test script according to at least one domain name and its corresponding time stamp; using the test script to send The target DNS sends the request information, so that the target DNS parses the request information; receives the analysis result of the target DNS, and calculates the first preset index according to the analysis result; when the first preset index satisfies the first preset condition, the target DNS corresponding The query rate QPS per second is used as the actual capacity of the target DNS. According to the embodiment of the present invention, it is possible to measure the actual capacity of the target domain name system DNS in a simulated live network environment as realistic as possible.

The embodiment of the invention provides a server system expansion method and apparatus and an expansion processing device. The method comprises the following steps: for a non access layer server, determining a server expansion number of each service cluster; after the server expansion number of each service cluster of the non access layer server is determined, and the non access layer server is expanded, simulating a business request in an access layer service cluster; in a process of simulating the service request, if the query rate per second QPS corresponding to the access layer service cluster cannot be improved by increasing the flow of the business request, after the number of the access servers of the access layer service cluster is increased according to a preset number, judging whether the QPS is increased; and if so, determining the number of the access servers of the access layer service cluster, which needs to be increased according to the set number again, until the QPS reaches a set QPS value. The expansion manner provided by the invention is relatively rational, the condition of resource waste of the server of the expanded server system can be reduced.

The embodiment of the invention discloses a data searching method and device. The method comprises the steps that the query per second is calculated by a target server receiving search requests; when the query per second is smaller than an upper threshold value of the query per second, the target server searches for target data meeting the preset requirements in a local database; when the target data is found, the target data is returned to a target client; when the target data is not found, a second server cluster is made to search for the target data in a local cluster database; the second server cluster feeds the found target data back to the target server, and then the target data is returned back to the target client; under the situation that the second server cluster does not find the target data, and the query per second is smaller than a lower threshold value of the query per second, a third server cluster searches for the target data in the local cluster database, and the target data is fed back to the target server; the target server returns the target data to the target client. By means of the data searching method and device, the stability and effectiveness of search service can be improved.

The invention relates to a server resource request method and device, electronic equipment and a storage medium, and the server resource request method comprises the steps: recording the number of task requests received in a current time period, and after the current time period is ended, sending the task requests to a server; generating a target resource request according to the number of the task requests received in the current time period, and sending the target resource request to a main server; and receiving resources returned by the main server according to the target resource request, responding to the task requests received in the current time period by using the resources returned by the main server, and collecting the task requests received in the current time period. The resource request is carried out in batches according to the number of tasks received in the current time period, the number of times of interaction between the server and the main server is reduced, the number of query requests per second is increased, the efficiency of the resource request is improved, communication resources of the system are saved, and the user experience is improved.

The invention relates to the technical field of software development, in particular to a partitioning / sharding persistent scheme for high QPS (Query Per Second) data. The partitioning / sharding persistent scheme is characterized by adopting the following method comprising the following steps: firstly, saving and processing system data, and periodically performing sharding processing on generation time of the data, wherein each datum generated in a specified time is only subjected to service logic processing in a corresponding time slice database; secondly, dividing the service logic processing process in the first step into three core stages: a saving stage, a stage of unloading into other persistent systems and a stage of cleaning up current time slice accumulating data. The partitioning / sharding persistent scheme has the advantages that a flow is simple; the problems except the problems related to data and time; the partitioning / sharding persistent scheme is irrelevant to other demands of the system; high QPS persistent data are kept into a database in the long-term operation process of the system; the working efficiency of the system is improved, and the like.