36results about How to "Reduce reading and writing pressure" patented technology

The embodiment of the invention provides a method and system for reading and writing user data under virtual environment and a physical machine. The method includes the steps that whether every virtual machine has a user data reading and writing request or not is detected; the user data reading and writing requests of all the virtual machines are collected; the reading requests are added into a reading request queue, and the writing requests are added into a writing request queue; when the correspondence degree between first request addresses in the reading request queue and / or the writing request queue and second request addresses in the reading request queue and / or the writing request queue meets a preset condition, the request queue corresponding to the first request address is adjusted according to a request queue adjustment method corresponding to the preset condition; a request in the front of the request queue with the high priority is called and sent to a server where the user data are stored. By means of the method, reading and writing pressure of communication interfaces of the physical machine and the reading and writing pressure of the server where the user data are stored are substantially reduced, and user data reading and writing efficiency is improved.

The invention discloses an index generation method and device for repeated data deletion. The method comprises the steps that a data flow composed of a plurality of data pieces is received, each data piece corresponds to a fingerprint and a serial number, and the sequence of the serial numbers are the same as the sequence of the data pieces in the data flow; when the serial numbers of more than two data pieces are adjacent, and data piece IDs of data pieces, corresponding to the fingerprints of the more than two data pieces with the adjacent serial numbers, in a single case library are also adjacent, a piece of merged index information is generated according to the more than two data pieces with the adjacent serial numbers. According to the embodiment, the piece of merged index information is generated according to the more than two data pieces with the adjacent serial numbers, so that the number of pieces of index information is reduced, the storage space is saved, and the read-write pressure generated when the data flow is recovered is lowered.

The invention relates to a high-concurrency agricultural product online trading system and a construction method and an operation method thereof. The system comprises a presentation layer, a service layer and a persistence layer which are connected in turn. The presentation layer communicates with the service layer through a service middleware Dubbo. The service layer is connected with the persistence layer through a cache database Redis. The presentation layer and the cache database Redis use ActiveMQ as a message queue. According to the high-concurrency system based on Redis and ActiveMQ, the Redis is used as cache to reduce the pressure of database reading and writing, and the ActiveMQ is used as the message queue to eliminate the high concurrency access peak and accelerate the responsespeed of the website so as to greatly increase the concurrent access to the website and meet the needs of subsequent business operation.

The invention relates to a data writing method and device. The method comprises the steps that the number of first sub-data block sets in a data block is determined, wherein data stored in each sub-data block in the first sub-data block sets are all dirty data; when the number of the first sub-data block sets exceeds a threshold value, second sub-data block sets are determined, wherein the second sub-data sets are the set of continuous sub-data blocks where the data stored in the first sub-data block and the data stored in the last sub-data block are all dirty data; when the number of the second sub-data block sets does not exceed the threshold value, all the second sub-data block sets in the data block respectively generate a first data writing request, and the first data writing requests are sent to a storer; when the number of the second sub-data block sets exceeds the threshold value, all the second sub-data block sets in the data block generate a second data writing request, and the second data writing request is sent to the storer. According to the data writing method and device, in the writing-in process, the data does not need to be read from a disk to a cache, and reading-writing pressure on RAID is relieved.

The invention discloses a data collection method, a device, equipment and a readable storage medium, and relates to the technical field of big data and the technical field of cloud computing. According to the specific implementation scheme, the method comprises the steps of obtaining an operation request of a user, and extracting user operation information from the operation request; generating aninsertion statement for writing the user operation information into a service database; submitting the insertion statement and the business data change statement to the business database, so that thebusiness database writes the user operation information by executing the insertion statement and changes the business data by executing the business data change statement; and according to the binarylog of the service database, collecting the service data before and after the change and the user operation information. According to the embodiment of the invention, user operation information and service data before and after change are completely collected; meanwhile, data collection and service decoupling are achieved.

The invention discloses a similar image retrieval method, which comprises the following steps of: obtaining an image to be retrieved, and calculating a binary hash characteristic value; carrying out segmentation processing on the binary hash characteristic value; carrying out retrieval in the constructed image feature value database, finding out all images containing at least one section of Hash feature value, and sorting the images into a final candidate image list; calculating a characteristic value distance between the to-be-retrieved image and each image in the candidate image list, and determining an image with the minimum characteristic value distance from the to-be-detected image in the candidate image list; judging whether the characteristic value distance between the image and theto-be-retrieved image is smaller than a preset threshold value or not; wherein the image is a similar image of the to-be-retrieved image if the image is smaller than the preset threshold value. According to the method, image retrieval is carried out in a manner of segmenting the binary hash characteristic values of the to-be-retrieved images, so that the similar image retrieval speed is greatly increased.

The invention relates to the field of computer storage systems, and discloses a method for accelerating cloud storage internal data transmission. The method comprises the following steps: transmitting the data from a server to a storage through a network, wherein the data is written in a caching module to be stored temporarily in advance before being transmitted from the server, starting transmission through a transmission control module, and when the data reach the storage side, firstly storing the data in a cache, and then slowly writing the data in the storage. By virtue of the technology, the transmission rate will be greatly increased as compared to that of the traditional structure, which meets requirements of users who are harsh on operating lag; the read-write stress of the storage which directly faces the server is reduced; the method is beneficial to protect the storage, and prolongs the service life of the storage.

An embodiment of the invention provides a method and device for querying data in a blockchain system, wherein the blockchain system comprises N mutually independent blockchains. The method comprises the steps that a query request for first data is received, wherein the query request comprises a first data identifier of the first data; that the first data is stored in a first block chain in the N block chains is determined based on the query request; based on the first data identifier, the first data is read in the first block chain.

The embodiment of the invention provides a configuration method and device and relates to the field of Internet technology. The method comprises the steps that responding to an access request of a program on variables, configuration information corresponding to the variables is searched for in a memory; if the configuration information is not found through search, the configuration information corresponding to the variables is acquired from a far-end server; and the configuration information, acquired from the far-end server, corresponding to the variables is stored in the memory for the program to access. Through the embodiment, server pressure and network overhead can be lowered.

The invention discloses an OSD (object storage device) slow disk processing method, system and device and a computer storage medium and applies to distributed storage systems. The method comprises: acquiring attribute of a slow disk, and receiving a user's selection command; determining a processing mode of the slow disk according to the selection command; if the determined processing mode is a manual processing mode, generating a reminder message according to the attribute of the slow disk so that operation and maintenance staff may manually process the slow disk according to the reminder message; if the determined processing mode is an automatic processing mode, judging whether to adjust weight of an OSD corresponding to the slow disk to zero according to the selection command; if not, directly and automatically decreasing the weight of the OSD corresponding to the slow disk; if yes, judging whether it is possible to adjust the weight of the OSD corresponding to the slow disk to zero; if yes, adjusting the weight of the OSD corresponding to the slow disk to zero, and migrating original data in the slow disk having the weight zeroed to other OSDs in a cluster. The OSD slow disk processing method, system and device and the computer storage medium have the advantage that overall performance of a distributed storage system is improved.

The invention discloses a design and implementation method of an intelligent body temperature monitoring and early warning big data platform, and relates to the technical field of body temperature monitoring. The system comprises data acquisition equipment, a data analysis module and a data display and summarization module, wherein the data acquisition equipment is used for acquiring student identity information data and body temperature data in real time and transmitting the student identity information data and the body temperature data to the data analysis module, and the data analysis module is used for comprehensively analyzing the student identity information data and the body temperature data; and the data display and summarization module is used for displaying and summarizing the student identity information data and the body temperature data analyzed by the data analysis module. The temperature, iris and other data are collected through the infrared detection technology and the iris collection technology, the data is sent to the cloud platform, the data is stored and analyzed in real time by means of the cloud server and the body temperature big data analysis platform, thedata visualization is achieved, the data query is facilitated, and the real-time early warning is achieved.

A data writing method and device, comprising: determining the number of first sub data chunk sets in the data chunks, all data stored in each sub data chunk in the first sub data chunk sets being corrupted data; when the number of first sub data chunk sets exceeds a threshold, determining second sub data chunk sets, a second sub data chunk set being a set of continuous sub data chunks with all data stored in the first and last sub data chunks being corrupted data; when the number of second sub data chunk sets does not exceed the threshold, generating a first data writing request respectively for each second sub data chunk set in the data chunks, and sending the first data writing request to storage; and when the number of second sub data chunk sets exceeds a threshold, generating a second data writing request for data chunks and sending the second data writing request to storage. The data writing method and device do not allow data to be read from a disk to a cache when writing the data, thus reducing the read and write pressure on the RAID.

The invention discloses a method and a device for asynchronously updating data based on a message queue, comprising: a modeling service module determines initial data to be updated according to a data update instruction; the message service module writes target data determined by the initial data into a first message queue; The cache service module reads the target data from the first message queue and loads it into the memory, and the data loaded into the memory is used for publishing; the write library service module writes the target data read from the first message queue into the database. It can be seen that the present invention can receive data through a message queue and store it locally in a persistent manner, avoiding the process of writing data to the database first, and also pushing the data to the cache service and publishing it by the cache service, reducing the process of directly reading data from the database. Finally, the write library service asynchronously writes the data to the database, reducing the amount of data written to the database, reducing the pressure on reading and writing the database, reducing the stall phenomenon during data update, and improving the efficiency of data update.

The embodiment of the present invention proposes a hot file self-adaptive copy method and system. The method includes: receiving a file download request; obtaining index information of the file to be downloaded according to the file download request; determining the storage disk of the downloaded file according to the index information; The file heat of the downloaded file; according to the file heat of the file to be downloaded, it is judged whether the file to be downloaded needs a hot copy; if copying is required, at least one buffer disk with the lowest load is obtained; multiple copies of the file to be downloaded are copied to the obtained buffer disk. The invention can ensure the file access rate of the server, reduce the reading and writing pressure, and reduce the delay of disk reading and writing.

The invention discloses an online processing method and system oriented to streamed data. The method comprises the following steps: 1, establishing an online memory cache layer, extracting the property of the streamed data according to a key value structure, and storing in the online memory cache layer; 2, establishing a hybrid index structure for the streamed data in the memory cache layer; 3, adding an access flag bit to each streamed datum for which the index structure is established, and recording the access state of the streamed data by each analysis program, wherein the flag bits are used for flagging the register condition of the streamed data by different analysis programs; 4, cleaning data: if a certain streamed datum is accessed by all specified analysis programs in the memory cache layer, cleaning the streamed datum. By adopting the method and the system, the data reading and writing stress in a streamed processing procedure is reduced greatly, the stress of a database in a large-scale streamed data processing system can be effectively relieved, and the real-time processing speed of the streamed data can be increased.

The embodiment of the invention discloses a data deleting method, a device, equipment and a storage medium. The invention is applied to the field of virtual machines of cloud computing, and is specifically applied to delayed deletion processing of virtual machine snapshots generated by the virtual machines. The method comprises the following steps: when a trigger event for executing a virtual machine snapshot deletion operation is detected, obtaining at least one virtual machine snapshot of which the data state is a deleted state; obtaining a target virtual machine snapshot matched with the target computing node from the at least one virtual machine snapshot according to the parameter information of each virtual machine snapshot; and deleting the target virtual machine snapshot. By adopting the embodiment of the invention, the virtual machine snapshot is deleted in a delayed manner, and the read-write pressure of the storage cluster in the cloud computing system is effectively reduced,so that the stability of user services can be improved.

The embodiment of the invention discloses a snapshot backup method of data. The method comprises the following steps that: determining the category of respective basic data in the data of a plurality of virtual machines in a cloud platform; uniformly carrying out one-time snapshot backup on the basic data in the data of the virtual machine corresponding to same category of the basic data, and independently carrying out snapshot backup on respective discrepant data in the data of the virtual machine corresponding to the same basic data. The embodiment of the invention simultaneously discloses a snapshot backup device of the data.

Embodiments of the present invention provide a method, system, and physical machine for reading and writing user data in a virtualized environment, wherein the method includes: detecting whether each virtual machine has a user data read and write request; collecting the user data read and write requests of each virtual machine, Add the read request to the read request queue, and add the write request to the write request queue; when the correspondence between the first request address and the second request address in the read request queue and / or write request queue meets the preset condition , adjust the request queue corresponding to the first request address according to the request queue adjustment method corresponding to the preset condition; call the previous request in the request queue with relatively high priority, and send the called request to the server storing user data request. The embodiment of the present invention greatly reduces the reading and writing pressure of the communication interface of the physical machine and the server storing user data, and improves the reading and writing efficiency of user data.