31results about How to "Improve write performance" patented technology

The invention discloses a multi-NDP (Near-Data Processing) system read-write acceleration method based on a key value separation technology, and the method comprises the steps: when data is written, splitting a data key value pair written into a batch processing region of a multi-NDP system, determining target equipment, then carrying out key processing and value processing, carrying out key storage on keys, and carrying out data storage on the target equipment; a value corresponding to the key is added to a value log file of the target equipment; during data query, hash operation is carried out on a query key, target device position information of a corresponding value is obtained, then the master-slave multi-level hot cache is queried, and query retrieval of the corresponding value is carried out according to the obtained value position index. According to the method, the key value separation technology is introduced into the multi-NDP system, when the Compaction is executed, the execution time of the Compaction is shortened, the influence of the Compaction on the writing operation of the user is reduced, and the writing performance is greatly improved.

The invention discloses a traceability application-oriented block chain database data management system, particularly relates to the technical field of block chain database data management, and is used for solving the problems of easy data tampering and low verification efficiency caused by traceability data logic association and physical storage splitting. The method comprises the following steps: firstly, matching a parent physical sector offset through an association analysis module, determining a storage partition, and assembling a write-in instruction; then, a topology binding module executes a disk write operation, synchronously reads parent node hash and performs combinatorial operation in combination with a physical address to generate a topology binding check value bound with the physical position; the aggregation anchoring module uses a Merkel tree aggregation check value to generate a root fingerprint and perform uplink, and establishes an anchoring voucher; and finally, a path verification module positions a physical position according to the certificate, executes reverse recursive addressing and step-by-step Hash recalculation, and assembles a pedigree verification packet, so that a closed-loop trust system from physical storage to chain certificate storage is constructed, and the tamper-proof capability and verification efficiency of the traceability data are improved.

The invention discloses a ClickHouse-based fused CDN (Content Delivery Network) business data processing method, device and equipment, and the method comprises the steps: obtaining business data of a corresponding fused CDN manufacturer through calling a data interface of each fused CDN manufacturer in real time; performing preprocessing including data format standardization, field expansion and invalid data filtering on the service data by using Spark, packaging the preprocessed service data and sending the packaged service data to a Kafka cluster; the method comprises the following steps: creating a distributed data table based on a ReplacingMergeTree engine in a ClickHouse database, consuming service data in a Kafka cluster in real time by utilizing Spark, and writing the service data into the distributed data table of the ClickHouse database in batches; and retrieving target business data from the ClickHouse database according to a data query request of a user, and converting the target business data into a JSON format to serve as a query result to be returned. The query performance, the storage efficiency, the expandability and the write-in performance can be improved.

The invention provides a method for prolonging the service life of a solid state disk, which belongs to the technical field of solid state disk storage and comprises the following steps of: 1, acquiring historical read-write data of the solid state disk, and determining an access state of each data block of each data packet in the solid state disk; 2, health data of each flash memory block in the solid state disk are monitored and obtained in real time based on a preset monitoring tool, and then the state of each flash memory block is determined; 3, determining a data distribution strategy based on the access state in each data block of each data packet in the solid state disk and the state of each flash memory block, and distributing all the data packets in the solid state disk; and 4, temporarily storing the data to be written into the cache region of the solid state disk, obtaining an analysis result of the data, and further writing the data packet to be written into the corresponding flash memory block. The service life of the solid state disk is prolonged, and meanwhile the reliability and writing performance of data storage are improved.

The invention relates to the technical field of computer storage, in particular to an address mapping method in storage equipment. Comprising the following steps: S1, setting a plurality of partitions with different mapping units in a storage device; s2, dividing the mapping table of the storage equipment according to the partitions in the step S1, wherein each partition corresponds to one sub-mapping table; s3, dividing the interior of the sub-mapping table into a plurality of mapping groups; s4, setting a mapping mode in the mapping group; and S5, when the mapping table is updated after the data is written, determining mapping table items partially overlapped with the written data LBA in the mapping table, and updating the first PBA and / or information representing PBA continuity in the mapping table items. Compared with the prior art, a plurality of partitions with different mapping units smaller than 4K can be arranged in the storage equipment and are used for improving the writing performance when small data blocks are accessed; and meanwhile, the mapping tables of different partitions can adopt a more compact mapping form according to the continuity of data writing, so that the storage space occupied by the whole mapping table is reduced, and the memory cost of the storage equipment is further reduced.

This invention provides a redundant array management method and system based on ZNS solid-state drives, comprising: for each stripe, upon receiving a write request, sequentially writing the data to be written into the corresponding data blocks, and overwriting and updating the check data of the check block; positioning the logical actual write pointer to the tail position of the consecutively successfully written data blocks, generating metadata corresponding to each data block, and writing the metadata into a spare area; the logical actual write pointer is used to point to the next writable position within the logical area; in the event of a failure, repositioning the logical actual write pointer based on the metadata in the spare area; based on the repositioning of the logical actual write pointer after the failure, discarding the data blocks outside the logical actual write pointer, restoring the data blocks within the logical actual write pointer, and recalculating and generating check data to complete the redundant array management of the ZNS solid-state drive.

The log collection method and system provided in this specification builds a target Pod, mounts the pre-set target directories in multiple application Pods on the physical machine to the created target Pod, and takes over the writing and collection of log data of all target logs under the target directory. The target Pod writes the log data of all application Pods into a target file, and then writes the target logs in the target file into the storage path corresponding to the application Pod in sequence according to the classification of the application Pod to which they belong, so that random writes to the storage device become sequential writes, greatly improving the write performance.

The embodiment of the invention provides a data writing method and a server, and the method comprises the steps: configuring a persistent memory into a direct access mode on an operating system of the server; mounting the persistent memory to a file system of a server, and opening a direct access parameter to support direct access to the persistent memory; acquiring a data writing request, and writing a pre-writing log into the persistent memory according to the data writing request; wherein the data writing request is used for requesting to write to-be-written data; and in response to the data writing request, writing the to-be-written data into the database. According to the scheme, the pre-write logs can be prevented from being lost, and the system reliability is improved.

The application provides a log processing method of a database, a storage medium and equipment. The log processing method comprises: obtaining log configuration information, and determining whether log configuration has been completed according to the log configuration information; if not, assigning a preset level to a log entry to be written; generating an entry number according to the log level; and writing the log into a pre-set shared memory according to the entry number. Therefore, even if the log configuration has not been completely parsed, the log can be written into the pre-set shared memory by assigning a preset level to the log entry, thereby avoiding the problem of log loss before the log configuration is completely parsed. Moreover, because the log exists in the shared memory, even if the program is interrupted, the log in the shared memory will not be lost, and the log that has been written in advance can still be obtained.

The application discloses a calligraphy mechanical arm control method based on stroke flow direction prediction, which comprises the following steps: collecting image information in a calligraphy mechanical arm writing area through a shooting device; collecting the image information and current time data as historical data; inputting the historical data into a preset KNN model, clustering the historical data into a plurality of clusters, each cluster containing a plurality of data samples, the number of clusters being set according to the covariance requirement in the cluster and the covariance requirement between the clusters, temporarily storing the obtained result in the KNN model, then adding current measured data to re-cluster, clustering the current measured data into the corresponding cluster, inputting n data A1-An in the cluster where the data is located and the current measured data An+1 into an LSTM model, inputting the historical data into the LSTM model to initialize the model, inputting the data A1-An+1 obtained from the KNN model into the trained LSTM model in sequence to obtain corresponding output values, and taking the weighted average of the output values as a result output.

This application discloses a data processing method, apparatus, electronic device, and computer-readable medium, relating to the field of computer technology. A specific embodiment includes: obtaining a data stream and a cluster identifier based on a received data processing request; obtaining node data corresponding to the cluster identifier; generating a first hash value based on the node data; calculating a second hash value for each data item in the data stream; determining the target node corresponding to each data item in the data stream based on the first hash value, the second hash value, and a hash ring; determining the number of data items to be written for each target node; configuring an independent connection pool for each target node based on the number; and writing each data item to the corresponding target node based on the connection pool. This improves write performance and throughput when writing data to the cluster, avoids single-point overload, ensures uniform data distribution in the cluster, increases data write speed, and shortens data write latency.

The present invention relates to the field of storage technology, and provides a data storage method, device, storage device and readable storage medium, wherein the method comprises: receiving a write request sent by a client for writing data to be written, wherein the write request comprises a file to be written, a length to be written of the data to be written, and a primary cache area and a secondary cache area are pre-bound to the file to be written; if the length to be written is not greater than a preset value, the data to be written is written into the primary cache area and the secondary cache area, so that the data to be written is stored from the primary cache area or the secondary cache area to the storage area; if the length to be written is greater than a preset value, the data to be written is written into the secondary cache area, so that the data to be written is stored from the secondary cache area to the storage area; and a response message for the write request is returned to the client. The present invention can quickly return a response message to the client, thereby improving the overall write performance of the storage device in the storage system.

The invention provides a storage space allocation method and device, electronic equipment and a storage medium. The method comprises the following steps: in response to a storage space allocation instruction for a virtual disk, determining a segment allocation cache pool corresponding to the virtual disk; wherein the segment allocation cache pool is used for caching segments obtained by segmentation of a fragment group, the fragment group is composed of fragments from at least two physical disks, and the fragments are local parts of the physical disks; if it is determined that there is no available segment in the segment allocation cache pool, determining a target fragment group from at least two preset fragment groups; determining a preset number of segments from the target fragment group, and caching the preset number of segments into a segment allocation cache pool; and determining a target segment from the segment allocation cache pool as a storage space allocated to the virtual disk. Through the method provided by the invention, the balance of IO loads of different disks can be ensured.

This invention discloses a mapping management method, apparatus, computer device, and storage medium for solid-state drives (SSDs). The method includes: establishing an SLC cache mapping table independent of the main mapping table; storing data written by the host in the SLC cache area and recording the mapping relationship between the logical page address and physical page address corresponding to the data in the SLC cache mapping table; when the SLC cache area meets the migration conditions, migrating the data in the SLC cache area to multiple programming data blocks and completing all programming steps for the data in the multiple programming data blocks; after confirming that the data has completed all programming steps in the multiple programming data blocks, deleting the mapping relationship corresponding to the data from the SLC cache mapping table and updating the mapping relationship to the main mapping table. By implementing the method of this invention, the performance and lifespan problems caused by dual mapping updates in existing QLC SSDs using SLC caching technology can be effectively solved.

The invention relates to a data flow write-in control method, system, device and equipment, a readable storage medium and a program product. Comprising the following steps: receiving write-in performance characteristics of a data stream processing engine in a first cycle period; according to the write-in performance characteristics, a first operator routing table of a first cycle period is adjusted, a second operator routing table of a second cycle period is obtained, the second operator routing table comprises a routing record in the second cycle period, and the routing record indicates a to-be-written data stream in the data stream processing engine and a corresponding data write-in operator; a second operator routing table is sent to the data flow processing engine, the second operator routing table is used for indicating the data flow processing engine to write the data flow to be written corresponding to the data write operator by using the data write operator indicated by each routing record in the second operator routing table in a second cycle period, and the data flow to be written corresponding to the data write operator is sent to the data flow processing engine. And writing the data writing operator and a file corresponding to the data stream to be written. By adopting the method, the data stream writing performance can be effectively improved.

The embodiment of the invention provides a method and device for improving Dramless QLC SSD enabled RAID writing performance, equipment and a medium, and relates to the technical field of solid state disks. The method comprises the following steps: acquiring an RAID (Redundant Array of Independent Disk) part of a Dramless QLC SSD (Solid State Disk); the RAID (Redundant Array of Independent Disk) part is programmed and written into the Dramless QLC SSD for the first time; temporarily storing the RAID (Redundant Array of Independent Disks) part in a preset cache region; and extracting the RAID (Redundant Array of Independent Disk) part from the cache region, and writing the RAID part into the Dramless QLC SSD after the RAID part is programmed for the second time. According to the method and the device, the RAID part is temporarily stored in the preset cache region, so that the RAID part does not need to be repeatedly calculated during the second writing, the RAID part can be directly extracted from the cache region, the occupation of calculation resources is reduced, the efficiency is greatly improved, and the writing performance of the Dramless QLC SSD enabled RAID is improved.

The present disclosure relates to a data operation method, device, equipment and medium, the method is applied to a data node, including: determining a hard disk connected to the data node, and reporting the hard disk information to a metadata server, wherein the hard disk information includes the hard disk medium type, available capacity and location information; receiving a data operation instruction sent by the metadata server, wherein the data operation instruction carries the medium type and location information of the hard disk where the operated data block is located; according to the medium type, calling a service unit corresponding to the medium type to perform a data block operation according to the location information of the hard disk. The present disclosure simplifies the use of a distributed storage cluster and reduces the difficulty of cluster deployment and maintenance.

The application relates to a storage device, a storage system and a write method of the storage device. The storage device comprises a front-end module, a back-end module, a cache module, a storage medium and a processor. The cache module is connected with the front-end module, the processor and the back-end module, the back-end module is connected with the processor and the storage medium, the cache module comprises a cache management unit, a first cache and a second cache, the capacity of the second cache is greater than or equal to that of the first cache, and the read-write speed of the second cache is less than that of the first cache. The front-end module transmits to-be-written data to the first cache. The cache management unit controls mutual transfer of the to-be-written data between the first cache and the second cache. The back-end module writes target to-be-written data into the storage medium through the first cache. The application uses the first cache with high read-write speed and the front-end module and the back-end module to form a write channel, the delay is small, the second cache with large capacity and the first cache are used to mutually transfer data to increase cache data, and the write performance is effectively improved.

The invention belongs to the field of computer information processing, and particularly discloses a load balancing security log system and a management method thereof, and the system comprises a log server, an access application program interface and a configuration tool; the application program access interface is used for providing an API (Application Program Interface), so that an external application sends an instruction to the log server by accessing the API to carry out log processing; the configuration tool is used for configuring parameters of the log server, and the parameters comprise the maximum entry of log storage, the maximum entry of repeated logs, log viewing configuration parameters and log writing configuration parameters; and the log server is used for executing log configuration modification, log writing, log reading and legality check in response to the input instruction according to the configuration parameters of the configuration tool in combination with the input instruction for accessing the application program interface so as to ensure effective management and storage of logs. According to the invention, the security of the log system can be improved and the system performance optimization can be realized.

The invention discloses a key value storage method and system based on a nonvolatile memory and a solid state disk, and belongs to a key value storage system.The key value storage method comprises the steps that key value pairs are divided into large key value pairs and small key value pairs based on the size of storage space occupied by the key value pairs, the small key value pairs are stored in Smem and Simmm in an NVM and Skv in an SSD, and the large key value pairs are stored in Bmem and Bimmm in the NVM and Bkv in the SSD; small key value pairs are stored in order, and large key value pairs are not required to be stored in order; the key value pairs are firstly stored in Smem or Bmem in the NVM, and when the corresponding Smem or Bmem is full, the key value pairs are brushed down to the SSD; and maintaining a corresponding position recording structure in the NVM to record the storage position of each key value pair in the NVM or the SSD. According to the method, the characteristics of the NVM and the SSD can be fully utilized, and the key value storage performance is improved.

The invention provides a data writing method and device, electronic equipment and a medium. The method comprises the steps that a write request sent by a client is received, the write request comprises a write-in path, and the write-in path corresponds to a target directory; according to the write request, obtaining the directory quota usage amount and the residual capacity of the target directory; if the directory quota usage amount is smaller than the usage amount threshold value and the residual capacity is larger than the preset capacity, applying for a first directory quota capacity of which the size is the preset capacity; according to the first directory quota capacity, writing to-be-written data corresponding to the writing request; and if other write requests sent by the client are received in a future preset time period, writing other to-be-written data corresponding to the other write requests into the target directory according to the first directory quota capacity.

The invention discloses a garbage collection method for a solid state disk in the technical field of solid state disks, which comprises the following steps: S1, monitoring freevb in real time, and triggering GC when the freevb reaches a threshold value; s2, if the number of newly added nodes is larger than or equal to thres1, or the number of newly added ST2 vb is larger than or equal to thres2, or vb in source vb pool is exhausted, executing S3, otherwise, executing S4; s3, establishing a source vb pool (Source vb pool); s4, the source vb is selected from the source vb pool, and the source vb is selected from the source vb pool; s5, a target vb is created; s6, effective data are recorded; s7, moving effective data; s8, updating the L2P mapping relation; s9, the source vb is released; and S10, updating the vblist. According to the garbage collection method, the time for establishing the source vb pool is controlled through the threshold value, and the writing performance of the solid state disk is improved by reducing the frequency for establishing the source vb pool, shortening the GC time consumption and improving the GC efficiency.

The present application relates to a hardware acceleration method, device, computer equipment and storage medium for updating a summary page of an SSD, the method comprising: writing LPA0 by a host and allocating a physical address for writing LPA0; determining whether the current physical address is a position for writing raidparity; if not, determining whether the current physical address is a position for writing a summary page; if not, updating a mapping table and transferring information required for updating a summary page to hardware; calculating a position for storing LPA in a summary page table by hardware according to bank / page / offset in the physical address and the information required for updating a summary page, and then updating the current summary page table; and writing user data into NAND according to the updated current summary page table. The present application processes the initialization and content update of a summary page by hardware, which can improve the write performance of a system and better meet the requirements.

The invention discloses a data writing method, electronic equipment and a computer readable storage medium. The method comprises the following steps: acquiring write-in data, and determining the number of times of a configuration parameter flow and a corresponding write-in command based on the write-in data; wherein each parameter configuration process is used for performing parameter configuration related to the write-in command before the corresponding write-in command is executed; and in response to completion of the partial configuration parameter process, executing a write-in command corresponding to the partial configuration parameter process, and synchronously configuring at least one of the remaining partial configuration parameter processes. In this way, the writing performance of the storage device can be improved.

The utility model is suitable for the technical field of crayons, and provides a solid painting pen which comprises a pen tube assembly, a refill assembly and a pen cap assembly, and a first opening is formed in the first end of the pen tube assembly; the pen core assembly comprises a wax core, the wax core is movably installed in the pen tube assembly, a pen point section is formed at the first end of the wax core, the cross section area of the pen point section is gradually reduced in the direction from the second end of the wax core to the first end of the wax core, and the pen point section can penetrate into the external environment through the first opening; the pen cap assembly is detachably installed at the first end of the pen tube assembly, a profiling protection cavity is formed in the pen cap assembly, the shape of the profiling protection cavity is matched with the shape of the pen point section, and the pen cap assembly can store and protect the pen point section through the profiling protection cavity. According to the solid paint pen, the volatilizable space of the pen point section can be reduced, and the shape of the pen point section can be well protected.

The invention relates to the technical field of distributed clusters, and discloses a data writing method and device, a distributed cluster system, equipment and a medium. The distributed cluster system comprises a cluster shared storage unit and at least one cluster node; a cache unit is arranged on each cluster node; the method comprises the following steps: when to-be-written data of a target cluster node is obtained, writing the to-be-written data into a cache unit of the target cluster node, and synchronizing the to-be-written data to cache units of other cluster nodes through the cache unit of the target cluster node; and writing the cache data stored in the cache unit of the target cluster node into the cluster shared storage unit. According to the scheme, the writing speed of writing the to-be-written data into the shared storage unit through the cache unit is higher, so that the overall writing performance of the system is improved.

The invention discloses a log structure merging tree based on a dual-heterogeneous system and a data reading and writing method, and belongs to the technical field of data storage management. Comprising a heterogeneous calculation module, and the heterogeneous calculation module comprises a dynamic random access memory used for calculating and generating a key value data pair; the field programmable gate array is connected with the dynamic random access memory and is used for reading in the key value data pairs and sorting the key value data pairs; the heterogeneous storage module is connected with the heterogeneous calculation module, the heterogeneous storage module comprises a phase change random access memory used for storing a log file and a balance tree, and sorted key value data pairs are stored in the log file; and the solid state disk is connected with the phase change random access memory and is used for storing the plurality of hierarchical components. The technical scheme has the beneficial effects that the random write operation of the solid state disk is converted into batch sequential write, the size of the log structure merging tree is reduced, the write amplification is reduced, and the high-throughput write performance is improved.

The application discloses a data writing method and device, equipment and a storage medium, and relates to the field of data management. The method comprises the following steps: acquiring a data update request for current target data in a new data directory sent by upper-layer software; the data update request comprises to-be-written new data and a target logical address; migrating the current target data to an old data directory to obtain first target old data, and newly creating a mapping relationship between the first target old data and the target logical address in a local first mapping relationship database; performing a local writing operation on the to-be-written new data, searching for the first target old data corresponding to the target logical address from the old data directory by using the first mapping relationship database when the writing fails, and migrating the first target old data back to the new data directory to complete a data recovery operation. The application manages the old data by using the target logical address, avoids the step of managing the old data by performing a disk-falling operation on the copied metadata, and reduces the data double-writing operation.

The application provides a hybrid storage implementation method, system, storage device and management device, and belongs to the technical field of data storage. The method comprises the following steps: obtaining a data write request initiated by a client; applying for storage space to a management device according to the data write request; the management device allocates a solid state disk erasure coding group space and a mechanical hard disk erasure coding group space corresponding to the data write request at one time, and establishes a binding relationship between the solid state disk erasure coding group space and the mechanical hard disk erasure coding group space; obtaining the data size of to-be-written data corresponding to the data write request; when the data size is lower than a preset threshold, writing data into the solid state disk erasure coding group space; when the data size is higher than the preset threshold, directly writing data into the mechanical hard disk erasure coding group space. The application reduces data interaction and repeated writing, and improves data write storage performance.

The embodiments of the present specification provide a data writing method and device, wherein the data writing method comprises: receiving a data writing request, wherein the data writing request carries the data to be written and the data attributes related to the medium operation type of the data to be written; determining the corresponding target medium operation type according to the data attributes of the data to be written; searching the corresponding target partition namespace from the partition namespace of the storage device according to the target medium operation type, wherein the partition namespace is provided with the corresponding medium operation type; writing the data to be written into the target partition namespace according to the writing working mode corresponding to the target medium operation type. In this way, the partition namespace can provide different service qualities, ensure that the allocated partition namespace matches the service quality requirements of the data to be written, improve the data writing performance, and thus provide a better cost-effectiveness for the service of the storage device.