94 results about "Distributed object" patented technology

The embodiment of the invention relates to a distributed object storage method and device, equipment and a medium. The method comprises the steps that a to-be-stored target object is split into multiple pieces of key value pair data; determining target partitions corresponding to the plurality of key value pair data from a plurality of partitions of a preset distributed object storage system; determining a data storage node currently holding the target partition based on a preset partition view to obtain data storage nodes corresponding to the plurality of key value pair data; wherein the partition view is used for indicating meta-information of the plurality of partitions, and the meta-information at least comprises information of data storage nodes holding the partitions; the data storage nodes are storage nodes of the distributed object storage system; and caching the plurality of key value pair data to respective corresponding data storage nodes. According to the embodiment of the invention, the cluster local overload problem of the object storage service can be effectively relieved.

The invention relates to an intelligent network card and a distributed object access method based on the intelligent network card. The intelligent network card comprises a read-write merging module and a self-adaptive prefetching module. And the read-write merging module is used for merging the access request times of the read-write requests for accessing the same destination address from the computing node in the time window so as to reduce the redundant read-write request times. The self-adaptive prefetching module is used for checking whether an on-chip memory of the intelligent network card is hit or not when the computing node initiates an access request to a far-end memory node, adding an entry in a read-write request into a pseudo prefetching queue in the message buffer area if the on-chip memory of the intelligent network card is not hit, and sending the read-write request to the message buffer area; and according to the access cache hit rate, the pre-fetching module corresponding to the read-write request is scored, and the access priority is adjusted based on the score of each pre-fetching module, so that the pre-fetching strategy is dynamically adjusted. According to the invention, the intelligent network card is used as a coordinator of different computing resources in the computing node, and uniform access and prefetching are carried out on the on-chip resources to reduce redundancy.

The invention provides a life cycle deletion optimization method for a distributed object storage system, which comprises the following steps of: S1, detecting the business volume of a cluster, starting a pre-scanning operation when the business volume is lower than a preset threshold value, and identifying an object to be deleted according with a life cycle rule; s2, respectively writing the to-be-deleted object information of different dates into corresponding omaps, and storing omap data in a disk according to a preset packaging frequency; s3, when the life cycle expires, directly reading the omap of the corresponding date, skipping a real-time scanning step, and executing batch deletion operation; and S4, recording statistical information such as the deletion quantity scanned every day on the xattr of the barrel, so that management software performs data trend analysis and operation and maintenance decision. According to the embodiment of the invention, the life cycle deletion efficiency can be improved, the metadata operation pressure is reduced, and the cluster storage space utilization rate is optimized.

The invention discloses a data processing method and device, a nonvolatile storage medium and electronic equipment. The method comprises the following steps: acquiring target authentication information of the distributed storage system; generating a target file comprising distributed object storage block equipment information; using the target authentication information to execute a deployment operation of the containerized data import related component, and after the deployment operation is completed, adding a preset label in a persistent volume statement corresponding to the data storage volume; obtaining target encrypted information required for mapping of the distributed object storage block device from the data storage volume, and mounting the target encrypted information to a preset directory; and adding the target authentication information to configuration information of the target virtual machine, and starting the target virtual machine. The technical problem of low data transmission efficiency caused by the fact that the target virtual machine management platform cannot directly access the target distributed storage system is solved.

The invention discloses a resource adjustment method, electronic equipment, a storage medium and a program product, and relates to the technical field of distributed storage, the weight can be dynamically adjusted according to the use condition, the emergency state and the priority of real-time IO resources of clients, the differentiated IO requirements of multiple clients can be better met, and the user experience is improved. A stable low-delay demand or a periodic burst demand can be timely and effectively responded, and the dynamic adaptability is high; by dynamically adjusting the resource allocation strategy, the bandwidth can be flexibly and reasonably allocated according to the real-time demand of each client, so that the system can operate more efficiently, and services are provided for more clients; in a distributed object storage environment and under the same hardware resource condition, the optimized IO scheduling algorithm enables the system to be more efficient when processing a client request, system response delay is reduced, the data read-write speed is increased, and therefore the performance of the whole distributed storage system is improved.

Disclosed in the present application are an erasure code data updating method and apparatus. The method comprises: in response to a write operation for a target shard of a target object in a main object storage device, acquiring a state identifier of the target shard; if the state identifier is a first state identifier, storing, in a reserved object in a parity object storage device, initial data in the target shard and initial parity data for the target object, using updated data to overwrite the initial data, and modifying the state identifier to a second state identifier; if the state identifier is the second state identifier, using the updated data to overwrite current data in the target shard and updated data that is stored in the reserved object; and when a distributed object storage system is idle, determining target parity data on the basis of the initial parity data and the initial data and the updated data that are currently stored in the reserved object, and modifying the state identifier to the first state identifier.

The embodiment of the invention provides a personnel allocation method and device, computer equipment, a storage medium and a computer program product, and relates to the technical field of artificial intelligence. The method comprises the steps of obtaining object basic information of a to-be-distributed object, and obtaining an object semantic vector of the to-be-distributed object according to the object basic information; determining matching object data of a plurality of matching objects matched with the to-be-distributed object in an object semantic vector library according to the object semantic vector; determining a plurality of candidate matching personnel corresponding to the to-be-distributed object according to the plurality of matching objects and the matching object data of each matching object; acquiring a personnel evaluation result of each candidate matching personnel according to the at least one evaluation index of each candidate matching personnel and the evaluation basis data of each evaluation index by using a personnel evaluation model; and determining a target matching person of the to-be-allocated object according to the person evaluation result. According to the method, the accuracy and efficiency of personnel allocation are improved.

The invention discloses a metadata space management method, device and equipment and a readable storage medium, and relates to the technical field of computers, the method comprises the following steps: selecting a target metadata service as a default metadata service; the default metadata service is used for distributing the logic space of the management storage pool; when the metadata spaces of other metadata services except the target metadata service are lower than a lower use limit threshold value, sending a space application instruction to the target metadata service; and after the target metadata service responds to the space application instruction and distributes part of logic space of the storage pool to other metadata services, the other metadata services store the distributed space as own metadata space. In this way, the metadata service manages the metadata space, management of the metadata service space of multiple nodes is achieved only through one storage pool, and no longer depends on an object storage daemon process and reliable and automatic distributed object storage.

Systems and methods are provided for receiving a vector to be inserted into a set of vectors, wherein the set of vectors is stored as a plurality of leaf objects of a distributed proximity-based graph across a plurality of storage devices of a distributed object storage system, identifying a subset of the plurality of leaf objects based at least partly on a similarity of the vector to one or more vectors stored in the subset of the plurality of leaf objects, storing the vector in an insert buffer associated with the subset of the plurality of leaf objects, and in response to a vector query, generating query results comprising the vector stored in the insert buffer, and at least one of the one or more vectors stored in the subset of the plurality of leaf objects.

The invention provides a virtual item processing method and device, equipment, a storage medium and a program product, and relates to the technical field of computers, the method comprises the following steps: determining a camp skill pool at least composed of object skills of camp objects in a first camp; in response to the fact that the first camp meets the skill distribution condition, at least part of object skills in the camp skill pool are distributed to the first camp, so that the first camp can control camp objects in the first camp to fight with the second camp by using the distributed object skills; in response to the attack of the first virtual object in the second camp, controlling to configure a virtual item for the first camp; and in response to a trigger operation for the virtual item, controlling the target object skill allocated to the first camp to use the virtual item, so that the target object skill obtains a specified skill gain when acting on a second virtual object in the second camp. The resource utilization rate of the terminal equipment can be improved.

This application relates to a method, apparatus, computer device, storage medium, and computer program product for distributing objects awaiting review. The method includes: responding to an object retrieval request, searching for a historical review index matching the requester; determining, based on the historical review index, a historical object identifier and a historical source identifier for the historical review object corresponding to the requester; searching for at least one target class index including the historical object identifier, and obtaining a first object awaiting review associated with each target class index, wherein the target class indexes are established based on object similarity; searching for a target source index from multiple source indexes based on the historical source identifier, and obtaining a second object awaiting review associated with each target source index; and determining a target object awaiting review matching the requester based on the first and second objects awaiting review, and pushing the target object awaiting review. This method enables accurate delivery of objects awaiting review.

Provided herein is an asset storage server that assigns related files to asset names and assigns permissions to asset names so that related files with unrelated names can be assigned permissions independently of file naming conventions without requiring a user to individually set permissions for each file. The asset storage server can also generate a modified version of the original file name and index a distributed object storage area based on the modified version so that related files with related names are not listed in the same partition of the distributed object storage area. Indexing the distributed object storage area based on the modified version of the original file name can reduce data retrieval latency.

The present invention provides a distributed object storage tag retrieval method, system and device, the method comprising: obtaining an object query request, the object query request is used to query an object, each object is provided with multiple classification identifiers, each classification identifier corresponds to a request condition, multiple storage spaces are created according to the classification identifiers, each storage space only stores the request conditions corresponding to the classification identifiers in different objects; extracting the request conditions in the object query request; obtaining the classification identifier corresponding to the request condition based on a dictionary tree; calibrating the corresponding storage space according to the classification identifier, using an ant colony algorithm in the calibrated storage space to process the query request, and obtaining the query result and the optimal retrieval path of the object. The present invention has the following beneficial effects: It solves the technical problems of hash conflicts, limitations in range searches and low efficiency in processing large-scale data in the prior art.

The present application discloses a method, apparatus, device and storage medium for migrating a database to the cloud, including: performing a physical backup on the entire target database to obtain a target physical backup file, and uploading the target physical backup file to a distributed object storage on the same cloud platform; obtaining a validity verification result obtained by the cloud platform for performing validity verification on the target physical backup file; if the validity verification result indicates that the target physical backup file is a valid file, determining the import type of the target physical backup file; and importing some or all of the target physical backup files into a target database instance on the cloud platform according to the import type. The present application imports in the form of physical files and pre-verifies the physical backup files before import to prevent the impact caused by directly importing into the target database instance, and can migrate the physical backup data of the database to the cloud database instance more safely and efficiently.

This invention proposes a 3D semantic scene completion method based on hierarchical grouping and aggregation. The model proposes a novel dual-branch structure. The explicit constraint branch starts with the overall 3D scene features, focusing on regional aspects of the scene and aggregating explicitly similar features into grouped regional features. The implicit diffusion branch starts with the visible voxel query suggestions output by the query suggestion network, focusing on the details of objects in the scene and learning the implicit fine-grained features of the objects. A hierarchical grouping module is also proposed, which divides the 3D space into multiple sub-regions and performs similarity calculation and aggregation on the voxel features within each region, thereby generating a compact regional feature representation. This grouping method can focus on the local regions of specific objects in the scene, avoiding mutual interference from features of independently distributed objects in the distance, thus extracting more detailed and accurate semantic features.

This application relates to a method, apparatus, device, and computer-readable medium for dynamic garbage data recycling. The method includes: collecting operating parameters of a distributed object storage cluster and submitting these parameters to an analysis cluster; constructing dynamic load profiles of multiple storage buckets in the distributed object storage cluster based on the operating parameters in the analysis cluster; generating corresponding dynamic recycling strategies for each storage bucket according to its dynamic load profile; and recycling garbage data from each storage bucket according to the dynamic recycling strategies. This application constructs dynamic load profiles of storage buckets based on the collected operating parameters, achieving real-time perception and accurate characterization of the cluster's load status. This allows for the generation of targeted dynamic recycling strategies adapted to the load characteristics of each storage bucket, replacing the traditional static threshold triggering mechanism. This avoids resource contention during peak business periods and solves the technical problem that the static threshold triggering mechanism for garbage data recycling cannot adapt to dynamic load changes, leading to system performance degradation.

The invention relates to a data storage method and device based on distributed object storage and electronic equipment. The method comprises the following steps: under the condition that a target storage request of a target user is received, obtaining target data to be stored according to the target storage request; the target data is stored in a target storage block in a target storage bank of a data node, the data node comprises a plurality of storage banks, and each storage bank comprises a plurality of storage blocks; a unique object identifier of the target data in the target memory bank is generated, and a corresponding relation between the unique object identifier and a memory block is recorded in the target memory bank; and storing the storage bank unique identifier of the target storage bank and the corresponding relationship between the unique object identifier and the unique user identifier of the target user. The technical problem of storage space resource waste in distributed object storage is solved.

The invention provides a memory page allocation method, a readable storage medium and a computer program product. The allocation method is applied to a random access memory of a microcontroller, and comprises the following steps: obtaining a to-be-allocated object; sequentially taking each page in the N pages as a to-be-selected page, and according to the to-be-distributed object and the adjacent relation data, determining a page cutting overhead reduction amount between the to-be-distributed object and all adjacent objects when the to-be-distributed object is distributed to the to-be-selected page; according to the page cutting overhead reduction amount, the size of the to-be-distributed object and the page size of the to-be-selected page, calculating the income density when the to-be-distributed object is distributed to the to-be-selected page; wherein the adjacent object is an object having an adjacent access relationship with the to-be-distributed object; determining a target page from the N to-be-selected pages according to the revenue density, and distributing the to-be-distributed object to the target page; according to the method, the page cutting overhead of the RAM of the microcontroller can be reduced, and the ROM space of the microcontroller can be optimized.

The application discloses a data backup system and method, a storage medium and an electronic device, and relates to the field of financial technology. The system comprises: a ledger database, which is used to acquire information of a database group and generate a backup strategy corresponding to the database group; a backup management platform, which is used to acquire the backup strategy, execute a backup task of the database group through a backup database in the case that a backup mode is remote backup, and backup data generated by the backup task to a distributed object storage system; and the distributed object storage system, which is used to receive the data generated by the backup task. Through the application, the problems of large occupation of built-in SSD storage disk space by backup data and difficult management of backup data in the related art are solved.

The invention relates to the technical field of distributed storage, and discloses a method and device for updating distributed object storage homonymous large objects, and the method comprises the steps: obtaining an initial serial number of each segment in an initial large object stored in a distributed system, and a first verification value of the initial segment corresponding to each initial serial number; obtaining the data volume of a first segment in the initial large object, and dividing the to-be-uploaded large object into multiple segments based on the data volume to obtain multiple to-be-uploaded segments; calculating a verification value of each to-be-uploaded segment to obtain a to-be-uploaded serial number of the to-be-uploaded segment and a second verification value of the to-be-uploaded segment corresponding to each to-be-uploaded serial number; and under the condition that the target to-be-uploaded serial number is the same as the numerical value represented by the initial serial number, if the second verification value of the to-be-uploaded segment corresponding to the target to-be-uploaded serial number is different from the first verification value of the initial segment corresponding to the initial serial number, uploading the to-be-uploaded segment corresponding to the target to-be-uploaded serial number.

The invention discloses a water conservancy knowledge base and intelligent question and answer construction method based on a large model and RAG, and the method comprises the steps: 1, combing a water conservancy knowledge system, and sorting water conservancy professional documents and data as materials of the knowledge base; step 2, constructing a water conservancy knowledge library management system through a J2EE system, a B / S architecture and a distributed object storage system; step 3, based on the RAG technology, forming different knowledge bases according to the sorted knowledge classification and confidentiality security level of the water conservancy industry, and establishing a water conservancy knowledge intelligent question-answering system on the knowledge bases; 4, performing extended application on the basis of the constructed water conservancy knowledge intelligent question-answering system; the problems that in the prior art, due to the fact that unstructured data processing is weak when a water conservancy knowledge base is constructed, a large number of image-text mixed documents such as PDF and PPT cannot be effectively utilized, and the recall rate and the accuracy rate are not high are solved.

The invention provides a real-time online visualization method and device for multi-resolution grid data, and belongs to the technical field of data processing and visualizing.The method comprises the steps that firstly, grid data are merged into sequence columns in the Z direction to form a multi-resolution stacked grid model, the multi-resolution stacked grid model is stored in an HDF5 file according to resolution levels, and parallel I / O is achieved in combination with MinIO distributed object storage; secondly, a Z-Order index and directory server strategy is designed, and the data scheduling and transmission efficiency is improved; a browser end adopts a virtual quadtree structure to organize data, and realizes multi-thread real-time data receiving in combination with WebWorker and WebSocket, so that the blockage of a main thread is avoided; finally, the LOD level is dynamically selected based on the viewpoint distance, and efficient volume rendering is achieved through a GPU texture array and LOD InfoTexture. According to the method, online real-time updating is supported, the loading delay is remarkably reduced, and the visual frame rate is improved.

The present application belongs to the field of data storage, and relates to a space arrangement method for distributed object storage. The method comprises: acquiring, as source objects, aggregation large objects meeting a hole ratio requirement, and inserting, into a hash list, source objects, fragment spaces of which meet a hole minimum threshold; selecting, from the hash list, the source objects having the highest fragment rate to undergo space arrangement, creating a target aggregation large object on the basis of requirements and using the target aggregation large object as a target object; generating a target object mapping relationship table; migrating data of small objects on the source objects to the target object on the basis of the target object mapping relationship table; after migration is successful, updating layout information corresponding to metadata of the small objects themselves; and executing a cleaning operation, so as to end the task. In the method of the present application, by means of migrating undeleted small objects from original large objects onto a new empty large object, the fragmentation and waste of a storage space are effectively reduced, thereby better improving the utilization rate of the storage space.

This invention provides a method and apparatus for real-time online visualization of multi-resolution grid data, belonging to the field of data processing and visualization technology. First, the grid data is merged into hierarchical columns along the Z-axis to form a multi-resolution stacked grid model, which is then stored in HDF5 files according to resolution levels. Parallel I / O is achieved using MinIO distributed object storage. Second, a Z-Order index and directory server strategy are designed to improve data scheduling and transmission efficiency. On the browser side, a virtual quadtree structure is used to organize the data, and WebWorker and WebSocket are used to achieve multi-threaded real-time data reception, avoiding main thread blocking. Finally, the LOD level is dynamically selected based on the viewpoint distance, and efficient volume rendering is achieved using GPU texture arrays and LODInfoTexture. This method supports online real-time updates, significantly reduces loading latency, and improves the visualization frame rate.

The invention discloses a distributed object storage metadata recovery method and system and a storage medium. The method comprises the steps that when metadata operation is carried out, an incremental log is generated; the main OSD traverses the increment log for each object needing to be recovered, and all increment information of each key of each object needing to be recovered is found; the main OSD determines the last piece of increment information of each key of each object needing to be recovered in all increment information of each key of each object needing to be recovered; when the OSD lacks the key, executing an operation corresponding to the last piece of increment information on each key of each object needing to be recovered; when the main OSD lacks the key, the main OSD executes the operation corresponding to the last piece of increment information on each key of each object needing to be recovered. According to the method and the device, incremental recovery can be adopted during data recovery, so that all key value pairs are prevented from being traversed, the data recovery time is shortened, and the recovery speed and efficiency of the metadata are greatly improved.

The invention discloses a building space landing point selection method and device and a computer readable storage medium, and the method comprises the steps: obtaining the object attribute of a to-be-distributed object and the task attribute of a building distribution task in the execution process of the building distribution task; generating observation options for the current building area according to the object attributes, and obtaining observation data of the current building area according to the observation options; determining to-be-selected landing points of the building distribution task according to observation data, and performing priority ranking on the to-be-selected landing points according to the task attributes; executing the landing operation of the building distribution task according to the priority from high to low; therefore, for dense building scenes such as commercial street areas and residential districts, the landing point selection strategy of the building space is optimized, the pickup difficulty is reduced, and the flight safety is guaranteed.

The application discloses a resource adjustment method, an electronic device, a storage medium and a program product, relates to the technical field of distributed storage, and can dynamically adjust the weight according to the use condition, burst state and priority of real-time IO resources of a client, better meets differentiated IO demands of multiple clients, can be in time and effective response to stable low-delay demands or periodic burst demands, and has strong dynamic adaptability; the application can flexibly and reasonably allocate bandwidth according to real-time demands of each client by dynamically adjusting a resource allocation strategy, enables the system to be more efficient, and provides services for more clients; in a distributed object storage environment, under the condition of equivalent hardware resources, the optimized IO scheduling algorithm enables the system to be more efficient when processing client requests, reduces system response delay, improves data read-write speed, and thus improves the performance of the entire distributed storage system.

The invention discloses an aircraft protection method and device and a computer readable storage medium, and the method comprises the steps: judging whether the current height exceeds a first height threshold value or not when an aircraft enters an emergency state; if so, opening a main parachute of the aircraft; if not, the free landing speed of the current height is estimated, and whether the free landing speed exceeds a second speed threshold value or not is judged; if not, a main air bag of the aircraft is opened, and if yes, a delayed opening instruction of an auxiliary parachute and / or an auxiliary air bag of the delivery object is set firstly, and after the auxiliary parachute and / or the auxiliary air bag are / is separated from the aircraft, the main air bag of the aircraft is opened; therefore, the air bag protection scheme of the unmanned aerial vehicle can be optimized to a great extent aiming at building dense scenes such as commercial streets and residential districts, the distribution requirements of the building dense scenes such as the commercial streets and the residential districts are met, and the flight safety and object safety in emergency situations are guaranteed.

The application discloses a distributed object storage system data monitoring method and device and a medium thereof, relates to the technical field of distributed object storage, and aims to provide an implementation mode for data monitoring, and provides a distributed object storage system data monitoring method for solving the problem that the current data monitoring mode has a great influence on performance. According to the distributed object storage system data monitoring method, the request information is first stored in a cache, part of data in the request information is counted according to a preset monitoring requirement, data monitoring information is obtained, and the data monitoring information is saved in a storage pool, so that the data monitoring information can be called when needed. Therefore, the amount of data written into the storage system or the amount of data read from the storage system when a user queries the data monitoring information is less than the amount of data stored in the storage system in the current mode, and the difference is more obvious when the request information contains more requests.

The present application belongs to the field of data storage, and relates to a space arrangement method for distributed object storage. The method comprises: acquiring, as source objects, aggregation large objects meeting a hole ratio requirement, and inserting, into a hash list, source objects, fragment spaces of which meet a hole minimum threshold; selecting, from the hash list, the source objects having the highest fragment rate to undergo space arrangement, creating a target aggregation large object on the basis of requirements and using the target aggregation large object as a target object; generating a target object mapping relationship table; migrating data of small objects on the source objects to the target object on the basis of the target object mapping relationship table; after migration is successful, updating layout information corresponding to metadata of the small objects themselves; and executing a cleaning operation, so as to end the task. In the method of the present application, by means of migrating undeleted small objects from original large objects onto a new empty large object, the fragmentation and waste of a storage space are effectively reduced, thereby better improving the utilization rate of the storage space.