Parallel metadata acquisition system

Abstract

The invention provides a parallel metadata acquisition system, which is improved on the basis of an existing parallel metadata acquisition frame by combining network technology, mobile Agent technology and an OAI (Open Archives Initiative) frame model to improve the metadata acquisition rate and realize high-efficiency federated search service. By adopting a grouping strategy, the parallel metadata acquisition system realizes parallel acquisition of metadata outside and in groups, thereby improving the acquisition rate on the whole. Additionally, the metadata are stored in balanced classification, so that the system can perform parallel search when responding to a search request, and the searching speed and accuracy are improved.

Description

technical field [0001] The invention relates to a metadata parallel collection system, which belongs to the field of network communication and is used for peer-to-peer network search optimization and resource collection optimization. Background technique [0002] OAI provides an application-independent interoperability framework based on metadata collection, and has two main roles: data provider (DataProvider, DP) and service provider (ServiceProvider, SP). DP is the owner of metadata, and metadata is expressed by public metadata DC, which complies with OAI to publish metadata, and responds to collection requests in XML format. The SP is the data collector and the main body that provides value-added services to users by using the collected metadata. The working principle and process of OAI-based metadata collection are as follows: figure 1 Shown: [0003] In the OAI framework, SP collects and extracts metadata from each DP, processes and merges them and stores them centra...

AI Technical Summary

Problems solved by technology

However, in the OAI framework, there are shortcomings such as performance bottlenecks, long acquisition periods, high performance requirements, single point failures, and heavy loads. In order to solve these shortcomings and improve the performance of the acquisition system, many research institutions and scholars have explored the use of emerging grid technology, trying to use multiple collection nodes to collect DPs in parallel to speed up the collection of metadata. For example, the Digital Library Research Group of OldDominion University uses grid technology to improve the performance of metadata collection and Dr. Zheng Zhiyun in China studies digital library under grid. interoperability framework, etc.

[0004] The existing metadata parallel collection framework has the following disadvantages: (1) The real-time monitoring problem of the collection scheduling server: in the process of resource scheduling for the collection nodes, although the existing cluster technology is used, the RSS algorithm can be used between the collection nodes. Achieve balanced distribution of collection tasks, but the dynamic changes of the collection nodes and DP nodes are not considered during the collection process, the dynamics of the grid are not reflected, the collection nodes are not monitored in real time, and the integrity of the collected data cannot be guaranteed

(2) Synchronous update problem: SP cannot actively and timely reflect the data update operation in DPs

Images