Data reading method, system and device and medium

Abstract

The invention provides a data reading method, system and device and a medium, and the method comprises the steps: obtaining data, storing the data to a local disk and a ceph sharing disk through employing a micro-service, enabling the ceph sharing disk to store full data through employing a distributed file system, and enabling a part of data stored in the local disk to be exclusively shared by the corresponding micro-service; when the micro-service receives the query request, judging whether the service state of each pod is normal or not, and when the service state of each pod is normal, distributing the query request to the Pod with the normal service state to query the local disk, and summarizing and sequencing to obtain a query result; and when the service state of the pod is abnormal,querying partition data corresponding to the pod with the abnormal service state corresponding to the ceph shared disk, and summarizing and sequencing to obtain a query result. The ceph shared disk and the local disk are mixed for use, so that the stability of the system is improved; the performance of the local disk is exclusively shared by a single micro-service, so that the data reading efficiency is improved; and part of data is queried through each pod instead of all data, the whole is broken up into parts, and the overall query speed is provided.

Description

technical field

[0001] The present application relates to the technical field of data processing, in particular to a data reading method, system, device and medium. Background technique

[0002] With the development of storage technology, distributed systems have been widely used. Distributed systems include master nodes and multiple slave nodes for storing the same data. Terminals can write data into distributed systems through master nodes, and read Data stored in any node.

[0003] However, distributed micro-cloud services need to share feature data. Most of the existing technologies use shared storage ceph to store feature data, which has the advantages of high reliability, access to all services, and all nodes. However, in the actual use process, as the data is continuously connected to form a large amount of data, it will cause data redundancy; at the same time, as the disks between nodes use different file reading systems, the speed of data retrieval becomes very slo...

Images