Data storage systems and data storage management methods
A decentralized cloud storage system with resource pools and a unified management platform addresses the challenges of centralized data storage by reducing complexity and costs, minimizing network load, and improving management efficiency through logical binding and load balancing.
Patent Information
- Application Number
- PCT/CN2025/111132
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2024-08-26
- Filing Date
- 2025-07-29
- Publication Date
- 2026-03-05
AI Technical Summary
Existing data storage systems face high implementation difficulties and costs due to centralized construction, leading to data silos and significant communication latency and network traffic pressure in multi-domain scenarios, complicating unified management across domains.
A decentralized cloud storage system with resource pools and a unified management platform, where sensing devices are logically bound to resource pools, reducing traffic interaction between pools and enabling unified management and load balancing through a topology configuration file.
The solution reduces deployment complexity and costs while minimizing network load and enhancing system management efficiency by eliminating inter-pool traffic and facilitating unified storage and access across domains.
Smart Images

Figure CN2025111132_05032026_PF_FP_ABST
Abstract
Description
DATA STORAGE SYSTEMS AND DATA STORAGE MANAGEMENT METHODSCROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application claims priority to Chinese application No. 202411177403.3 filed on 08 / 26 / 2024, the entire contents of which are incorporated herein by reference.TECHNICAL FIELD
[0002] This disclosure relates to distributed storage technology, and in particular, relates to a data storage system and a data storage management method.BACKGROUND
[0003] To effectively store the massive data generated by front-end sensing devices, it is common practice to centrally construct a storage system for unified data management. However, this approach faces relatively high implementation difficulties and costs. To address this, existing technologies propose decentralized multi-domain deployment with unified management portals. Nevertheless, in multi-domain scenarios, each domain independently manages device access and data storage for its own front-end sensing devices, which tends to create data silos. While cross-domain data sharing and aggregation are required, data sharing across domains remains inconvenient, and aggregating data from different domains incurs significant overhead, potentially causing traffic pressure on core backbone networks and substantial communication latency. Additionally, despite the unified management portal, the need for separate operations and management across individual domains complicates workflows, preventing genuine global unified management, unified load, and unified storage.
[0004] Therefore, there is a need to provide a data storage system and a data storage management method that reduces the deployment complexity and costs of the system while minimizing the load pressure on the core backbone networks and enhancing the ease of system management.SUMMARY
[0005] One of the embodiments of the present disclosure provides a cloud storage system, comprising: sensing devices configured to collect data; resource pools, wherein each of the resource pools includes data nodes, and each of the sensing devices is configured to be in a communication connection with at least one data node in one of the resource pools; and a management platform communicatively connected to the resource pools, and configured to manage the resource pools and / or the data nodes within each resource pool.
[0006] One of the embodiments of the present disclosure provides a method for cloud storage management, executed by a management platform in a cloud storage system, wherein the cloud storage system further includes sensing devices and resource pools with no traffic interaction between the resource pools, and each of the sensing devices is logically bound to a resource pool of the resource pools, and output data of the sensing device is stored in the resource pool, the method comprising: obtaining a topology configuration file of the cloud storage system; establishing a communication connection between the management platform with each of the resource pools, wherein the topology configuration file defines the resource pools and data nodes within each resource pool; and managing the each resource pool and / or the data nodes within the each resource pool based on the topology configuration file.
[0007] One of the embodiments of the present disclosure provides a method for cloud storage management, executed by a management platform in a cloud storage system, wherein the cloud storage system further includes resource pools, and at least one sensing device is logically bound to a resource pool of the resource pools , output data of the at least one sensing device being stored in the resource pool, the method comprising: establishing a communication connection with each of the resource pools, wherein each resource pool includes data nodes; receiving and responding to a business request, wherein the business requests include at least one of an intra-pool access request, an intra-pool forwarding request, an intra-pool storage request, and a distributed storage request.
[0008] One of the embodiments of the present disclosure provides an electronic device comprising a processor and a memory connected to each other, wherein the memory is configured to store a program, and the processor is configured to execute the program to implement a method according to any of the embodiments of the present disclosure.
[0009] One of the embodiments of the present disclosure provides a computer-readable storage medium having a program stored thereon, the program being executed by a processor to implement a method described in any of the embodiments of the present disclosure.BRIEF DESCRIPTION OF THE DRAWINGS
[0010] This disclosure will be further illustrated by way of exemplary embodiments, which will be described in detail by means of the accompanying drawings. These embodiments are not limiting, and in these embodiments, the same numbering denotes the same structure, wherein:
[0011] FIG. 1A is a schematic diagram illustrating an exemplary cloud storage system according to some embodiments of the present disclosure;
[0012] FIG. 1B is a schematic diagram illustrating an exemplary cloud storage system according to some other embodiments of the present disclosure;
[0013] FIG. 2 is a schematic diagram illustrating a topology configuration file of data nodes in a cloud storage system according to some embodiments of the present disclosure;
[0014] FIG. 3 is a schematic diagram illustrating exemplary hardware and / or software components of a computing device according to some embodiments of the present disclosure;
[0015] FIG. 4 is an exemplary flowchart illustrating a process of cloud storage management according to some embodiments of the present disclosure;
[0016] FIG. 5 is an exemplary flowchart illustrating a process of cloud storage management according to some other embodiments of the present disclosure;
[0017] FIG. 6 is a diagram of an electronic device according to some embodiments of the present disclosure; and
[0018] FIG. 7 is a diagram of a computer-readable storage medium according to some embodiments of the present disclosure.DETAILED DESCRIPTION
[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the accompanying drawings required to be used in the description of the embodiments are briefly described below. Obviously, the accompanying drawings in the following description are only some examples or embodiments of the present disclosure, and it is possible for a person of ordinary skill in the art to apply the present disclosure to other similar scenarios in accordance with these drawings without creative labor. Unless obviously obtained from the context or the context illustrates otherwise, the same numeral in the drawings refers to the same structure or operation.
[0020] It should be understood that the terms “system, ” “device, ” “unit” and / or “module” used herein are one method to distinguish different components, elements, parts, sections, or assemblies of different levels. However, the terms may be displaced by another expression if they achieve the same purpose.
[0021] As shown in this disclosure and the claims, the words “a, ” “one, ” and / or “the” do not refer specifically to the singular forms but may also include the plural forms as well, unless the context clearly indicates otherwise. Generally, the terms “including” and “comprising” suggest only the inclusion of clearly identified steps and elements, and do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.
[0022] The terms "system" and "network" are often used interchangeably in this disclosure. The term "and / or" , as used in this disclosure, is simply a way of describing the associative relationships of associated objects, indicating that three relationships can exist, for example, A and / or B can mean: A alone, both A and B, and B alone. Also, the character " / " , in this disclosure, generally indicates that the before and after associated objects are an "or " relationship. Additionally, the character "more" in this disclosure indicates two or more.
[0023] As mentioned earlier, the cloud storage system adopts a multi-domain management mode with a unified management portal. However, in multi-domain scenarios, cross-domain data sharing and aggregation are often required. While cross-domain data sharing remains inefficient, aggregating data from different domains incurs significant overhead, which may lead to traffic pressure on core backbone networks and prolonged communication latency. Additionally, despite the unified management portal, the need for separate operations and management across individual domains complicates workflows, preventing genuine global unified management, unified load, and unified storage.
[0024] This disclosure provides a data storage system including resource pools and a management platform, and each of the resource pools includes data nodes. A data node may be configured to provide one or more services associated with data processing. In some embodiments, each of the data nodes includes sub-nodes, and each of the sub-nodes is configured to provide a service in response to a corresponding service request. The service may include at least one of an intra-pool access service, an intra-pool forwarding service, an intra-pool storage service, or a distributed storage service. At least one sensing device is configured to be logically bound to a resource pool, such that output data of the at least one sensing device may be stored in at least one data node in the resource pool. The management platform is communicatively connected to the resource pools, and configured to manage the resource pools and / or the data nodes within each resource pool. The data storage system (cloud storage system) provided in the embodiments of the present disclosure, on the one hand, due to the decentralized construction of the resource pools, helps to reduce the difficulty of construction and construction cost of the cloud storage system, as compared to the centrally constructed system; on the other hand, as storage, access, forwarding, and other services are provided in the same resource pool, and there is no traffic interaction between the resource pools, storage services and other traffic can be avoided as much as possible over the core backbone networks, which helps to reduce the load pressure on the core backbone networks; and on the other hand, as the resource pools are managed together through one management platform, unified management, unified load, and unified storage can be provided as compared to the multi-domain model. Thus, the cloud storage system and the cloud storage management method provided in the present disclosure are able to reduce the load pressure of the core backbone networks as much as possible under the premise of reducing the difficulty of constructing and the construction cost of the cloud storage system and improve the convenience of system management.
[0025] FIG. 1A is a schematic diagram of an exemplary cloud storage system according to some embodiments of the present disclosure.
[0026] As shown in FIG. 1A, the cloud storage system 100 includes a management platform 110, resource pools 120, and sensing devices 130. Each of the sensing devices 130 is configured to be logically bound to a resource pool of the resource pools, and output data of one of the sensing devices 130 may be stored in at least one of data nodes within the resource pool. For example, a first sensing device is logically bound to a first target resource pool and is in a communication connection with a first target node (also referred to as a first target data node) in the first target resource pool. The first target node is configured to obtain and store output data of the first sensing device.
[0027] The management platform 110 may be communicatively connected with each of the resource pools 120, and configured to perform management (e.g., load balancing and scheduling management) of each of the resource pools and / or data nodes within each of the resource pools. It should be noted that the management platform 110 may also be referred to as a data node in the could storage system 100.
[0028] In some embodiments, the management platform 110 may be implemented by a single server or a group of servers. In some embodiments, the management platform 110 may be implemented on a processor or a group of processors. In some embodiments, the management platform 110 may be implemented on a user terminal.
[0029] In some embodiments, the management platform 110 includes a platform as a service (PaaS) platform. PaaS is a cloud computing service model that provides a platform for users to be able to develop, run, and manage applications without having to be concerned about the underlying infrastructure and software configuration.
[0030] In some embodiments, the management platform 110 includes a software as a service (SaaS) platform. The SaaS platform is a model for providing software applications over the Internet, which can be accessed and utilized by users over the Web without the need to install or maintain the software applications on a local device. For example, when the cloud storage system 100 is applied in a streaming media storage and management scenario, the SaaS platform is an online platform that provides streaming media-related services (e.g., streaming media forwarding services, streaming media storage services, streaming media access services, etc. ) to users via cloud computing.
[0031] In some embodiments, the management platform 110 includes a PaaS management platform and a SaaS management platform. For example, the management platform 110 includes a service management platform 112 and a storage management platform 113. The service management platform 112 is the SaaS management platform and the storage management platform 113 is the PaaS management platform. For more details on the service management platform 112 and the storage management platform 113, please refer to the description below.
[0032] In some embodiments, the management platform 110 (e.g., the storage management platform 113) obtains a topology configuration file of the resource pools and manages each resource pool and / or the data nodes within each resource pool based on the topology configuration file. The topology configuration file may define the resource pools and the data nodes within each of the resource pools. The topology configuration file may define a topology structure of resource pools and / or data nodes in the could storage system 100. For example, the topology configuration file may include the identity of each resource pool, the type of each resource pool, the name of the resource pool, information of the data nodes (e.g., the number of data nodes in the resource pool and / or the IP address of each data node) contained in each resource pool, etc.
[0033] In some embodiments, the management platform 110 (e.g., the service management platform 112 and the storage management platform 113) is configured to receive and respond to a service request. The service request may include at least one of an intra-pool access request, an intra-pool forwarding request, an intra-pool storage request, and a distributed storage request.
[0034] A resource pool is a collection of resources and properties configured to manage and / or process data (e.g., media content such as videos, audios, images, etc. ) , and each resource pool includes data nodes. A data node refers to a physical server or a virtual machine in the cloud storage system that is used to store data, manage data, and / or access data. For example, a data node may include a CPU, memory, a storage device (e.g., a disk) , an internet port, etc.
[0035] A resource pool may include a data center, one or more servers-mounted on a rack, cloud-direct storage nodes, etc.
[0036] The data center may include a plurality of servers (e.g., 5, 10, 20, 30, 50 servers, etc. ) , and each server may be a data node. For example, the resource pool 120 is a data center including a plurality of servers. The data center may adopt a modular, hierarchical, and flat design idea with virtualized segmentation and management technology to divide the plurality of servers into units, manage them one by one, and implement the delivery, acceleration, displaying, computing, and storing of data and information.
[0037] The rack is a structure in the data center that is configured to mount and secure hardware equipment such as servers, network equipment, storage devices, etc. For example, the resource pool 120 includes a plurality of servers mounted and secured in a single rack or includes a plurality of servers mounted and secured in a plurality of racks (each of the plurality of racks has one or more servers mounted and secured therein) .
[0038] A cloud direct storage node, also known as a storage node, may be used for storing data (including caching, or persistent storage) , data management, and access. Direct storage refers to data access directly through a storage interface, bypassing traditional storage protocols (e.g., file systems) . Each of the cloud direct storage nodes may include a plurality of servers. For example, each of the resource pools 120 includes a plurality of cloud direct storage nodes, and each of the plurality of cloud direct storage nodes includes a plurality of servers, i.e., the resource pool 120 includes a plurality of sets of servers.
[0039] In some embodiments, the types of the resource pools may be the same, or at least partially different. For example, the first resource pool in the cloud storage system 100 is composed of a data center, the second resource pool is composed of one or more servers mounted on a rack, and the third resource pool is composed of a plurality of cloud direct storage nodes. As another example, the first resource pool and the second resource pool in the cloud storage system 100 are composed of data centers, and the third resource pool is composed of a plurality of servers mounted on a rack or a plurality of cloud direct storage nodes. As still another example, the resource pools in the cloud storage system 100 are data centers.
[0040] By setting the data center, the servers mounted on a rack, or the plurality of cloud direct storage nodes as a resource pool, the cloud storage system 100 is capable of realizing sub-pool management (management of different resource pools) of different hardware types (e.g., different configurations, different models, different operating systems) , and realizing hardware homogeneity within the pool (within the same resource pool) and heterogeneity between the pools (between different resource pools) in compatible scenarios, so as to be able to support the construction of multi-phase expansion of heterogeneous hardware in the same cloud storage system, which helps to solve the problem of heterogeneous expansion.
[0041] The resource pools in the cloud storage system 100 are in the same storage domain (e.g., a PaaS domain) . In some embodiments, at least some of the resource pools have no traffic interaction with each other. In some embodiments, the resource pools have no traffic interaction with each other. In some embodiments, the resource pools have traffic interactions with each other. For example, with the purpose of load balancing of the cloud storage system 100 as a whole, the data stored within resource pool 1 is distributedly stored to one or more resource pools of the cloud storage system 100 other than the resource pool 1.
[0042] In cloud architecture, a domain refers to a fully functional PaaS cluster (i.e., the PaaS domain) , which can access front-end sensing devices (e.g., Internet of Things (IoT) devices) , store data, transmit data, analyze data, and share data, and provide interfaces to provide data-based service capabilities for the upper-layer service platform.
[0043] The cloud architecture refers to the structure and processes for designing, deploying, and integrating computational resources, storage resources, network resources, and services in a cloud computing environment. A complete cloud architecture includes front-end components, back-end components, a network, and a management and security layer.
[0044] The front-end components may include a user interface and a user terminal. For example, the user interacts with the cloud service with the aid of the user terminal via a browser, a mobile application, or a dedicated interface. The user terminals may include personal computers, mobile devices, and other connectable smart devices (e.g., the sensing device 130) .
[0045] The back-end components may include servers, storage devices, and databases. The servers may be distributed in one or more data centers, and the cloud service provider may process the service request from the front-end components through the servers. The storage device is used for cloud storage of user data and application data. The cloud storage may be categorized into object storage, block storage, and file storage. The database, also referred to as a cloud database, may include a relational database (e.g., MySQL, PostgreSQL) or a non-relational database (e.g., MongoDB, Cassandra) .
[0046] The cloud architecture may connect the front-end components to the back-end components over the Internet or a private network to ensure rapid data transfer and communication.
[0047] Management tools for the management and security layer may be configured to monitor, manage, and maintain cloud resources including, but not limited to, load balancing, auto-scaling, etc. Security measures of the management and security layer may include authentication, encryption, access control, etc. for safeguarding data security and privacy.
[0048] The PaaS cluster is a cluster architecture including a plurality of PaaS instances or nodes. The PaaS cluster may include a master node and a worker node. For example, the cloud storage system 100 may be a cloud architecture that includes a PaaS cluster. The management platform 110 in the cloud storage system 100 is a PaaS master node, and each of the resource pools 120, or each data node 121 within each of the resource pools, corresponds to a PaaS worker node. The resource pools 120, or each data node 121 within each of the resource pools may be back-end components. The management platform 110 may be a management and security layer. The sensing devices may be front-end components.
[0049] In an embodiment of the present disclosure, the resource pools in the cloud storage system 100 share a single management platform 110, and thus the cloud storage system 100 is regarded as a PaaS domain having decentralized multi-location construction characteristics in terms of a cloud architecture while being managed by only one management platform 110.
[0050] In some embodiments, the resource pool 120 includes the data nodes 121 as shown in FIG. 1A. For example, each server in the data center is a data node 121 when the resource pool 120 is the data center. As another example, the resource pool 120 includes a plurality of servers installed in a single rack, each server is the data node 121; or the resource pool 120 includes a plurality of servers installed in a plurality of racks, each server or a plurality of servers in each rack is the data node 121. As still another example, when the resource pool 120 includes a plurality of cloud direct storage nodes, each cloud direct storage node is a data node.
[0051] In some embodiments, each of the data nodes 121 in the resource pool 120 is located in the same geographic location, and the data nodes 121 belonging to different resource pools are located in different geographic locations. That is, the resource pools are dispersed and built in a plurality of locations. In some embodiments, the same geographic location is the same physical space (e.g., within the same computer room) . For example, a plurality of data nodes 121 (e.g., one data node is a server) organized in the same server room belongs to the same resource pool 120, and different data nodes in different server rooms belong to different resource pools. In other words, different resource pools are set up in different server rooms. In some embodiments, the same geographic location may be the same data center. For example, a plurality of data nodes 121 (e.g., one data node is a server) in the same data center belong to the same resource pool 120, and data nodes in different data centers belong to different resource pools. In other words, different resource pools are set up in different data centers. In some embodiments, the same geographic location may be the same administrative area (e.g., district, city, etc. ) . For example, a plurality of data nodes 121 (e.g., one data node is a server) in the same administrative region belong to the same resource pool 120, and the data nodes in different administrative regions belong to different resource pools. In other words, different resource pools are set up in different administrative regions.
[0052] It should be noted that, as previously mentioned, the specific range of the geographic location can be set according to the actual situation, for example, it can be set as a server room, a data center, an administrative area, etc., and there is no limitation on the specific range of the geographic location herein; in order to achieve load balancing within a resource pool, the number of data nodes contained in each of the resource pools can be set according to the actual situation, and no limitation is made herein.
[0053] In some embodiments, each of the data nodes 121 may be one or more servers. In some embodiments, each of the data nodes 121 has a corresponding storage device, i.e., dividing the distributed storage nodes for poll resources as well. Merely as an example, the cloud storage system 100 includes a distributed storage structure including a plurality of servers, the distributed storage structure including a plurality of storage nodes, and each of the data nodes is a storage node in the distributed storage structure.
[0054] The sensing device 130 is configured to acquire data (e.g., acquire environmental data such as temperature, humidity, light, gas concentration, motion, etc. ) , and / or to interact with a user and perform basic processing (e.g., control motors, light brightness, switches, etc. ) , etc. Formats of the acquired data may include, but are not limited to: video, images, structured data, etc. The sensing device may include an IoT sensing device. The IoT sensing device is a device that is in the first line of operation and data generation in an IoT system, also referred to as a front-end sensing device.
[0055] In some embodiments, the sensing device 130 includes a sensor and an actuator. For example, the sensor includes, but is not limited to, a temperature sensor, a humidity sensor, an accelerometer, a gyroscope, a pressure sensor, etc. The actuator may include a microcontroller (e.g., Arduino, Raspberry Pi, ESP8266 / ESP32, etc. ) or a microprocessor. In some embodiments, the sensing device 130 also includes a communication module and a power supply. The communication module may be used for data transfer between devices, or between a device and a cloud platform. The communication protocol may include Wi-Fi, Bluetooth, Zigbee, LoRa NB-IoT, etc. The power supply may include a battery, solar power, power adapter, etc. The above examples are only a few possible examples in the process of practical application, and the sensing device may also include other kinds of devices, and the specific kinds of sensing devices are not limited here.
[0056] The sensing device 130 is configured to be communicatively connected to at least one data node in one of the resource pools. For example, the sensing device 130 is in a communication connection with a first transmission node within a first target resource pool, or with a second transmission node within the first target resource pool. As another example, the sensing device 130 is in a communication connection with a third target node within a third target resource pool. It is noted that a resource pool 120 may be logically bound to a plurality of sensing devices 130, and a sensing device 130 is configured to be logically bound to a resource pool.
[0057] In some embodiments, each data node in each of the resource pools includes sub-nodes, each of the sub-nodes being configured to provide a corresponding service (or business) in response to a service request. The service may include at least one of an intra-pool access service, an intra-pool forwarding service, an intra-pool storage service, and a distributed storage service.
[0058] The intra-pool access service, configured to provide access to the sensing device 130, refers to a service of establishing a communication connection between the sensing device 130 and a data node 121 within the resource pool 120. For example, in the streaming scenario, the intra-pool access service, which also is referred to as a streaming access service, is configured to provide a service of establishing a communication connection between a front-end IoT device and at least one data node 121 within a resource pool 120. A service request for providing the intra-pool access service is referred to as an intra-pool access request.
[0059] The intra-pool forwarding service is configured to forward stored data (e.g., video, audio, images, or structured data) to other end devices (e.g., other user terminals) or other systems. For example, in a streaming scenario, the intra-pool forwarding service includes at least one of a video playback (i.e., forwarding of video data) service and an image access (i.e., forwarding of image data) service. The service request for providing the intra-pool forwarding service is referred to as an intra-pool forwarding request.
[0060] The intra-pool storage service is configured to manage data storage, such as formatting uniform conversions, calling interfaces, etc., to enable storage. For example, in a streaming media scenario, the intra-pool storage service is configured to obtain streaming media data (e.g., video, pictures, audio, structured data, etc., acquired by the front-end IoT device) and call the cloud storage interface of the distributed cloud storage service to provide the storage of the streaming media data. A service request for providing the intra-pool storage service may be referred to as the intra-pool storage request.
[0061] The distributed cloud storage service is configured to provide the data slicing and storage function, that is, the data is scattered and stored in a plurality of storage nodes or storage spaces. For example, in a streaming media scenario, the distributed cloud storage service is configured to provide for slicing and then decentralized storage of streaming media data in the data nodes in the same resource pool 120. The service request configured to provide the distributed cloud storage service is referred to as the distributed storage request.
[0062] That is, the intra-pool storage service is mainly configured to process data related to the storage service, and does not perform specific storage functions, which are provided by the distributed cloud storage service.
[0063] A plurality of service modules (also referred to as logical or functional modules) may be set up for each data node, and each of the services (or businesses) described in the preceding section may be performed by one service module of the data node. For example, each service is executed by a virtual machine in a server (or a cloud direct storage node) corresponding to the data node, the virtual machine being one or more logical computing capabilities (e.g., in the form of one or more computational programs stored in the server) created on the server using virtualization techniques.
[0064] In this embodiment, each sub-node in the data node 121 may be a logical module for providing a corresponding service, and each sub-node is configured to perform one service. For example, an intra-pool access sub-node defines how to implement streaming access, such as associated protocols, signaling transmissions, etc., without limitation herein. As another example, an intra-pool storage sub-node defines how to implement streaming storage, such as a distributed storage protocol, a long-lasting storage protocol, a caching protocol, etc. Similarly, other "sub-nodes" can be defined with corresponding information, and examples will not be provided here.
[0065] FIG. 1B, for example, is a schematic diagram illustrating an exemplary cloud storage system according to some other embodiments of the present disclosure. As shown in FIG. 1B, the data node 121 includes an intra-pool access sub-node for providing the intra-pool access service in response to receiving an intra-pool access request; an intra-pool forwarding sub-node for providing the intra-pool forwarding service in response to receiving an intra-pool forwarding request; an intra-pool storage sub-node for providing the intra-pool storage service in response to receiving an intra-pool storage request; and a distributed storage sub-node for providing the distributed storage service (also called the distributed cloud storage service) in response to receiving a distributed storage request. It should be noted that the sub-nodes shown in FIG. 1B are merely for illustration, and in some embodiments, at least two sub-nodes may be integrated into a single sub-node. For example, the intra-pool storage sub-node and the distributed storage sub-node may be integrated into one single sub-model for storage. As another example, the intra-pool access sub-node and the intra-pool forwarding sub-node may be integrated into one single sub-model for access and forwarding.
[0066] Merely as an example, when the sensing device 130 is logically bound to the resource pool 120, or is in a communication connection with a data node within the resource pool 120, the output data of the sensing device 130 is transparently transmitted to an intra-pool storage sub-node of the data node 121 that is in a communication connection with the sensing device 130. The intra-pool storage sub-node is configured to invoke the distributed storage sub-node to store the output data.
[0067] The output data of the sensing device includes environmental data acquired by the sensing device, such as at least one of an image, a video, an audio, or structured data.
[0068] In some embodiments, the intra-pool forwarding service may include a video playback service and an image retrieval service. The video playback service and an image retrieval service may be provided through different sub-nodes. For example, the data node 112 includes a video forwarding sub-node and an image gateway sub-node (not shown in the figure) for providing the video playback service (also referred to as a video forwarding service) and the image retrieval service, respectively.
[0069] It should be noted that in some embodiments, the data node 121 may not contain a sub-node, and the data node 121 directly performs at least one of the intra-pool access service, the intra-pool forwarding service, the intra-pool storage service, and the distributed storage service.
[0070] In some embodiments, the intra-pool access service, the intra-pool forwarding service, the intra-pool storage service, etc., are PaaS-related services. For example, the streaming media access service specifically is a PaaS streaming media access service, the streaming media forwarding service specifically is a PaaS streaming media forwarding service, and the streaming media storage service specifically is a PaaS streaming media storage service.
[0071] In some embodiments, the management platform 110 includes a service management platform 112 and a storage management platform 113, as shown in FIG. 1B.
[0072] The service management platform 112 is communicatively connected to the resource pools 120. The service management platform 112 is configured to receive and respond to a service request.
[0073] In some embodiments, the service management platform may be communicatively connected to a terminal (e.g., the sensing device 130, a user terminal) . A user may remotely access, manage, and utilize data (e.g., video, images, audio, etc. ) for storage, processing, access, playback, and forwarding, and other related services, through the user terminal.
[0074] The service management platform may include a front-end application and a back-end server. The application is a software program that provides specific functionality to a user and is installed on the user terminal of the user. The server may provide the application program with services related to storage, processing, access, playback, forwarding, etc. The user may perform input operations via a user interface of the application to request the server for the provision of corresponding services.
[0075] As shown previously, the service management platform 112 may be the SaaS management platform.
[0076] For example, taking the service management platform 112 as the SaaS management platform as an example, a user inputs a service request to the SaaS management platform (i.e., the service management platform 112) via an application program (e.g., a software for the SaaS at the user's terminal) installed on the user terminal, the service management platform 112 receives the service request (e.g., a streaming media access request) sent by the user, and performs a corresponding service in response to receiving the service request (e.g., determining a first target resource pool and / or a first target node in response to receiving the streaming media access request) .
[0077] The service management platform 112 may be configured to receive a service request (e.g., the intra-pool access request, the intra-pool forwarding request, the intra-pool storage request, the distributed storage request, etc. ) and respond to the received service request. For example, the service management platform 112 receives the intra-pool access request and sends the intra-pool access request to the storage management platform 113 so that the storage management platform 113 determines, based on a physical location of the first sensing device, a first target resource pool and / or a first target node. As another example, the service management platform 112 receives the intra-pool access request and determines, based on the physical location of the first sensing device, a first target resource pool and / or a first target node.
[0078] In some embodiments, the service management platform 112 includes one or more servers.
[0079] The storage management platform 113 may be communicatively connected with each resource pool 120 for load balancing and scheduling management within each resource pool 120 based on the topology configuration file. That is, the load balancing and scheduling management may both be done within the pool (within the same resource pool) . For example, the storage management platform 113 determines a first target node of the first sensing device based on the load of each of the data nodes 121 within the first target resource pool. The first target node may be configured to receive output data of the first sensing device. The storage management platform 113 may include one or more servers. In some embodiments, the storage management platform 113 and the service management platform 112 are integrated on the same server. In some embodiments, the storage management platform 113 and the service management platform 112 are integrated on different servers.
[0080] The load balancing refers to balancing and distributing loads (e.g., services) across a plurality of operating units, such as FTP servers, Web servers, enterprise core application servers, and other major task servers, in order to collaborate on work tasks.
[0081] The load is the total amount of data traffic handled by a system device (e.g., a server, etc. ) during a specific time period. The load may be measured in bits per second (bps) , bytes per second (Bps) , or packets per second (pps) . Load size may be related to the number of system accesses (e.g., the number of the sensing devices accesses) , data type (e.g., video, images, etc. ) , data volume size, traffic patterns (e.g., burstiness and persistence of traffic) , network topology, and other properties. For example, the higher the number of the sensing devices accessed by the same data node, the higher the corresponding load. As another example, a data node acquiring and / or storing data such as video, large files, and other types of data corresponds to a larger load due to more network bandwidth consumption. As still another example, a data node suddenly needs to provide a plurality of services at a certain time period (there is bursty traffic) , which will result in an increased load on that data node during that time period; or the data node provides a plurality of services on a continuous basis, with a correspondingly higher load. As still another example, a data node that contains a higher number of sub-nodes has a correspondingly higher load due to the need to process more services.
[0082] In some embodiments, the storage management platform 113 includes the PaaS management service and a cloud storage management service.
[0083] The PaaS management service may be configured to enable service management within each of the resource pools 120. Specifically, the PaaS management service may communicate with each sub-node in each data node 121 within each resource pool 120 to send corresponding service to the each sub-node (e.g., sending the intra-pool access request to the intra-pool forwarding sub-node, sending the intra-pool forwarding request to the intra-pool forwarding sub-node, sending the intra-pool storage request to the intra-pool storage sub-node, sending the distributed storage request to the distributed storage sub-node, etc. ) , and the respective sub-node may provide a corresponding service according to the service request.
[0084] The cloud storage management service is configured to provide the management and load balancing of storage services within each resource pool 120. Specifically, the cloud storage management service in the storage management platform 113 may communicate with the intra-pool storage sub-node of each data node 121 within each resource pool 120 to send a corresponding instruction (e.g., the intra-pool storage request, etc. ) to the intra-pool storage sub-node, and the intra-pool storage sub-node may call the distributed storage sub-node in accordance with the instruction to provide the distributed cloud storage service.
[0085] Merely as an example, in conjunction with FIG. 1B, the cloud storage system 100 shown in FIG. 1B may be the PaaS cluster, with the service management platform 112 and the storage management platform 113 as two master nodes, and each of the service management platform 112 and the storage management platform 113 is the PaaS node. The data nodes 121 are a plurality of worker nodes, each of which corresponds to a PaaS node, and each of the intra-pool access sub-node, the intra-pool forwarding sub-node, the intra-pool storage sub-node, and the distributed cloud storage sub-node in each data node 121 is a PaaS sub-node. The PaaS master node service management platform 112 and the storage management platform 113 may communicate with each of the PaaS worker nodes and the PaaS sub-nodes to perform management of each of the PaaS worker nodes and the PaaS sub-nodes.
[0086] In some embodiments, when the service request is the intra-pool access request, the management platform 110 is configured, in response to receiving the intra-pool access request associated with a first sensing device, to obtain, based on a physical location of the first sensing device, a distance between the first sensing device and each of the resource pools respectively; and determine a first target resource pool from the resource pools based on the distance between the first sensing device and the each of the resource pools respectively, and the first sensing device is configured to logically bind to the first target resource pool.
[0087] In some embodiments, the intra-pool access request is manually entered by a user via a user interface of an application (e.g., a SaaS application) implemented on a user terminal. For example, by launching the application of the user terminal that is in communication with the first sensing device, the user enters an instruction for an intra-pool access request by voice, text, etc., in the user interface of the application.
[0088] In some embodiments, the intra-pool access request is automatically generated. For example, the management platform 110 automatically generates the intra-pool access request based on a predetermined communication protocol with the sensing device 130. Merely as an example, the first sensing device sends an intra-pool access request to the management platform 110 (e.g., the service management platform 112) using the HTTP protocol or WebSocket to request logical binding to the resource pool 120.
[0089] Combined with FIG. 1B, with the first sensing device being any of the sensing devices 130 in the figure as an example, the service management platform 112 may, in response to receiving the intra-pool access request (e.g., the streaming media access request) , obtain, based on the physical location of the sensing device 130, the distance between the sensing device 130 and each of the resource pools 120 in the cloud storage system 100 respectively, and select, based on the distance between the sensing device 130 and each of the resource pools 120 respectively, a resource pool 120 as the first target resource pool for logical binding with the sensing device 130. The physical location may include a specific administrative area (e.g., a province, a city, a district, etc. ) , latitude and longitude coordinates, etc., without specific limitation herein.
[0090] In some embodiments, the storage management platform 113 obtains, based on the physical location of the first sensing device, a distance between the first sensing device and each of the resource pools respectively; and determines, based on the distance between the first sensing device and the each of the resource pools respectively, a first target resource pool from the resource pools. For example, after receiving the intra-pool access request, the service management platform 112 forwards the intra-pool access request to the storage management platform 113, and the storage management platform 113 determines the first target resource pool of the first sensing device.
[0091] As previously described, the topology configuration file may define the resource pools 120 and data nodes 121 contained in each resource pool 120. That is, the topology configuration file may reflect topological relationships between the resource pools in the cloud storage system 100.
[0092] Please refer in conjunction to FIG. 2, which is a schematic diagram illustrating a topology configuration file according to some embodiments of this disclosure. In FIG. 2, pool_id is used to identify the ID of the resource pool, pool_type is used to identify the pool category of the resource pool, pool_name is used to identify the name of the resource pool, datanode _list is used to reflect the data nodes contained in the resource pool. In some embodiments, the pool category reflects the type of data that the resource pool is responsible for accessing (e.g., video, images, audio, structured data, etc. ) . In some embodiments, the pool category reflects other metrics, which are not limited herein. In some embodiments, the data nodes contained in the resource pool are represented using the IP addresses of the data nodes, i.e., the datanode_list specifically contains the IP addresses of the individual data nodes in the resource pool.
[0093] As shown in FIG. 2, as an example, the cloud storage system 100 defined by the topology configuration file contains two resource pools 120, one of which has an ID of 148618787703220001, a name of BeijingPool, and a pool category of 1 (e.g., the data type is video, image, and audio) , and which contains two data nodes, the two data nodes having IP addresses of 192.168.2.10 and 192.168.2.11, respectively. Another resource pool 120 has an ID of 148618787703220002, a name of ShanghaiPool, a pool category of 2 (e.g., the data type is structured data) , which contains two data nodes, and the IP addresses of the two data nodes are 192.168.2.20 and 192.168.2.21, respectively.
[0094] It should be noted that FIG. 2 is only a possible example of the topology configuration file, and should not be construed as limiting the specific forms in which the topology configuration file reflects the resource pools and their contained data nodes. No further examples of the topology configuration file will be enumerated here.
[0095] As described above, the topology configuration file may contain configuration information such as names, IDs, etc., of individual resource pools. Based on this, in some embodiments, the management platform 110 (e.g., the service management platform 112 and the storage management platform 113) determines, based on the physical location of the sensing device 130 and the configuration information for each of the resource pools, the distance between the sensing device 130 and each of the resource pools 120 respectively. Merely as an example, in the case where the sensing device 130 is physically located at "Intersection C, District B, City A" , each of the resource pools defined in the topology configuration file includes: a resource pool with the pool_name "Acity" , a resource pool with the pool_name "D city" , and a resource pool with the pool_name "E city" , then the service management platform 112 determines the distance between the sensing device 130 and each of the resource pools by determining the distance between city A and city D and the distance between city A and city E, respectively. Since the physical locations of the sensing device 130 and the resource pool with the pool_name "City A" are City A, it is possible to determine that the distance between the sensing device 130 and the resource pool with the pool_name "Acity" is 0, or that the sensing device 130 is closest to the resource pool with the pool_name "Acity" . The above examples are only one possible example of determining the distance, and other possible approaches are not limited herein.
[0096] In some embodiments, the management platform 110 selects the resource pool 120 with the shortest distance as the first target resource pool based on the distance between the sensing device 130 and each of the resource pools 120. For example, in conjunction with the above example, the service management platform 112 identifies the resource pool with the pool_name "City A" as the first target resource pool.
[0097] By measuring the distance between a sensing device and each resource pool respectively, and selecting and logically binding one resource pool to the sensing device, this approach enables proximity-based access to resource pools, thereby providing the proximity load and proximity distribution of the output data of the sensing device.
[0098] In some embodiments, when the load of a resource pool closest to the first sensing device reaches a preset number or is relatively high compared to other resource pools (e.g., the load of the current resource pool is 10%higher than the resource pool with the largest load among the other resource pools) , the management platform 110, determines, based on the principle of proximity allocation and / or the principle of load balancing, the first target resource pool from the resource pools. For example, the management platform 110 ranks the distances between the first sensing device and each resource pool in order of proximity, and when the load of the first-ranked resource pool reaches the preset number, selects the second-ranked resource pool as the first target resource pool; if the load of the second-ranked resource pool also reaches the preset number, the third-ranked resource pool is selected as the first target resource pool; based on this, the resource pool among the resource pools whose load is less than the preset number and who is closer to the first sensing device is preferentially selected as the first target resource pool. As another example, when the load of the resource pool closest to the first sensing device reaches a preset number or is relatively high compared to other resource pools, the management platform 110 directly, based on the load balancing principle, selects the resource pool with the lowest load among the resource pools as the first target resource pool. In some embodiments, the preset number is related to the number of sensing devices accessed by the resource pool 120, and / or the resources (e.g., computational resources, network bandwidth, memory occupancy, etc. ) of the servers corresponding to the data nodes 121 contained in the resource pool 120.
[0099] In some embodiments, after the first sensing device is successfully logically bound to the first target resource pool, attribute information of the first sensing device is logically bound to the attribute information of the first target resource pool. Merely as an example, after the first sensing device has successfully accessed the first target resource pool, a device identifier used by the first sensing device to uniquely identify itself is logically bound to a pool_id of the first target resource pool.
[0100] In some embodiments, the management platform 110 is further configured to determine a first target node of the first sensing device based on the load of each of the data nodes within the first target resource pool. The first sensing device is configured to be in a communication connection with the first target node.
[0101] Merely as an example, the storage management platform 113 determines, based on the load of each data node 121 within the first target resource pool, that the data node with the smallest current load is the first target node of the first sensing device.
[0102] In some embodiments, after the first sensing device is logically bound to the first target resource pool, the management platform 110 determines, based on the topology configuration file, data nodes within the first target resource pool, and obtains a load profile of the data nodes within the first target resource pool to determine the first target node of the first sensing device based on load balancing purposes. Merely as an example, the storage management platform 113 queries, based on the configuration information of individual resource pools in the topology configuration file, the data nodes within the first target resource pool logically bound to the first sensing device. Taking three data nodes 121 as an example, designated as "node 01" , "node 02" , and "node 03" for differentiation, if the loads of "node 02" and "node 03" are close to or the same, and both are higher than the load of "node 01" , then "node 01" is selected as the first target node of the first sensing device. The above example is only a possible example in the process of practical application, and does not therefore limit other possible cases, and no examples will be given here.
[0103] In some embodiments, an intra-pool access sub-node of the sub-nodes of the first target node is configured to establish the communication connection between the first sensing device and the first target node in response to receiving the intra-pool access request.
[0104] For example, in conjunction with FIG. 1B, the data node 121 includes an intra-pool access sub-node for providing an intra-pool access service, and in a streaming media scenario, after determining the first target node of the first sensing device, the storage management platform 113 sends the streaming media access request of the first sensing device to the first target node, and the intra-pool access sub-node in the first target node establishes a communication connection with the first sensing device based on a communication protocol in response to receiving the streaming media access request. After the first sensing device is successfully accessed by the first target resource pool, attribute information (e.g., a device identifier) of the first sensing device is logically bound to the first target resource pool and to the first target node and stored in the first target node.
[0105] In some embodiments, when the service request is the intra-pool storage request, an intra-pool storage sub-node in the first target node that is in a communication connection with the first sensing device is configured to obtain output data of the first sensing device and store the output data by invoking a target distributed storage sub-node in the first target resource pool where the first target node belongs to. At this time, the first target node is still essentially a data node in the first target resource pool logically bound to the first sensing device, and is a data node that prioritizes the transparent transmission of the output data of the first sensing device, so that in the present embodiment, the first target node is also referred to as a first transmission node.
[0106] After the first sensing device is logically bound to the first target resource pool, the management platform 110 selects the data node that prioritizes the transparent transmission of the output data as the first transmission node for the first sensing device based on load balancing, which is able to achieve load balancing within each resource pool 120 as much as possible through the same management platform 110.
[0107] The data node where the target distributed storage sub-node is located may be the first target node (the first transmission node) or another data node in the first target resource pool that is different from the first target node. For example, an intra-pool storage sub-node in the first transmission node obtains the output data of the first sensing device and stores the output data by invoking the distributed storage sub-node in the first transmission node. As another example, the intra-pool storage sub-node in the first transmission node stores the output data by invoking a distributed storage sub-node of other data nodes in the first target resource pool.
[0108] Merely as an example, the intra-pool storage sub-node of the first target node that is in a communication connection with the first sensing device generates a distributed storage request and sends the distributed storage request to the distributed storage sub-node within the first target node to invoke a cloud storage interface in the distributed storage sub-node for data writing (i.e., a distributed cloud storage) . The distributed storage sub-node may send the distributed storage request to the management platform 110, the management platform 110 may determine a target data node based on the cloud storage interface of the distributed storage request, and allocate at least one storage space within the first target resource pool to enable the cloud storage interface to store each of the segments to be written obtained by slicing based on the data to be written to each of the at least one storage space. For more descriptions regarding the distributed cloud storage, please refer to the description below.
[0109] By invoking the target distributed storage sub-node, the output data of the first sensing device may be stored within the first target node or within two or more data nodes of the first target resource pool. For example, when the load of the first target node satisfies a preset condition, the output data of the first sensing device is stored within the first target node; when the load of the first target node does not satisfy the preset condition, the output data of the first sensing device is stored within two or more data nodes of the first target resource pool.
[0110] In some embodiments, the management platform 110 is further configured to: in response to determining that the load of the first target node (the first transmission node) does not satisfy the preset condition, determine, from the data nodes within the first target resource pool, a new first target node (also referred to as the second transmission node) based on the load of each of the data nodes in the first target resource pool to obtain the output data from the first sensing device via the new first target node. That is, at this time, the new first target node is configured to act as a data node for prioritizing the transparent transmission of the output data of the first sensing device, and is therefore also referred to as the second transmission node. It should be noted that the second transmission node and the first transmission node are both located in the first target resource pool, but the second transmission node is a data node different from the first transmission node.
[0111] In some embodiments, the management platform 110 (e.g., the storage management platform 113) determines, from the data nodes within the first target resource pool, the new first target node based on the topology configuration file and the load of each of the data nodes within the first target resource pool. The new first target node is determined in a manner similar to the manner in which the first target node is determined in the preceding section, see the preceding description for more, and will not be repeated here.
[0112] For example, the intra-pool storage sub-node of the first target node that is in a communication connection with the first sensing device generates a distributed storage request and sends the distributed storage request to the distributed storage sub-node within the first target node to invoke an SDK interface in the distributed storage sub-node within the first target node for data writing. The distributed storage sub-node may send the distributed storage request to the management platform 110, and the management platform 110 may determine, based on the cloud storage interface of the distributed storage request, that the load of the first target node where the distributed storage sub-node is located does not satisfy the preset condition, and determine, based on the loads of other data nodes within the first target resource pool, the new first target node (i.e., the second transmission node) . Further, the management platform 110 may send the distributed storage request to an intra-pool storage sub-node of the second transmission node, and the intra-pool storage sub-node of the second transmission node performs the data writing by invoking the distributed storage sub-node within the second transmission node to store the output data of the first sensing device.
[0113] The preset condition may include an imbalance in a current load of the first transmission node, relative to the loads of other data nodes 121 within the first target resource pool. In some embodiments, the preset condition includes that a ratio of the current load of the first transmission node to the loads of the other data nodes in the first target resource pool where the first transmission node is located is greater than a preset threshold. The preset threshold may be set according to the actual application needs, such as 10%, 20%, 30%, etc., and is not specifically limited herein.
[0114] Merely as an example, in the case where three data nodes 121 are included in the first target resource pool which is logically bound by the first sensing device, in the case when the "node 01" serves as the first transmission node to transparently transmit the output data of the first sensing device and the ratio of the load of the "node 01" to the loads of the "node 02" and "node 03" is greater than the preset threshold, the storage management platform 113 selects a data node with a smaller load among the "node 02" and "node 03" as the second transmission node, so as to provide load balancing as much as possible within the first target resource pool. The above example is only one possible example in practice, and other scenarios can be cited by analogy, and will not be cited here.
[0115] When the current load of the first target node does not satisfy the preset condition, the management platform 110 selects one of the other data nodes as the new first target node based on the loads of the data nodes 121 in the first target resource pool that are logically bound by the first sensing device, which can ensure as much as possible load balancing within the pool during the intra-pool storage process of each resource pool.
[0116] In some embodiments, when the service request is the distributed storage request (also referred to as a data write request) , the management platform 110 is configured to: obtain a distributed storage request from the distributed storage sub-node in a data node; and determine, based on the distributed storage request, that a data node where a cloud storage interface of the distributed storage request is located is a third target node, and allocate at least one storage space within a third target resource pool where the third target node is located. The cloud storage interface may be used to store individual segments to be written obtained by slicing data to be written to each of the at least one storage space.
[0117] In some embodiments, the distributed storage request includes identification information of the distributed storage sub-node. The identification information of the distributed storage sub-node may be a node address (e.g., an IP address) of a data node where the distributed storage sub-node is located.
[0118] The cloud storage interface is a software development kit (SDK) interface capable of being exposed externally for a cloud storage client (e.g., the sensing device 130) . The SDK interface is a set of tools, libraries, and documentation that can be utilized by a developer to facilitate specific functionality extensions and integrations within a software application. Specifically, the SDK interface is a set of programming interfaces used by the developer to interact with the distributed storage sub-node, which allows the developer to implement file uploading (i.e., uploading a file from a local location to the cloud storage system) , file downloading (i.e., downloading a file from the cloud storage system to a local location) , file management (e.g., creating, reading, updating, deleting, etc. ) , sharing, and other functions in the application. In some embodiments, the SDK interface also has features such as permission control (i.e., managing access to files, such as setting shared links, user permissions, etc. ) , version control (i.e., supporting the management of file versions, allowing users to use different versions of the file) , and event handling (i.e., supporting notification of changes to the file and event callbacks) . In some embodiments, the SDK interface may be defined with, for example, protocols related to data writing, protocols related to data querying, and the like, without limiting the specifics of the SDK interface herein.
[0119] The storage space may be at least a portion of an area of a storage device of the data node 121.
[0120] In conjunction with FIG. 1B, the data node 121 may further include an intra-pool storage sub-node for providing an intra-pool storage service (e.g., a streaming storage service) , and a distributed storage sub-node for providing a distributed storage service. As described in the preceding section, the intra-pool storage sub-node may generate a distributed storage request and send the distributed storage request to the distributed storage sub-node to invoke a cloud storage interface in the distributed storage sub-node for data writing. In some embodiments, when the intra-pool storage sub-node (e.g., the first target node or the intra-pool storage sub-node in the second target node) invokes the cloud storage interface for data writing (i.e., initiates the distributed storage request) , the storage management platform 113 obtains the distributed storage request from the distributed storage sub-node, determines, based on the IP address of the cloud storage interface, a data node 121 where the cloud storage interface that initiated the distributed storage request is located, and determines the data node as the third target node (not shown) . Further, the storage management platform 113 may apply for and allocate at least one storage space within the third target resource pool where the third target node is located, and the cloud storage client may store the individual segments to be written, obtained by slicing the data to be written, to each of the at least one storage space via the cloud storage interface.
[0121] In this embodiment, data writing within the resource pool 120 is achieved by calling the cloud storage interface through the intra-pool storage sub-node, which can achieve self-completion within the resource pool, which can in turn achieve persistent storage of the data generated by the sensing device 130.
[0122] In conjunction with the above, the storage management platform 113 may include the Paas management service and the cloud storage management service. In some embodiments, the cloud storage management service loads and recognizes the topology configuration file, and performs a unified load and space allocation to the data nodes 121 within each of the resource pools 120 based on the topology configuration file.
[0123] In some embodiments, when the intra-pool storage sub-node invokes the cloud storage interface to initiate the distributed storage request, the cloud storage management service determines, based on the IP address of the cloud storage interface of the distributed storage request, the third target node where the distributed storage sub-node is located, and prioritizes, based on the topology configuration file, the allocation of storage spaces within the third target resource pool where the third target node is located, so that the cloud storage client slices the data to be written and stores to the third target node via the cloud storage interface, so as to realize in-pool sliced storage of service data and storage traffic without crossing the pool, helping minimize the bandwidth pressure on the core backbone network.
[0124] In some embodiments, the cloud storage management service determines a third target node where the distributed storage request is located based on the IP address of the cloud storage interface of the distributed storage request, and sends the distributed storage request to an intra-pool storage sub-node of the third target node. The intra-pool storage sub-node of the third target node may slice the data to be written and store the sliced data in storage spaces of the third target node via the cloud storage interface.
[0125] In some embodiments, when the intra-pool storage sub-node sends the distributed storage request to the distributed storage sub-node, in response to receiving the distributed storage request, the distributed storage sub-node selects one or more data nodes as a data storage node from the third target resource pool where the intra-pool storage sub-node is located, to ensure load balancing within the pool, and the SDK interface of the distributed storage sub-node slices the data to be written and stores it separately to the determined data storage node.
[0126] In some embodiments, when the service request is the distributed storage request, the management platform 113 is configured to obtain the distributed storage request from the distributed storage sub-node in a data node, the distributed storage request including an identifier of the distributed storage sub-node; determine, based on the distributed storage request, a targe data node where the cloud storage interface is located; and allocate at least one storage space in a third target resource pool where the third target node is located, wherein the cloud storage interface is configured to store each write segment obtained by slicing data to be stored into a respective storage space.
[0127] In some embodiments, when the service request is the intra-pool forwarding request (e.g., the video playback request, the image retrieval request) , the intra-pool forwarding request may include identification information of target data to be forwarded. The management platform 110 (e.g., the service management platform 112) obtains the intra-pool forwarding request, and forwards the intra-pool forwarding request to a second target resource pool based on the identification information in the intra-pool forwarding request. In response to the intra-pool forwarding request, the second target resource pool sends target data corresponding to the intra-pool forwarding request within the second target resource pool to at least one target device. The second target resource pool is a resource pool logically bound by a second sensing device, and the second sensing device is a sensing device that acquires the target data.
[0128] The target data is data to be forwarded or read, and may include at least one of a video, an audio, an image, etc.
[0129] The identification information of the target data may include a device identifier of the sensing device (e.g., the second sensing device) acquiring the target data, and acquisition time / acquisition time period. Merely as an example, the device identifier includes channel information, for example, in the format of a "city-district-road" . For example, the device identifier of the sensing device that acquired the requested image is "City A, District B, Intersection of Road C and Road D" . The above example is only one possible example of the device identifier of the sensing device, and does not thereby limit the specific content of the device identifier. Additionally, the device identifier may be represented by a number as shown in the preceding pool_name, and the specific encoding of the device identifier is not qualified herein.
[0130] In some embodiments, the identification information for the target data further includes a pool identifier, the pool identifier being used to uniquely characterize a resource pool (e.g., the second target resource pool) that is logically bound by the sensing device that acquires the target data.
[0131] The target device may be the same as the second sensing device or a different device. In some embodiments, the target device includes a sensing device (e.g., a first sensing device, a second sensing device, a third sensing device, etc. ) , a cloud storage system (e.g., the cloud storage system 100) , the management platform 110, or other systems or devices, without specific limitation herein.
[0132] In some embodiments, the management platform 110 (e.g., the service management platform 112) determines, based on the identification information of the target data (e.g., the pool identifier in the identification information) , the second target resource pool logically bound by the second sensing device, and forwards the intra-pool forwarding request to an intra-pool forwarding sub-node of each data node within the second target resource pool. In response to receiving the intra-pool forwarding request, the intra-pool forwarding sub-node of each data node may determine, based on the identification information, whether or not the output data of the sensing device corresponding to the device identifier is stored in the data node. If an intra-pool forwarding sub-node determines that the output data is stored in the data node, the intra-pool forwarding sub-node may further determine whether the output data includes the target data acquired in the acquisition time in the identification information, and if the target data is included in the output data, read the target data and send the target data to the at least one target device.
[0133] In some embodiments, when the service request is the intra-pool forwarding request, the management platform 110 determines, based on a mapping configuration file, a second target resource pool logically bound by the second sensing device, and / or a second target node that is in a communication connection with the second sensing device. The mapping configuration file defines a mapping relationship of a logical binding between a sensing device 130 and a resource pool 120, the mapping relationship may be contained in a resource pool logically bound by the sensing device or a a data node to which the sensing device is in a communication connection within the resource pool. For example, the mapping configuration file is stored in the storage management platform 113, and the storage management platform 113 (e.g., by the PaaS management service in the storage management platform 113) performs a query in the mapping configuration file based on the device identifier in the identification information, thereby determining a second target resource pool that is logically bound by the second sensing device, and / or a second target node that is in a communication connection with the second sensing device within the second target resource pool.
[0134] In conjunction with the foregoing, the storage management platform 113 may include the PaaS management service and the cloud storage management service. In some embodiments, the PaaS management service determines, based on the identification information and the mapping configuration file, the second target resource pool, and forwards the intra-pool forwarding request to the second target resource pool. The second target resource pool, in response to the intra-pool forwarding request, sends the target data corresponding to the intra-pool forwarding request within the second target resource pool to the at least one target device (e.g., the storage management platform 113) .
[0135] Finding the second target resource pool by the mapping configuration file can improve the accuracy of determining the second target resource pool.
[0136] In some embodiments, when the service request is the intra-pool forwarding request, the management platform 110 determines, based on the identification information, the second target node that is in a communication connection with the second sensing device. The second target node is located within a second target resource pool. The management platform 110 forwards the intra-pool forwarding request to an intra-pool forwarding sub-node in the second target node. The intra-pool forwarding sub-node is configured to read the target data from the second target resource pool in response to receiving the intra-pool forwarding request and send the target data to the at least one target device.
[0137] In some embodiments, when the intra-pool forwarding request is a video playback request, the target data includes a target video. The video playback request includes first identification information (also referred to as a first request identifier) of the target video. The management platform 110 (e.g., the service management platform 112) forwards, based on the first request identifier in the video playback request, the video playback request to the second target resource pool. The second target resource pool reads the target video in response to receiving the video playback request and sending the target video to the at least one target device. For example, the first request identifier includes a first device identifier of a sensing device (e.g., the second sensing device) that acquired the target video requested playback, a first pool identifier (e.g., the aforementioned pool_id) of a resource pool (e.g., the second target resource pool) that is logically bound by the sensing device, and an acquisition time period during which the target video was acquired (e.g., the time period between D o'clock to E o'clock in the morning of B month C of year A) .
[0138] Merely as an example, in conjunction with FIG. 1B, the data node 121 includes an intra-pool forwarding sub-node for providing an intra-pool forwarding service (e.g., a streaming media forwarding service) , and the service management platform 112 may forward, based on the first pool identifier in the first request identifier, the video playback request to an intra-pool forwarding sub-node of each data node within the second target resource pool. The intra-pool forwarding sub-node of each data node may determine, based on a first device identifier in the first request identifier, whether a data node stores output data of the sensing device 130 corresponding to the first device identifier. If the data node stores output data of the sensing device 130 corresponding to the first device identifier, the intra-pool forwarding sub-node of the data node in which the output data is stored continues to determine whether there exists a target video located at the acquisition time period of the target video in the output data, and if so, the intra-pool forwarding sub-node reads the target video and sends the target video to the at least one target device.
[0139] When the intra-pool forwarding request is the video playback request, the management platform 110 forwards the video playback request to an intra-pool forwarding sub-node of each of data nodes within the second target resource pool based on the first identification information. Each intra-pool forwarding sub-node is configured to query and / or return the target video corresponding to the video playback request, enabling intra-pool forwarding of video data through the intra-pool forwarding sub-node.
[0140] In some embodiments, when an intra-pool forwarding request is a video playback request, the management platform 110 (e.g., the service management platform 112) identifies, based on the first identification information of the target video, the second target node and forwards the video playback request to the intra-pool forwarding sub-node of the second target node. In response to receiving the video playback request, the intra-pool forwarding sub-node reads the target video from the second target resource pool and sends the target video to the at least one target device. For example, in conjunction with the above example, the intra-pool forwarding sub-node reads a target video located in the acquisition time period between D o'clock to E o'clock in the morning of B month C of year A based on the first request identifier by invoking a distributed storage sub-node that belongs to the data node to which the intra-pool forwarding sub-node belongs or a distributed storage sub-node that belongs to another date node, and sends the target video to the service management platform 112.
[0141] In some embodiments, when the intra-pool forwarding request is an image retrieval request, the target data includes a target image. The image retrieval request includes second identification information (also referred to as a second request identifier) of the target image. The management platform (e.g., the service management platform 112) is configured to forward, based on the second request identifier in the image retrieval request, the image retrieval request to the second target resource pool, the second target resource pool reading the target image in response to receiving the image retrieval request and sending the target image to the at least one target device.
[0142] In some embodiments, when the intra-pool forwarding request is an image retrieval request, the management platform 110 determines the second target node based on the second identification information of the target image and forwards the image retrieval request to the intra-pool forwarding sub-node of the second target node. The intra-pool forwarding sub-node reads the target image from the second target resource pool in response to receiving the image retrieval request and sending the target image to the at least one target device.
[0143] Merely as an example, in conjunction with FIG. 1B, the data node 121 includes an intra-pool forwarding sub-node for providing an intra-pool forwarding service (e.g., an image gateway service) , and the service management platform 112 determines a second target resource pool based on the second pool identifier in the second request identifier and forward the image retrieval request to an intra-pool forwarding sub-node of a second target node in the second target resource pool. The intra-pool forwarding sub-node of the second target node may download, based on the second request identifier, the target image corresponding to the acquisition time within the second target resource pool, by invoking a distributed storage sub-node that belongs to the data node to which the intra-pool forwarding sub-node belongs or a distributed storage sub-node that belongs to another data node, and send the target image to the service management platform 112. For example, the intra-pool forwarding sub-node of the second target node returns a uniform resource locator (URL) of the target image to the service management platform 112 upon successfully downloading the target image within the second target resource pool.
[0144] By forwarding the image retrieval request to the intra-pool forwarding sub-node of the second target node, and the intra-pool forwarding sub-node downloading the target image and returning the target image to the management platform, intra-pool forwarding of image data can be provided.
[0145] In the above embodiments, by using a set of management platforms (as described in the previous section the management platform 110 includes the service management platform 112 and the storage management platform 113, and thus is also referred to as a set of management platforms) to provide the management and load balancing of the streaming services (e.g., the intra-pool access service, the intra-pool storage service, and the intra-pool forwarding service, etc. ) and the storage services (e.g. storage space allocation, data persistent storage to storage media) of each resource pool, it effectively avoids the complexity of management in multi-domain scenarios, traffic cross-domain, and other problems. Based on the above cloud storage system 100, it is possible to make the entire streaming media service whole process, from device access, storage service calling cloud storage interface to storage, and then to data forwarding (for example, video data, image data and other data reading) are all carried by the data node 121 within the resource pool 120, that is, all completed within the resource pool, and there is no traffic interaction between the pools, which is able to significantly reduce the bandwidth pressure of the streaming media service on core backbone networks.
[0146] In some embodiments, the management platform 110 (e.g., the storage management platform 113) is further configured to determine a fourth target node from a resource pool where a failed data node is involved in response to a failure of any of the data nodes. The fourth target node is a normal data node (i.e., a non-failed data node) within the resource pool where the failed data node is located, and configured to perform the services (e.g., access, forwarding, storage, etc. ) of the failed data node. That is, when a data node within the resource pool 120 fails, the original logical binding relationship between the sensing device 130 and the resource pool 120 is not altered, and other normal data nodes 121 within the resource pool 120 share the service pressure of the failed data node, thereby realizing intra-pool disaster recovery.
[0147] Merely as an example, taking the resource pool 120 containing three data nodes 121 as an example, after the storage management platform 113 discovers that "node 01" in the resource pool 120 has failed, the storage management platform 113 selects a fourth target node among the normal data nodes (e.g., "node 02" and "node 03" ) of the resource pool 120 where "node 01" is located, in order to make a sensing device 130 that transparently transmits output data to "node 01" to transparently transmit the output data to the fourth target node.
[0148] In some embodiments, the storage management platform 113 determines, based on the topology configuration file, a resource pool where the failed data node is located. Referring in conjunction with FIG. 2, exemplarily, the storage management platform 113 first obtains the IP address of the failed data node, such as 192.168.2.10, then determines, based on the topology configuration file as shown in FIG. 2, that the resource pool in which the failed data node is located is a resource pool with a "pool_name" of "BeijingPool" . Of course, the above example is only one possible example of the actual application process, and other situations may be similar, so no example will be given here.
[0149] In some embodiments, the management platform 110 determines the fourth target node based on the loads of the nodes (including data nodes and / or sub-nodes) in the resource pool where the failed data node is located. For example, the management platform 110 determines, based on the computational resources, the network bandwidth, or the memory occupancy rate of each data node in the resource pool where the failed data node is located, with the purpose of load balancing, the data node with the least load as the fourth target node.
[0150] In some embodiments, a specific count of the fourth target node may be greater than or equal to 1. In some embodiments, the count of the fourth target node is determined based on the count of sensing devices logically bound to the failed data node, and / or the load of the data node.
[0151] Merely as an example, still taking the resource pool 120 containing three data nodes 121 as an example, if there are two sensing devices that prioritize the transparent transmission of output data to the failed "node 01" : sensing device 01 and sensing device 02, that is, there are two sensing devices that establish a communication connection with the "node 01" , and loads of the currently normal "node 02" and "node 03" in the resource pool are close, then "node 02" can be selected to perform the services (e.g., access, storage, forwarding, etc. ) related to sensing device 01, and "node 03" can be selected to perform the services related to sensing device 02. Or, if there are two sensing devices that have established a communication connection with "node 01" and the load of "node 02" is significantly higher than the load of "node 03" , "node 02" and "node 03" being currently normal in the resource pool, then "node 03" can be selected to perform the services related to the two sensing devices. Of course, the above examples are only a few possible examples of the intra-pool disaster recovery in the actual application process, other cases can be categorized by analogy, and no more examples will be given here.
[0152] In some embodiments, the management platform 110 (e.g., the storage management platform 113) is further configured to determine, in response to a failure of at least one sub-node in at least one data node of the data nodes, the fourth target node from a resource pool in which the at least one data node is located, at least one sub-node in the fourth target node being configured to perform the services of the at least one failed sub-node in the at least one data node.
[0153] In some embodiments, a failure detection way of a node (e.g., the data node and / or the sub-node) includes but is not limited to, at least one of the following: detecting a node state in real-time by a monitoring device, determining a node state by analyzing node logs, determining a node state by the resource usage of the node (e.g., the sub-node is unable to function normally because it is running out of CPU, memory, disk, etc. ) , determine a node state by checking the service status of the node (e.g., whether the service corresponding to the sub-node is running normally) . For example, monitoring devices include Prometheus + Grafana, for monitoring the resource usage of nodes, and Zabbix and Nagios, for detecting whether a node is offline or if the load exceeds a preset value.
[0154] In some embodiments, an intra-pool move-back service is triggered when the failure is eliminated (anomaly recovery) . The intra-pool move-back service refers to relocating the output data of a sensing device that originally established a communication connection with an original data node back to the original data node. For example, in conjunction with the preceding example, when the failure of the "node 01" is eliminated or the "node 01 is returned to normal, the output data of the sensing device 130 that was previously communicatively connected to the "node 01" may be migrated back to the "node 01" .
[0155] Because the intra-pool move-back service is only limited to the interior of the resource pool, there is no cross-pool transmission of service traffic, which helps to minimize the bandwidth pressure on the core backbone networks.
[0156] It should be noted that different from the above failure handling in the embodiment, at the level of the streaming media service, although the system exists in a huge number of sub-nodes, all the data nodes and sub-nodes belong to the same PaaS domain, and its failure tolerance is by default the global disaster recovery within the domain. Therefore, when a PaaS sub-node failure occurs, the system internally randomly selects a PaaS sub-node within the domain based on the load algorithm to provide services (e.g., access, storage, forwarding, etc. ) for the sensing device of the failed node. However, this approach can lead to cross-pool transmission of the service network after a failure, increasing the bandwidth pressure on the streaming core network. With the failure handling way described in the above embodiment, when a PaaS sub-node failure occurs within the resource pool (for example, a sub-node failure in a data node) , the PaaS sub-node is preferentially selected within the resource pool to provide access to the sensing device and the data storage, forwarding, and other services, and the relationship between the channel ID and the resource pool Id does not change during the period, and other data nodes within the resource pool share the service pressure of the data node that fails. No cross-pool disaster recovery is carried out, and when the failure is eliminated, then the intra-pool relocation of services is triggered, and no cross-pool transmission of service traffic occurs in abnormal scenarios, so that the high utilization rate of the core backbone networks can be ensured.
[0157] In some embodiments, in response to a failure of any of the data nodes, the management platform 110 (e.g., the storage management platform 113) is further configured to determine a fifth target node based on the load of the data nodes within the resource pool in which the failed data node is located. The fifth target node is a normal data node for performing a data recovery task for the data node that has failed.
[0158] In some embodiments, in response to a failure of at least one sub-node in at least one data node of the data nodes, the management platform 110 determines, based on the loads of the data nodes within the resource pool where the at least one data node located, the fifth target node from the data nodes, and performs a data recovery task by the fifth target node for the at least one failed sub-node of the at least one data node.
[0159] Regarding the failure detection of a node, and the determination of the resource pool where a failed data node is located, please refer to the description above, and it is not repeated here.
[0160] Merely as an example, when any of the data nodes 121 fails (e.g., at least one sub-node in the data node fails) , the storage management platform 113 queries the resource pool where the failed data node is located based on the IP address of the failed data node and the topology configuration file as shown in FIG. 2, and further determines a target recovery node (i.e., the fifth target node) based on loads of the normal data nodes within the resource pool, such as determining the data node with the lowest load among the normal data nodes as the fifth target node. The target recovery node may perform the data recovery task using a recovery algorithm such as Erasure Coding (EC) , etc., to enable anomalous data recovery.
[0161] After a node failure occurs in a multi-domain mode, the use of global recovery will result in the cross-domain transmission of the service traffic, and the bandwidth pressure on the core backbone networks will greatly increase. In the above embodiments described herein, when a data node or a sub-node fails, by prioritizing, based on loads of the data nodes within the resource pool where the failed data node is located, to determine the fifth target node within the resource pool for performing a data recovery task, it is possible to provide load balancing as much as possible on the premise of providing data recovery within the resource pool, and since recovery within the pool does not need the across-pool transmission, it is thus possible to reduce the bandwidth pressure of the core backbone networks as much as possible.
[0162] In the above embodiment, the management platform 110 is in a communication connection with the resource pools 120 in the cloud storage system 100, the resource pools 120 including the data nodes 121, the resource pools 120 being configured to logically bind with the sensing devices 130, and the output data of each of the sensing devices 130 is stored in a resource pool that is logically bound by the sensing device. No traffic interaction occurs between different resource pools 120. The management platform 110 includes the storage management platform 113 communicatively connected to the resource pools 120 for load balancing and scheduling management based on the topology configuration file within each resource pool 120 respectively and the service management platform 112 communicatively connected with each resource pool 120 for receiving and responding to service requests. Firstly, the distributed deployment of the resource pools 120 reduces the construction difficulty and costs of the cloud storage system 100 compared to a centralized architecture. Secondly, by localizing storage, access, and forwarding services within each resource pool 120 and eliminating across-pool traffic interaction, the system minimizes data flow (e.g., storage and service traffic) through the core backbone networks, thereby alleviating the load pressure of the core backbone networks. Thirdly, compared to the multi-domain mode, unified management across all resource pools 120 is achieved via a single management platform 110 (i.e., including the service management platform 112 and the storage management platform 113) . Therefore, the above-described cloud storage system 100 is capable of minimizing the load pressure on the core backbone networks and enhancing the convenience of system management under the premise of reducing the difficulty of the system construction and the construction cost.
[0163] FIG. 3 is a schematic diagram illustrating exemplary hardware and / or software components of a computing device according to some embodiments of the present disclosure. As illustrated in FIG. 3, the computing device 300 includes a processor 310, a storage 320, an input / output (I / O) 330, and a communication port 340.
[0164] The processor 310 may execute computer instructions (program code) and perform functions of the management platform 110 in accordance with the techniques described herein. The computer instructions may include routines, programs, objects, components, signals, data structures, procedures, modules, and functions, which perform particular functions described herein. In some embodiments, the processor 310 may include a microcontroller, a microprocessor, a reduced instruction set computer (RISC) , an application-specific integrated circuits (ASICs) , an application-specific instruction-set processor (ASIP) , a central processing unit (CPU) , a graphics processing unit (GPU) , a physics processing unit (PPU) , a microcontroller unit, a digital signal processor (DSP) , a field programmable gate array (FPGA) , an advanced RISC machine (ARM) , a programmable logic device (PLD) , any circuit or processor capable of executing one or more functions, or the like, or any combinations thereof.
[0165] Merely for illustration purposes, only one processor is described in the computing device 300. However, it should be noted that the computing device 300 in the present disclosure may also include a plurality of processors, thus operations and / or method steps that are performed by one processor as described in the present disclosure may also be jointly or respectively performed by a plurality of processors. For example, if in the present disclosure, the processor of the computing device 300 executes both step A and step B, it should be understood that step A and step B may also be performed by two different processors jointly or respectively in the computing device 300 (e.g., a first processor executes step A and a second processor executes step B, or the first and second processors jointly execute steps A and B) .
[0166] The storage 320 may store data / information obtained from the sensing devices 130, or any other component of the cloud storage system 100 (e.g., the topology configuration file, the mapping configuration file, etc. ) . In some embodiments, the storage 320 may include a mass storage device, a removable storage device, a volatile read-and-write memory, a read-only memory (ROM) , or the like, or any combination thereof. For example, the mass storage device may include a magnetic disk, an optical disk, a solid-state drive, etc. The removable storage device may include a flash drive, a floppy disk, an optical disk, a memory card, a zip disk, a magnetic tape, etc. The volatile read-and-write memory may include a random access memory (RAM) . The RAM may include dynamic RAM (DRAM) , a double date rate synchronous dynamic RAM (DDR SDRAM) , static RAM (SRAM) , thyristor RAM (T-RAM) , and a zero-capacitor RAM (Z-RAM) , etc. The ROM may include a mask ROM (MROM) , a programmable ROM (PROM) , an erasable programmable ROM (EPROM) , an electrically erasable programmable ROM (EEPROM) , a compact disk ROM (CD-ROM) , a digital versatile disk ROM, etc. In some embodiments, the storage 320 may store one or more programs and / or instructions to perform exemplary methods described in the present disclosure. For example, the storage 320 may store a program for the management platform 110 to manage the resource pools 120 and / or the data nodes 121 in each resource pool.
[0167] The I / O 330 may input or output signals, data, or information. In some embodiments, the I / O 330 may enable user interaction with the management platform 110. In some embodiments, the I / O 330 includes an input device and an output device. Exemplary input devices include a keyboard, a mouse, a touch screen, a microphone, or the like, or a combination thereof. Exemplary output devices include a display device, a loudspeaker, a printer, a projector, or the like, or a combination thereof. Exemplary display devices include a liquid crystal display (LCD) , a light-emitting diode (LED) -based display, a flat panel display, a curved screen, a television device, a cathode ray tube (CRT) , or the like, or a combination thereof.
[0168] The communication port 340 may be connected to a network to facilitate data communications. The communication port 340 may establish connections between the management platform 110 and the resource pools 120, or the sensing devices 130. The connection may be a wired connection, a wireless connection, or a combination of both that enables data transmission and reception. The wired connection may include an electrical cable, an optical cable, a telephone wire, or the like, or any combination thereof. The wireless connection may include Bluetooth, Wi-Fi, WiMax, WLAN, ZigBee, mobile network (e.g., 3G, 4G, 5G, etc. ) , or the like, or a combination thereof. In some embodiments, the communication port 340 may be a standardized communication port, such as RS232, RS485, etc. In some embodiments, the communication port 340 may be a specially designed communication port. For example, the communication port 340 may be designed in accordance with the digital imaging and communications in medicine (DICOM) protocol.
[0169] It should be understood that the descriptions of FIG. 3 are only provided for the purpose of illustration and do not constitute a limitation to the present disclosure. For those skilled in the art, various changes and modifications can be made under the guidance of the present disclosure. Features, structures, manners, and other characteristics of embodiments of the present disclosure can be combined in various ways to obtain other and / or alternative embodiments. However, such changes and modifications do not exceed the scope of the present disclosure.
[0170] FIG. 4 is an exemplary flowchart illustrating a cloud storage management method according to some embodiments of the present disclosure.
[0171] In some embodiments, the process 400 may be performed by the cloud storage system 100 (e.g., the management platform 110) or the computing device 300. As shown in FIG. 4, process 400 includes:
[0172] Operation 410, a topology configuration file is obtained, and a communication connection is established between the cloud storage system and each of the resource pools. In some embodiments, step 410 is performed by the management platform 110.
[0173] As previously described, the topology configuration file defines the resource pools 120 included in the cloud storage system 100 and the data nodes 121 included in each of the resource pools 120, and at least one sensing device 130 is configured to be logically bound to a resource pool of the resource pools 120 Output data of the sensing device 130 may be stored in the resource pool. The management platform 110 (e.g., the service management platform 112, the storage management platform 113) may establish a communication connection with each resource pool 120 in the cloud storage system 100. See the preceding embodiment of the cloud storage system 100 for more description, which will not be repeated herein.
[0174] Operation 420, each resource pool and / or the data nodes within each resource pool are managed based on the topology configuration file. In some embodiments, step 420 is performed by the management platform 110.
[0175] In some embodiments, in response to determining that a first sensing device is logically bound to a first target resource pool, the management platform 110 determines a first target node (i.e., a first transmission node) within the first target resource pool based on the topology configuration file and the load of each of the data nodes within the first target resource pool, the first target node being configured to obtain output data from the first sensing device.
[0176] Merely as an example, in response to determining that the first sensing device has been logically bound to a resource pool, the storage management platform 113 determines the resource pool as the first target resource pool, and determines the first target node within the first target resource pool for the purpose of load balancing based on the topology configuration file and the load of each of the data nodes 121 within the first target resource pool, the first target node being configured to prioritize access to the output data of the first sensing device. Specifically, reference may be made to the foregoing description of the cloud storage system 100, which will not be repeated herein.
[0177] In some embodiments, in response to determining that a load of the first target node does not satisfy a preset condition, the management platform 110 (e.g., the storage management platform 113) determines a new first target node (i.e., a second transmission node) from the first target resource pool based on the topology configuration file and the loads of the data nodes within the first target resource pool, to obtain the output data of the first sensing device via the new first target node. For example, the storage management platform 113 determines, based on the topology configuration file, the data nodes contained in the first target resource pool, obtains a load profile of the data nodes, and determines the new first target node from the first target resource pool. Specifically, reference may be made to the foregoing description in the cloud storage system 100, which will not be repeated herein.
[0178] In some embodiments, when the service request is an intra-pool forwarding request for providing an intra-pool forwarding service, the management platform 110 also obtains a mapping configuration file. As previously described, the mapping configuration file defines a mapping relationship of the logical binding between the sensing device 130 and the resource pool 120. In some embodiments, the management platform 110 performs a query, based on identification information of the intra-pool forwarding request, in the mapping configuration file (e.g., by querying the mapping configuration file based on a device identifier in the identification information) to determine a second target resource pool logically bound by a second sensing device, or the second target resource pool and a second target node that establishes a communication connection with the second sensing device within the second target resource pool. The management platform 110 may send the image retrieval request to the second target resource pool or the second target node, and in response to receiving the intra-pool forwarding request, the second target resource pool or the second target node reads target data corresponding to the intra-pool forwarding request from the second target resource pool. The second sensing device is a sensing device that acquires the target data.
[0179] In conjunction with the foregoing, when the intra-pool forwarding request is an image retrieval request, the target data includes a target image requested to be accessed by the image retrieval request. In some embodiments, the management platform 110 performs a query in the mapping configuration file based on the image retrieval request to determine the second target node that establishes a communication connection with the second sensing device, and sends the image retrieval request to the second target node, and the second target node reads a target image corresponding to the image retrieval request from the second target resource pool in response to receiving the image retrieval request. At this time, the second sensing device is a sensing device that acquires the target image. For example, the PaaS management service in the storage management platform 113 performs a query, based on a device identifier in a second request identifier corresponding to the image retrieval request, in the mapping configuration file for information containing the device identifier, determines, based on the queried information, the second target node that is in a communication connection with the second sensing device, and further forwards the image retrieval request to the second target node. The second target node may, in response to the image retrieval request, obtain the target image from the second target resource pool and send the target image to the service management platform 112.
[0180] More descriptions regarding image retrieval, video playback, and other intra-pool forwarding service can be found in the description of the aforementioned embodiments of the cloud storage system 100, and will not be repeated here.
[0181] As previously described, an intra-pool storage sub-node of a data node 121 (e.g., a first sensing device) that is in a communication connection with a sensing device 130 (e.g., the first target node) may generate a distributed storage request and send the distributed storage request to a distributed storage sub-node within the data node 121 to invoke an SDK interface in the distributed storage sub-node of the data node 121 of the data node 121 for data writing.
[0182] In some embodiments, in response to an intra-pool storage sub-node in a target data node invoking a cloud storage interface for data writing, the management platform 110 (e.g., the storage management platform 113) obtains a distributed storage request from a distributed storage sub-node, the distributed storage request including identification information of the distributed storage sub-node, determine, based on the identification information of the distributed storage sub-node, a data node where the cloud storage interface issuing the distributed storage request belongs to be a third target node; and allocates, based on the topology configuration file, at least one storage space within the resource pool where the third target node is located to enable the cloud storage interface to store each of segments to be written obtained by slicing the data to be written to each of the least one storage space. For example, the storage management platform 113 obtains the IP address of the third target node, such as 192.168.2.20, and according to the topology configuration file shown in FIG. 2, determines that the resource pool where the third target node is located is a resource pool with a "pool_name" of "ShanghaiPool" , and further allocates at least one storage space in the "ShanghaiPool" based on load balancing.
[0183] In some embodiments, in response to a failure of at least one sub-node of at least one data node of the data nodes within the resource pools in the cloud storage system, the management platform 110 (e.g., the storage management platform 113) determines, based on the topology configuration file, a fourth target node from a resource pool in which the at least one data node is located. At least one sub-node of the fourth target node is configured to perform the service of the at least one failed sub-node of the at least one data node, the fourth target node being a normal data node. For example, the storage management platform 113 determines, based on the topology configuration file, a resource pool where the failed data node is located, determines, based on the load of each normal node in the resource pool, the fourth target node from the normal nodes for the purpose of load balancing, and prioritizes the transparent transmission of output data via the fourth target node to each sensing device of the failed data node to carry out an intra-pool disaster recovery. For more details, see the preceding description of embodiments of the cloud storage system 100, which will not be repeated herein.
[0184] In some embodiments, in response to a failure of at least one sub-node in at least one data node of the data nodes of the resource pool, the management platform 110 (e.g., the storage management platform 113) determines a fifth target node based on the topology configuration file and loads of the data nodes within the resource pool where the at least one data node is located, and performs a data recovery task at the fifth target node for the at least one failed sub-node in the at least one data node, the fifth target node being a normal data node. For more details, see the preceding description of an embodiment of the cloud storage system 100, which will not be repeated herein.
[0185] It should be noted that the foregoing description of the process 400 in FIG. 4 is for the purpose of example and illustration only and does not limit the application scope of the present disclosure. For a person skilled in the art, various modifications and changes may be made to the process 400 under the teaching of this disclosure. However, these modifications and changes remain within the scope of this disclosure.
[0186] FIG. 5 is an exemplary flowchart illustrating a cloud storage management method according to some other embodiments of the present disclosure. In some embodiments, the process 500 may be performed by the cloud storage system 100 (e.g., the management platform 110) or the computing device 300. As shown in FIG. 5, the process 500 includes:
[0187] Operation 510, a communication connection is established with each of the resource pools. In some embodiments, step 510 is performed by the management platform 110.
[0188] As described in the preceding section, the cloud storage system 100 may include the management platform 110 and the resource pools 120, at least one sensing device 130 may be configured to logically bind to a resource pool 120, and the output data of the at least one sensing device 130 may be stored to the resource pool. In some embodiments, the management platform 110 (e.g., the service management platform 112, the storage management platform 113) establishes a communication connection with each resource pool 120 in the cloud storage system 100. For further description, see the preceding embodiments of the cloud storage system 100, which will not be repeated herein.
[0189] Operation 520, a service request is received and responded to. In some embodiments, step 520 is performed by the management platform 110.
[0190] In conjunction with the foregoing, the service request may include at least one of the intra-pool access request, the intra-pool forwarding request, the intra-pool storage request, and the distributed storage request. The management platform 110 may include the service management platform 112 and the storage management platform 113. The service management platform 112 may be configured to receive the service request, and respond to the received service request. Or, the service management platform 112 may send the received service request to the storage management platform 113, and the storage management platform 113 may respond to the service request.
[0191] In some embodiments, when receiving an intra-pool access request associated with a first sensing device, the management platform 110 (e.g., the service management platform 112 or the storage management platform 113) obtains, based on a physical location of the first sensing device, a distance between the first sensing device and each of the resource pools respectively; and determines, based on the distance between the first sensing device and each of the resource pools respectively, a first target resource pool from the resource pools to logically bind the first sensing device to the first target resource pool.
[0192] In conjunction with the preceding, when receiving an intra-pool forwarding request, the management platform 110 (e.g., the service management platform 112) may determine, based on the identification information in the intra-pool forwarding request, a second target node that is in a communication connection with a second sensing device, the second sensing device being logically bound to a second target resource pool, the second target node being located in the second target resource pool, and forward the intra-pool forwarding request to an intra-pool forwarding sub-node of the second target node, the intra-pool forwarding sub-node of the second target node reading the target data from the second target resource pool in response to receiving the intra-pool forwarding request and sending the target data to the management platform.
[0193] In some embodiments, in response to determining that the intra-pool forwarding request is a video playback request, the service management platform 112 determines, based on first identification information (afirst request identifier) in the video playback request, a target node (i.e., the second target node) that is in a communication connection with the second sensing device, and forwards the video playback request to an intra-pool forwarding sub-node of the target node. The intra-pool forwarding sub-node may, in response to receiving the video playback request, read a target video corresponding to the video playback request from the second target resource pool. At this time, the second sensing device is a sensing device that acquires the target video.
[0194] In some embodiments, in response to determining that the intra-pool forwarding request is an image retrieval request, the service management platform 112 determines a target node that is in a communication connection with a third sensing device based on second identification information (asecond request identifier) in the image retrieval request, and forwards the image retrieval request to an intra-pool forwarding sub-node of the target node. The intra-pool forwarding sub-node may read a target image corresponding to the image retrieval request from a third target resource pool in response to receiving the image retrieval request. The third sensing device is a sensing device that acquires the target image, the target node is disposed within the third target resource pool, and the third sensing device is logically bound to the third target resource pool.
[0195] For further description regarding the intra-pool forwarding service, see the preceding description of the embodiments of the cloud storage system 100, which will not be repeated here.
[0196] It should be noted that the foregoing description of the process 500 is intended to be exemplary and illustrative only and does not limit the application scope of the present disclosure. For a person skilled in the art, various corrections and changes can be made to the process 500 under the guidance of this disclosure. However, these corrections and changes remain within the scope of this disclosure.
[0197] It should be noted that the terms "first sensing device, " "second sensing device, " "first target resource pool, " "second target resource pool, " "third target resource pool, " "first target node, " "second target node, " "third target node, " "fourth target node, " and "fifth target node" in the embodiments of the present disclosure are solely used for distinction and do not imply specific limitations. In some embodiments, the "first sensing device" and "second sensing device" may be identical or distinct devices, while the "first target resource pool, " "second target resource pool, " and "third target resource pool" may represent the same or different resource pools. The "first target node, " "second target node, " "third target node, " "fourth target node, " and "fifth target node" could be identical or different data nodes, potentially belonging to the same resource pool or distributed across multiple pools. For example, if the first sensing device and the second sensing device are the same device (e.g., a sensing device 130) , which is logically bound to a single resource pool (e.g., a resource pool 120) , both the associated target nodes (first target node and second target node) would correspond to the same data node (e.g., a node 121) within that resource pool. In this scenario, in response to receiving an intra-pool access request, the data node establishes communication with the sensing device, and in response to receiving an intra-pool access request, an intra-pool forwarding sub-node in the data node may subsequently retrieve and target data relevant to the intra-pool forwarding request from the resource pool 120, and send the target data to at least one sensing device. As another example, the third target resource pool and the first resource pool (or the second resource pool) may be the same, and the third target node and the first target node (or the second target node) may be the same target node within the same resource pool or different target nodes within the same resource pool. As still another example, the first target node, the second target node, the third target node, the fourth target node, and the fifth target node may be the same target node within one resource pool or different target nodes within one resource pool.
[0198] FIG. 6 is a frame diagram of an electronic device according to some embodiments of the present disclosure.
[0199] As shown in FIG. 6, the electronic device 600 includes a communication circuit 610, a memory 620, and a processor 630. The communication circuit 610 is coupled to the memory 620 and the processor 630 respectively, the memory 620 stores program instructions, and the processor 630 is configured to execute the program instructions to implement the steps in the foregoing embodiments of the cloud storage management method (e.g., process 400 and process 500) . The electronic device 600 may include, but is not limited to, servers, etc., and is not limited herein.
[0200] Specifically, the processor 630 may also be referred to as a central processing unit (CPU) . The processor 630 may be an integrated circuit chip with signal processing capabilities. The processor 630 may also be a general-purpose processor, a digital signal processor, a special-purpose integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc. Alternatively, the processor 630 may be implemented by a plurality of circuit-forming chips together.
[0201] FIG. 7 is a frame diagram of a computer-readable storage medium according to some embodiments of the present disclosure.
[0202] As shown in FIG. 7, the present disclosure also provides a computer-readable storage medium 700 having stored thereon program instructions 710 that are runnable by a processor, the program instructions 710 being executed by the processor to implement the cloud storage management method described in any embodiment of the present disclosure (e.g., process 400 and process 500) . The program instructions may be formed into a program file stored in the storage medium in the form of a software product to cause a computer device (which may be a personal computer, a server, or a network device, etc. ) or a processor to perform all or some of the steps of the method of the various embodiments of the present disclosure. The aforementioned storage medium includes a USB flash drive, a mobile hard drive, a read-only memory (ROM, Read-Only Memory) , random access memory (RAM, Random Access Memory) , a disk or a CD-ROM, and other kinds of medium that can store program code, or a terminal device such as a computer, a server, a cell phone, and a tablet.
[0203] It should be noted that the descriptions "first" , "second" , "third" , "fourth" , "fifth" and so on are used in the embodiments of this disclosure for the purpose of illustration and understanding. The "first" , "second" , "third" , "fourth" , "fifth" , etc., may be used interchangeably between devices of the same type. For example, "the first sensing device" and "the second sensing device" may be interchangeable, and each device of the same type may be the same device or different devices (e.g., the first sensing device and the second sensing device may be the same sensing device 130 or different sensing devices 130, the first target resource pool and the second target resource pool may be the same resource pool 120 or different resource pools 120) . However, a sensing device can only be logically bound to one resource pool, and a resource pool can logically bind a plurality of sensing devices.
[0204] Having thus described the basic concepts, it may be rather apparent to those skilled in the art after reading this detailed disclosure that the foregoing detailed disclosure is intended to be presented by way of example only and is not limiting. Various alterations, improvements, and modifications may occur and are intended for those skilled in the art, though not expressly stated herein. These alterations, improvements, and modifications are intended to be suggested by this disclosure and are within the spirit and scope of the exemplary embodiments of this disclosure.
[0205] Moreover, certain terminology has been used to describe embodiments of the present disclosure. For example, the terms “one embodiment, ” “an embodiment, ” and / or “some embodiments” mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. Therefore, it is emphasized and should be appreciated that two or more references to “an embodiment” or “one embodiment” or “an alternative embodiment” in various portions of this disclosure are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined as suitable in one or more embodiments of the present disclosure.
[0206] Furthermore, the recited order of processing elements or sequences, or the use of numbers, letters, or other designations, therefore, is not intended to limit the claimed processes and methods to any order except as may be specified in the claims. Although the above disclosure discusses through various examples what is currently considered to be a variety of useful embodiments of the disclosure, it is to be understood that such detail is solely for that purpose and that the appended claims are not limited to the disclosed embodiments, but, on the contrary, are intended to cover modifications and equivalent arrangements that are within the spirit and scope of the disclosed embodiments. For example, although the implementation of various components described above may be embodied in a hardware device, it may also be implemented as a software-only solution, e.g., an installation on an existing server or mobile device.
[0207] Similarly, it should be appreciated that in the foregoing description of embodiments of the present disclosure, various features are sometimes grouped together in a single embodiment, figure, or description thereof to streamline the disclosure aiding in the understanding of one or more of the various inventive embodiments. This method of disclosure, however, is not to be interpreted as reflecting an intention that the claimed object matter requires more features than are expressly recited in each claim. Rather, inventive embodiments lie in less than all features of a single foregoing disclosed embodiment.
[0208] In some embodiments, the numbers expressing quantities, properties, and so forth, used to describe and claim certain embodiments of the present disclosure are to be understood as being modified in some instances by the term “about, ” “approximate, ” or “substantially. ” For example, “about, ” “approximate” or “substantially” may indicate ±20%variation of the value it describes, unless otherwise stated. Accordingly, in some embodiments, the numerical parameters set forth in the written description and attached claims are approximations that may vary depending upon the desired properties sought to be obtained by a particular embodiment. In some embodiments, the numerical parameters should be construed in light of the number of reported significant digits and by applying ordinary rounding techniques. Notwithstanding that the numerical ranges and parameters setting forth the broad scope of some embodiments of the present disclosure are approximations, the numerical values set forth in the specific examples are reported as precisely as practicable.
[0209] Each of the patents, patent applications, publications of patent applications, and other material, such as articles, books, specifications, publications, documents, things, and / or the like, referenced herein is hereby incorporated herein by this reference in its entirety for all purposes, excepting any prosecution file history associated with same, any of same that is inconsistent with or in conflict with the present document, or any of same that may have a limiting effect as to the broadest scope of the claims now or later associated with the present document. By way of example, should there be any inconsistency or conflict between the description, definition, and / or the use of a term associated with any of the incorporated material and that associated with the present document, the description, definition, and / or the use of the term in the present document shall prevail.
[0210] If the technical solutions of this application involve personal information, prior to processing personal information, the products applying these technical solutions have clearly informed the individuals of the personal information processing rules and obtained autonomous consent from the individuals. If the technical solutions of this application involve sensitive personal information, prior to processing sensitive personal information, the products applying these technical solutions have obtained separate consent from the individuals while also fulfilling the requirement of "explicit consent" . For instance, clear and conspicuous signs may be placed at personal information collection devices such as cameras to notify individuals that they are entering a personal information collection area, and that personal information will be collected. If an individual voluntarily enters the collection area, this shall be deemed as consent to the collection of their personal information. Alternatively, under circumstances where clear signs / information indicating personal information processing rules are displayed on the processing devices, authorization may be obtained through pop-up notifications or by requesting individuals to voluntarily upload their personal information. The personal information processing rules may include details such as the processor of the personal information, purpose of processing, processing methods, and types of personal information involved.
[0211] In closing, it is to be understood that the embodiments of the application disclosed herein are illustrative of the principles of the embodiments of the application. Other modifications that may be employed may be within the scope of the application. Thus, by way of example, but not of limitation, alternative configurations of the embodiments of the application may be utilized in accordance with the teachings herein. Accordingly, embodiments of the present application are not limited to that precisely as shown and described.
Claims
1.A cloud storage system, comprising:sensing devices configured to collect data;resource pools, wherein each of the resource pools includes data nodes, and each of the sensing devices is configured to be in a communication connection with at least one data node in one of the resource pools; anda management platform communicatively connected to the resource pools, and configured to manage the resource pools and / or the data nodes within each resource pool.2.The cloud storage system according to claim 1, wherein the resource pools are within a same platform as a service (PaaS) domain.3.The cloud storage system according to claim 1, wherein each of the data nodes includes sub-nodes, each of the sub-nodes is configured to provide a service in response to receiving a corresponding service request, and the service includes one of an intra-pool access service, an intra-pool forwarding service, an intra-pool storage service, and a distributed storage service.4.The cloud storage system according to claim 3, wherein when the service request is an intra-pool access request for the intra-pool access service, the management platform is configured to:in response to the intra-pool access request associated with a first sensing device, obtain a distance between the first sensing device and each resource pool of the resource pools based on a physical location of the first sensing device; anddetermine, from the resource pools, a first target resource pool based on the distance between the first sensing device and the each resource pool, wherein the first sensing device is configured to be logically bound to the first target resource pool.5.The cloud storage system according to claim 4, wherein the management platform is further configured to:determine a first target node for the first sensing device based on a load of each of the data nodes within the first target resource pool, wherein the first sensing device is configured to be connected communicatively with the first target node.6.The cloud storage system according to claim 5, wherein in response to receiving the intra-pool access request, an intra-pool access sub-node of the sub-nodes of the first target node is configured to establish the communication connection between the first target node and the first sensing device.7.The cloud storage system according to claim 3, wherein when the service request is an intra-pool storage request for the intra-pool storage service, an intra-pool storage sub-node in a first target node that is in the communication connection with a first sensing device is configured to:obtain output data of the first sensing device; andstore the output data by invoking a target distributed storage sub-node in a first target resource pool where the first target node is located, wherein a data node where the target distributed storage sub-node is located is the same as or different from the first target node.8.The cloud storage system according to claim 7, wherein the management platform is further configured to:in response to determining that a load of the first target node does not meet a preset condition, determine, from the data nodes within the first target resource pool, a new first target node based on a load of each of the data nodes in the first target resource pool to obtain the output data of the first sensing device through the new first target node.9.The cloud storage system according to claim 3, wherein when the service request is an intra-pool forwarding request for the intra-pool forwarding service, the management platform is configured to:obtain the intra-pool forwarding request, wherein the intra-pool forwarding request includes identification information of target data that needs to be forwarded, and the identification information includes a device identifier of a second sensing device that acquires the target data and an acquisition time;determine a second target node that is in a communication connection with the second sensing device based on the identification information, wherein the second target node is located within a second target resource pool, and the second sensing device is logically bound to the second target resource pool; andforward the intra-pool forwarding request to a forwarding sub-node of the second target node, wherein in response to receiving the intra-pool forwarding request, the forwarding sub-node of the second target node is configured to read the target data from the second target resource pool and send the target data to at least one target device.10.The cloud storage system according to claim 9, wherein when the intra-pool forwarding request represents a video playback request, the target data includes a target video, the video playback request includes first identification information of the target video, and the management platform is configured to:determine the second target node based on the first identification information of the target video; andforward the video playback request to the forwarding sub-node of the second target node, wherein in response to receiving the video playback request, the forwarding sub-node is configured to read the target video from the second target resource pool and send the target video to the at least one target device.11.The cloud storage system according to claim 9, wherein when the intra-pool forwarding request represents an image retrieval request, the target data includes a target image, the image retrieval request includes second identification information of the target image, and the management platform is configured to:determine the second target node based on the second identification information of the target image, and forward the image retrieval request to the forwarding sub-node of the second target node, wherein in response to the image retrieval request, the forwarding sub-node is configured to read the target image from the second target resource pool and send the target image to the at least one target device.12.The cloud storage system according to claim 3, wherein when the service request is a distributed storage request, the management platform is configured to:obtain the distributed storage request from the distributed storage sub-node in a data node, the distributed storage request including an identifier of the distributed storage sub-node; anddetermine, based on the distributed storage request, a targe data node where the cloud storage interface is located; andallocate at least one storage space in a third target resource pool where the third target node is located, wherein the cloud storage interface is configured to store each write segment obtained by slicing data to be stored into a respective storage space.13.The cloud storage system according to claim 1, wherein the management platform is further configured to:in response to a failure of at least one sub-node in at least one data node of the data nodes, determine a fourth target node from a resource pool where the at least one data node is located, wherein at least one sub-node of the fourth target node is used to execute a service of the at least one failed sub-node in the at least one data node, and the fourth target node is a normal data node.14.The cloud storage system according to claim 1, wherein the management platform is further configured to:in response to a failure of at least one sub-node in at least one data node of the data nodes, determine a fifth target node from data nodes within a resource pool where the at least one data node is located based on a load of each data node of the data nodes, and perform a data recovery task for the at least one failed sub-node in the at least one data node on the fifth target node, wherein the fifth target node is a normal data node.15.A method for cloud storage management, executed by a management platform in a cloud storage system, wherein the cloud storage system further includes sensing devices and resource pools with no traffic interaction between the resource pools, and each of the sensing devices is logically bound to a resource pool of the resource pools, and output data of the sensing device is stored in the resource pool, the method comprising:obtaining a topology configuration file of the cloud storage system;establishing a communication connection between the management platform with each of the resource pools, wherein the topology configuration file defines the resource pools and data nodes within each resource pool; andmanaging the each resource pool and / or the data nodes within the each resource pool based on the topology configuration file.16.The method according to claim 15, wherein the method further comprises:in response to determining that a first sensing device is logically bound to a first target resource pool, detrmining, based on the topology configuration file and a load of each data node within the first target resource pool, a first target node within the first target resource pool, wherein the first target node is configured to obtain output data of the first sensing device.17.The method according to claim 16, wherein the method further comprises:in response to determining that a load of the first target node does not meet a preset condition, determining, based on the topology configuration file and the load of each data node within the first target resource pool, a new first target node from the data nodes within the first target resource pool, to obtain the output data of the first sensing device through the new first target node.18.The method according to claim 15, wherein when the service request is an intra-pool forwarding request representing an image retrieval request, the method further comprises:obtaining a mapping configuration file, the mapping configuration file defines a mapping relationship of a logical binding between the sensing device and the resource pool;querying the mapping configuration file based on the image retrieval request to determine a second target node that is in a communication connection with a second sensing device, wherein the second sensing device is a device that acquires a target image requested by the image retrieval request, the second sensing device is logically bound to a second target resource pool, and the second target node is located in the second target resource pool ; andsending the image retrieval request to the second target node, wherein in response to receiving the image retrieval request, the second target node reads the target image corresponding to the image retrieval request from the second target resource pool.19.The method according to claim 15, wherein the method further comprises:in response to determining that an intra-pool storage sub-node within the target data node invokes a cloud storage interface for data writing, obtaining a distributed storage request from a distributed storage sub-node, the distributed storage request including an identifier of the distributed storage sub-node;determining, based on the identifier of the distributed storage sub-node, a data node where a cloud storage interface issuing the distributed storage request is located as a third target node, andallocating at least one storage space within the resource pool where the third target node is located based on the topology configuration file, wherein the cloud storage interface stores each data segment obtained by slicing data to be written into a respective storage space.20.The method according to claim 15, wherein the method further comprises:in response to a failure of at least one sub-node in at least one data node of data nodes within the resource pool, determining based on the topology configuration file, a fourth target node from the resource pool where the at least one data node is located, wherein at least one sub-node in the fourth target node is used to execute a business of the at least one failed sub-node in the at least one data node, and the fourth target node is a normal data node.21.The method according to claim 15, wherein the method further comprises:in response to a failure of at least one sub-node in at least one data node of data nodes of the resource pool, determining a fifth target node based on the topology configuration file and a load of each data node within the resource pool where the at least one data node is located, and executing a data recovery task for the at least one failed sub-node in the at least one data node on the fifth target node, wherein the fifth target node is a normal data node.22.A method for cloud storage management, executed by a management platform in a cloud storage system, wherein the cloud storage system further includes resource pools, and at least one sensing device is logically bound to a resource pool of the resource pools , output data of the at least one sensing device being stored in the resource pool, the method comprising:establishing a communication connection with each of the resource pools, wherein each resource pool includes data nodes;receiving and responding to a business request, wherein the business requests include at least one of an intra-pool access request, an intra-pool forwarding request, an intra-pool storage request, and a distributed storage request.23.The method according to claim 22, wherein, in response to the business request being the intra-pool access request, the method further comprises:obtaining a distance between a first sensing device and each of the resource pools based on a physical location of the first sensing device; anddetermining, from the resource pools, a first target resource pool based on the distance between the first sensing device and each resource pool, wherein the first sensing device is logically bound to the first target resource pool.24.The method according to claim 22, wherein, in response to the service request being an intra-pool forwarding request presenting a video playback request, the method further comprises:determining a target node that is in a communication connection with a second sensing device based on first identification information in the video playback request, wherein the target node is located within a second target resource pool, and the second sensing device is logically bound to the second target resource pool;forwarding the video playback request to a read sub-node of the target node, wherein in response to the video playback request, the read sub-node reads a target video corresponding to the video playback request from the second target resource pool, and the second sensing device is a sensing device that collected the target video.25.The method according to claim 22, wherein, in response to the service request being an intra-pool forwarding request representing an image retrieval request, the method further comprises:determining a target node that is in a communication connection with a third sensing device based on second identification information in the image retrieval request, wherein the target node is located within a third target resource pool, and the third sensing device is logically bound to the third target resource pool;forwarding the image retrieval request to a read sub-node of the target node, wherein in response to the image retrieval request, the read sub-node reads a target image corresponding to the image retrieval request from the third target resource pool, and the third sensing device is a sensing device that captured the target image.26.An electronic device comprising a processor and a memory connected to each other, wherein the memory is configured to store a program, and the processor is configured to execute the program to implement the method according to any one of claims 15-25.27.A computer-readable storage medium having a program stored thereon, the program being executed by a processor to implement the method according to any one of claims 15-25.
Citation Information
Patent Citations
Front-end equipment access method and device, server and storage medium
CN107800694A
Data storing, reading and cleaning methods and devices and cloud storage system
CN107967117A
Cloud storage system, media data storage method and system
CN108632305A
Cloud storage system, media data equalization storage method and system
CN108737477A
Cloud storage system and cloud storage management method
CN118694764A