A cloud storage system and a cloud storage management method

By adopting a distributed resource pool structure and a unified management platform in the cloud storage system, the difficulties in building cloud storage systems in multi-domain scenarios and the load pressure on the core backbone network have been solved, achieving low-cost and efficient data storage and management.

CN118694764BActive Publication Date: 2024-11-29ZHEJIANG DAHUA TECH CO LTD
View PDF 12 Cites 0 Cited by

Patent Information

Application Number
CN202411177403.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-26
Publication Date
2024-11-29
Estimated Expiration
2044-08-26

AI Technical Summary

Technical Problem

In multi-domain scenarios, cloud storage systems are difficult and costly to build, and the core backbone network is under heavy load, making data sharing inconvenient and management complex.

Method used

The system adopts a distributed resource pool structure, with load balancing and scheduling management implemented within each resource pool. It is managed uniformly through a business management platform and a storage management platform. There is no traffic interaction between resource pools, and logical binding and data storage are performed using topology configuration files.

Benefits of technology

It reduces the difficulty and cost of building cloud storage systems, reduces the load pressure on core backbone networks, and improves the convenience of system management and the efficiency of data sharing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118694764B_ABST
    Figure CN118694764B_ABST
Patent Text Reader

Abstract

The present application discloses a cloud storage system and a cloud storage management method. The cloud storage system includes: a number of resource pools, a business management platform and a storage management platform. The resource pool includes a number of data nodes for logically binding with the front-end sensing device. The output data of the front-end sensing device is stored in the resource pool logically bound with the front-end sensing device. There is no traffic interaction between different resource pools. The business management platform is respectively communicated with each resource pool, at least for responding to business requests. The storage management platform is respectively communicated with each resource pool, at least for performing load balancing and scheduling management in each resource pool based on a topology configuration file. The topology configuration file defines each resource pool and the data nodes contained in the resource pool. The above scheme can reduce the load pressure of the core trunk network as much as possible and improve the convenience of system management while reducing the difficulty and cost of building a cloud storage system.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of distributed storage technology, and in particular to a cloud storage system and a cloud storage management method. Background Art

[0002] In order to effectively store the massive data of front-end sensing devices, a storage system is usually built to store the massive data in a unified manner. However, this method is relatively difficult and costly.

[0003] Based on this, the industry adopts a decentralized construction method in multiple locations, and unifies the management entrance through a multi-domain mode. However, in a multi-domain scenario, each domain implements device access and data storage of its own front-end sensing devices, which easily forms data islands, making cross-domain data sharing inconvenient, and aggregating data in different domains will bring huge overhead, which is likely to cause core trunk network traffic pressure and large communication delays. In addition, although there is a unified business entrance, the process is complicated because each domain needs to be operated and managed separately, and it is impossible to achieve truly unified global management, unified load, and unified storage. In view of this, how to reduce the core trunk network load pressure as much as possible and improve the convenience of system management while reducing the difficulty and cost of cloud storage system construction has become an urgent problem to be solved. Summary of the invention

[0004] The main technical problem solved by this application is to provide a cloud storage system and a cloud storage management method, which can reduce the load pressure of the core trunk network as much as possible and improve the convenience of system management while reducing the difficulty and cost of building a cloud storage system.

[0005] In order to solve the above problems, the first aspect of the present application provides a cloud storage system, including: several resource pools, a business management platform and a storage management platform, the resource pool includes several data nodes, the resource pool is used to logically bind with the front-end sensing device, the output data of the front-end sensing device is stored in the resource pool logically bound to the front-end sensing device, and there is no traffic interaction between different resource pools; the business management platform is communicated and connected with each resource pool respectively, and is at least used to respond to business requests, and the business requests include at least one of intra-pool access and intra-pool forwarding; the storage management platform is communicated and connected with each resource pool respectively, and is at least used to perform load balancing and scheduling management in each resource pool based on a topology configuration file, and the topology configuration file defines each resource pool and the data nodes contained in the resource pool.

[0006] In order to solve the above problems, the second aspect of the present application provides a cloud storage management method, including: obtaining a topology configuration file and establishing a communication connection with each resource pool in the cloud storage system; wherein the topology configuration file defines each resource pool and the data nodes contained in the resource pool, the resource pool is used to be logically bound to the front-end perception device, the output data of the front-end perception device is stored in the resource pool logically bound to the front-end perception device, and there is no traffic interaction between different resource pools. The cloud storage system also includes a business management platform that is communicatively connected to each resource pool, which is used to receive and respond to business requests, and the business requests include at least one of intra-pool access and intra-pool forwarding; based on the topology configuration file, load balancing and scheduling management are performed in each resource pool respectively.

[0007] In order to solve the above problems, the third aspect of the present application provides a cloud storage management method, including: establishing a communication connection with each resource pool in the cloud storage system; wherein the resource pool includes a number of data nodes, the resource pool is used to logically bind with the front-end sensing device, the output data of the front-end sensing device is stored in the resource pool logically bound to the front-end sensing device, there is no traffic interaction between different resource pools, and the cloud storage system also includes a storage management platform that is communicatively connected to each resource pool, and is used to perform load balancing and scheduling management in each resource pool based on a topology configuration file, and the topology configuration file defines each resource pool and the data nodes contained in the resource pool; receiving and responding to business requests; wherein the business request includes at least one of intra-pool access and intra-pool forwarding.

[0008] In the above scheme, the cloud storage system includes several resource pools, a business management platform and a storage management platform. The resource pool includes several data nodes. The resource pool is used to logically bind with the front-end sensing device. The output data of the front-end sensing device is stored in the resource pool logically bound with the front-end sensing device, and there is no traffic interaction between different resource pools. The business management platform is communicated with each resource pool respectively, and is at least used to respond to business requests. The business request includes at least one of access within the pool and forwarding within the pool. The storage management platform is communicated with each resource pool respectively, and is at least used to perform load balancing and scheduling management in each resource pool based on the topology configuration file. The topology configuration file The configuration file defines each resource pool and the data nodes contained in the resource pool. Therefore, on the one hand, the decentralized construction of multiple resource pools helps to reduce the difficulty and cost of cloud storage system construction compared to centralized construction. On the other hand, since storage, access, forwarding and other services are implemented in the resource pool, and there is no traffic interaction between resource pools, it can avoid storage, business and other traffic passing through the core trunk network as much as possible, which helps to reduce the load pressure of the core trunk network. On the other hand, since multiple resource pools are managed through a set of management platforms (including business management platforms and storage management platforms), unified management can be achieved compared to the multi-domain model. Therefore, under the premise of reducing the difficulty and cost of cloud storage system construction, the load pressure of the core trunk network can be reduced as much as possible, and the convenience of system management can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 It is a schematic diagram of the framework of an embodiment of the cloud storage system of the present application;

[0010] Figure 2 is a schematic diagram of an embodiment of a topology configuration file;

[0011] Figure 3 It is a flowchart of an embodiment of the cloud storage management method of the present application;

[0012] Figure 4 It is a flowchart of another embodiment of the cloud storage management method of the present application;

[0013] Figure 5 It is a schematic diagram of the framework of an embodiment of the electronic device of the present application;

[0014] Figure 6 It is a schematic diagram of a framework of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0015] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0016] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0017] The terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship. In addition, "many" in this article means two or more than two.

[0018] See also Figure 1 , Figure 1 Schematic diagram of the framework of an embodiment of the cloud storage system of the present application. Figure 1 As shown, the cloud storage system 10 of the present application may include several resource pools 11, a business management platform 12 and a storage management platform 13. The resource pool 11 includes several data nodes 111. The resource pool 11 is used to be logically bound to the front-end sensing device 20. The output data of the front-end sensing device 20 is stored in the resource pool 11 logically bound to the front-end sensing device 20, and there is no traffic interaction between different resource pools 11. The business management platform 12 is respectively connected to each resource pool 11 in communication, and is at least used to respond to business requests. The business request includes at least one of access within the pool and forwarding within the pool. The storage management platform 13 is respectively connected to each resource pool 11 in communication, and is at least used to perform load balancing and scheduling management in each resource pool 11 based on the topology configuration file. The topology configuration file defines each resource pool 11 and the data nodes 111 contained in the resource pool 11. In other words, load balancing and scheduling management can be completed in the pool. It should be noted that in the cloud architecture, a domain is a fully functional PaaS (Platform as a Service) cluster, which has the capabilities of front-end IoT (Internet of Things) device access, data storage, transmission, analysis, sharing, etc., and provides interfaces for the upper-level business platform to provide various data-based business capabilities. Since multiple resource pools 11 in this application share a management platform (i.e., including a business management platform 12 and a storage management platform 13), the cloud storage system 10 of this application can be regarded as a domain that has the characteristics of being built in multiple locations in terms of architecture and is managed by only one management platform. In addition, as a possible example, the business management platform 12 can specifically be a SaaS (Software as a Service) management platform.

[0019] In an implementation scenario, each data node 111 in the same resource pool 11 can be located in the same geographical location, and the data nodes 111 belonging to different resource pools 11 are located in different geographical locations. In other words, different resource pools 11 are constructed in multiple locations. It should be noted that a resource pool 11 can be a data center, a batch of racks, or several cloud direct storage nodes. The hardware architecture of the resource pool 11 is not limited here. Therefore, it is possible to implement pool management of different hardware types (different configurations, different models, different operating systems), and achieve compatible scenarios of isomorphic hardware within the pool and heterogeneous hardware between pools, and then support multi-phase expansion construction of heterogeneous hardware in the same system, which helps to solve the industry problem of heterogeneous expansion.

[0020] In a specific implementation scenario, as a possible example, the aforementioned same geographical location may refer to the same computer room. That is, a plurality of data nodes 111 may be formed in the same computer room as a resource pool 11.

[0021] In another specific implementation scenario, as another possible example, the aforementioned same geographical location may refer to the same data center. That is, a plurality of data nodes 111 may be formed in the same data center as a resource pool 11.

[0022] In another specific implementation scenario, as another possible example, the aforementioned same geographical location may refer to the same administrative region (eg, district, city, etc.). That is, a plurality of data nodes 111 may be formed in the same administrative region as a resource pool 11.

[0023] It should be noted that, as mentioned above, the specific scope of the geographical location can be set according to actual conditions, such as a computer room, data center, administrative area, etc. The specific scope of the geographical location is not limited here.

[0024] In one implementation scenario, as a preferred example, in order to achieve load balancing in the resource pool 11, the resource pool 11 may include multiple data nodes 111, and the total number of data nodes 111 in the resource pool 11 is not limited.

[0025] In an implementation scenario, the front-end sensing device 20 can be used to collect data such as the environment, and the data format may include but is not limited to: video, image, structured data, etc. Taking the front-end sensing device 20 collecting structured data as an example, the front-end sensing device 20 may specifically include but is not limited to: humidity sensors, temperature sensors, wind sensors, etc., to collect and form structured data containing environmental indicators (such as humidity, temperature, wind, etc.) at each moment. Of course, the above examples are only a few possible examples in the actual application process, and the front-end sensing device 20 may also include other types of devices, and the specific types of the front-end sensing device 20 are not limited here.

[0026] In one implementation scenario, as described above, the topology configuration file may define each resource pool 11 and the data nodes 111 contained in the resource pool, that is, the topology configuration file may reflect the topological relationship of multiple resource pools 11 in the cloud storage system 10. Figure 2 , Figure 2 Schematic diagram of an embodiment of a topology configuration file. Figure 2 As shown, Figure 2 The cloud storage system 10 defined by the topology configuration file shown includes two resource pools 11, wherein the unique identifier of one resource pool 11 (i.e., pool_id) is 148618787703220001, and the unique identifier of the other resource pool 11 (i.e., pool_id) is 148618787703220002. In addition, pool_type is used to identify the pool category of the resource pool 11. As a possible example, the pool category may reflect the type of data that the resource pool 11 is responsible for accessing (e.g., videos, images, structured data, etc.). Of course, the pool category may also reflect other indicators, which are not limited here. pool_name is used to identify the name of the resource pool 11. datanode_list is used to reflect the data nodes contained in the resource pool 11. As a possible example, Figure 2 As shown, the IP address of the data node can be used to represent the data node, that is, datanode_list can specifically include the IP address of each data node in the resource pool 11. It should be noted that Figure 2 The above is only a possible example of a topology configuration file, which does not limit the specific form of the topology configuration file reflecting each resource pool and the data nodes contained therein. The topology configuration file will not be given examples one by one here.

[0027] In an implementation scenario, the service management platform 12 can specifically respond to the service request for in-pool access, obtain the relative distances between the front-end sensing device 20 and each resource pool 11 based on the physical location of the front-end sensing device 20, and select a resource pool 11 to be logically bound to the front-end sensing device 20 based on the relative distances between the front-end sensing device 20 and each resource pool 11. In the above manner, by measuring the relative distances between the front-end sensing device 20 and each resource pool 11, a resource pool 11 is selected to be logically bound to the front-end sensing device 20, so that the front-end sensing device 20 can access the resource pool 11 as close as possible, and thus the output data can be loaded and distributed as close as possible.

[0028] In a specific implementation scenario, as a possible example, as mentioned above, the topology configuration file may include configuration information such as the name of each resource pool 11, then the relative distance between the front-end perception device 20 and each resource pool 11 can be determined based on the physical location of the front-end perception device 20 and the configuration information of each resource pool 11. Exemplarily, taking the physical location of the front-end perception device 20 as "City A, District B, Intersection C" as an example, the resource pools 11 defined in the topology configuration file include: resource pool 11 with pool_name "City A", resource pool 11 with pool_name "City D", and resource pool 11 with pool_name "City E", then it can be determined that the relative distance between the front-end perception device 20 and the resource pool 11 with pool_name "City A" is the shortest. Of course, the above example is only a possible example of determining the relative distance, and other possible methods are not limited here.

[0029] In a specific implementation scenario, after obtaining the relative distances between the front-end sensing device 20 and each resource pool 11 , the resource pool 11 corresponding to the shortest relative distance can be selected and logically bound to the front-end sensing device 20 .

[0030] In a specific implementation scenario, after the front-end sensing device 20 successfully accesses the resource pool 11, the device identifier of the front-end sensing device 20 used to uniquely identify itself can be logically bound to the resource pool 11. Exemplarily, the device identifier of the front-end sensing device 20 can be logically bound to the pool_id of the resource pool 11.

[0031] In one implementation scenario, the storage management platform 13 is also used to select the data node 111 as the first target node (not shown) for the front-end perception device 20 to transparently transmit data based on the load balancing of each data node 111 in the resource pool after the front-end perception device 20 is logically bound to the resource pool 11. In other words, the first target node is essentially still the data node 111 in the resource pool 11 that is logically bound to the front-end perception device 20, but it is not any data node 111 in the resource pool 11, but the data node 111 that transparently transmits data is preferred. In the above manner, after logical binding, the storage management platform 13 selects the data node 111 that transparently transmits data as the first target node for the front-end perception device 20 based on load balancing, and can achieve load balancing in each resource pool 11 as much as possible through the same set of management platforms.

[0032] In a specific implementation scenario, please continue to refer to Figure 1, the data node 111 may contain a streaming media access service, and the streaming media access service in the first target node may be responsible for accessing the front-end sensing device 20. It should be noted that many "services" in this application, such as streaming media access service, streaming media forwarding service, streaming media storage service, image gateway service, distributed cloud storage service, etc., are essentially program modules for implementing different functions. Exemplarily, the streaming media access service may define how to implement streaming media access, such as related protocols, signaling transmission, etc., which are not limited here. Other "services" can be deduced by analogy, and examples are not given one by one here. In addition, as a possible example, the above-mentioned streaming media related services may specifically be PaaS streaming media related services, such as the streaming media access service may specifically be a PaaS streaming media access service, the streaming media forwarding service may specifically be a PaaS streaming media forwarding service, and the streaming media storage service may specifically be a PaaS streaming media storage service.

[0033] In a specific implementation scenario, as mentioned above, the storage management platform 13 can be used for load balancing and scheduling management. Specifically, after the front-end perception device 20 is logically bound to the resource pool 11, the storage management platform 13 can determine the various data nodes 111 in the resource pool 11 logically bound to the front-end perception device 20 based on the topology configuration file, and obtain the load conditions of these data nodes 111, so as to select the data nodes 111 that preferentially transparently transmit out data for the front-end perception device 20 based on the purpose of load balancing. Exemplarily, taking the resource pool 11 logically bound to the front-end perception device 20 as an example, the three data nodes 111 can be named "node 01", "node 02" and "node 03" respectively for easy distinction. Since the loads of "node 02" and "node 03" are comparable and both higher than "node 01", "node 01" can be selected as the first target node for the front-end perception device 20 to preferentially transparently transmit out data. Of course, the above example is only a possible example in the actual application process, and does not limit other possible situations, and no one example is given here.

[0034] In one implementation scenario, the storage management platform 13 is also used to select a data node 111 other than the first target node as the second target node (not shown) for transparently transmitting the current output data of the front-end sensing device 20 based on the load balancing of each data node 111 in the resource pool 11 logically bound to the front-end sensing device 20 when the current load of the first target node corresponding to the front-end sensing device 20 does not meet the preset conditions. It should be noted that, as mentioned above, the first target node is the data node 111 that is determined to be the data node 111 that is preferentially transparently transmitted after the front-end sensing device 20 and the resource pool 11 are logically bound, and the first target node is located in the resource pool 11 logically bound to the front-end sensing device 20. For details, please refer to the above-mentioned related description, which will not be repeated here. In the above method, when the current load of the first target node does not meet the preset conditions, the storage management platform 13 selects a data node 111 other than the first target node as the second target node for the front-end perception device 20 to transparently transmit the current output data based on the load balancing of each data node 111 in the resource pool logically bound to the front-end perception device 20, so as to ensure the load balancing within the pool as much as possible during the storage process within each resource pool 11.

[0035] In a specific implementation scenario, the preset condition may include that the current load of the first target node is unbalanced relative to other data nodes 111 in the resource pool 11. Specifically, the preset condition may include that the current load of the first target node is relatively high in the resource pool 11. It should be noted that the specific degree of "relatively high" can be set according to actual application needs. Exemplarily, "relatively high" can be set to at least 10% higher, at least 20% higher, etc., and the specific degree of "relatively high" is not limited here.

[0036] In a specific implementation scenario, for ease of description, the resource pool 11 logically bound to the front-end sensing device 20 still contains three data nodes 111 as an example. When "node 01" is used as the first target node, if the current load of "node 01" in the resource pool 11 is relatively high when the output data of the front-end sensing device 20 is transparently transmitted, one of "node 02" and "node 03" with a relatively low load can be selected as the second target node to achieve load balancing as much as possible in the resource pool 11. Of course, the above example is only a possible example in the actual application process, and other situations can be deduced by analogy, and no examples are given here one by one.

[0037] In one implementation scenario, when the service request represents the intra-pool forwarding of the video playback request, the service management platform 12 can specifically forward the video playback request to the first target pool (not shown) based on the first request identifier in the video playback request, and the first target pool returns the target video corresponding to the video playback request in the first target pool in response to the video playback request. It should be noted that the first request identifier can include the first device representation, the first pool identifier and the video acquisition period, the first device identifier is used to uniquely represent the front-end perception device 20, the first pool identifier is used to uniquely represent the resource pool 11, the first target pool is the resource pool 11 represented by the first pool identifier, and the resource pool 11 uniquely represented by the first pool identifier is logically bound to the front-end perception device 20 uniquely represented by the first device identifier. In the above manner, when the service request is the intra-pool forwarding representing the video playback request, the service management platform 12 forwards the video playback request to the corresponding first target pool according to the first request identifier in the video playback request, so that the first target pool returns the requested target video accordingly, thereby realizing the intra-pool forwarding of the video data.

[0038] In a specific implementation scenario, the first request identifier includes the device identifier of the front-end perception device 20 that shot the requested video, the pool identifier of the resource pool 11 logically bound to the front-end perception device 20 (such as the aforementioned pool_id), and the video acquisition period of the requested playback video (such as, from D o'clock in the morning of C o'clock on B month of A year to E o'clock in the morning of C o'clock on B month of A year).

[0039] In a specific implementation scenario, please continue to refer to Figure 1 , the data node 111 may at least include a streaming media forwarding service, the business management platform 12 may forward the video playback request to the streaming media forwarding service of each data node 111 in the first target pool, and the streaming media forwarding service may query and return the target video corresponding to the video playback request. Exemplarily, the streaming media forwarding service may determine whether the output data of the front-end sensing device 20 corresponding to the first device identifier is stored in this node based on the aforementioned first device identifier. If so, it continues to determine whether there is video data in the aforementioned video acquisition period in these data. If so, it may be returned by the streaming media forwarding service as the target video. In the above manner, the data node 111 at least includes a streaming media forwarding service, and the business management platform 12 forwards the video playback request to the streaming media forwarding service of each data node 111 in the first target pool. The streaming media forwarding service is used to query and return the target video corresponding to the video playback request, and the in-pool forwarding of video data can be realized through the streaming media forwarding service.

[0040] In one implementation scenario, when the service request is an intra-pool forwarding representing an image access request, the service management platform 12 can be specifically used to forward the image access request to a second target pool (not shown) based on the second request identifier in the image access request, and the second target pool returns the corresponding target image in response to the image access request, and the second request identifier at least includes a second device identifier, and the second device identifier is used to uniquely represent the front-end perception device 20, and the second target pool is a resource pool logically bound to the front-end perception device 20 uniquely represented by the second device identifier. In the above manner, when the service request is an intra-pool forwarding representing an image access request, the service management platform 12 forwards it to the corresponding second target pool according to the second request identifier in the image access request, and the second target pool returns the corresponding target image, thereby realizing the intra-pool forwarding of image data.

[0041] In a specific implementation scenario, the second request identifier may at least include the device identifier of the front-end perception device 20 that captured the requested image. Exemplarily, the device identifier may specifically include channel information, such as being represented in the format of "city-district-road". For example, the device identifier of the front-end perception device 20 that captured the requested image may specifically be "the intersection of Road C and Road D in District B, City A". Of course, the above example is only a possible example of the device identifier of the front-end perception device 20, and does not limit the specific content of the device identifier. In addition, the device identifier may also be represented by a number such as that shown in the aforementioned pool_name, and the specific encoding method of the device identifier is not limited here.

[0042] In a specific implementation scenario, please continue to refer to Figure 1 , the data node 111 may at least include an image gateway service, the business management platform 12 may forward the image access request to the image gateway service of each data node 111 in the second target pool, and the image gateway service returns to the business management platform 12 after successfully downloading the target image in the second target pool. Exemplarily, after the image gateway service successfully downloads the target image in the second target pool, it may specifically return the URL (Uniform Resource Locator) of the target image to the business management platform 12. In the above manner, the image gateway service forwards the image access request to the second target pool, and then returns to the business management platform 12 after successfully downloading the target image, so that the image data can be forwarded within the pool through the image gateway service.

[0043] In a specific implementation scenario, the storage management platform 13 may also contain a mapping configuration file, which defines a mapping relationship of a logical binding between the front-end sensing device 20 and the resource pool 11. The storage management platform 13 may query the mapping configuration file based on the second device identifier to determine the second target pool. As a possible example, the storage management platform 13 may specifically contain a PaaS management service and a cloud storage management service. Specifically, the PaaS management service may find the second target pool based on the second device identifier and the mapping configuration file, and then forward the image retrieval request to the image gateway service in the second target pool for response. The image gateway service returns the request to the business management platform 12 after successfully downloading the target image in the second target pool. The above method can improve the accuracy of determining the second target pool by finding the second target pool through the mapping configuration file.

[0044] It should be noted that, as mentioned above, the entire streaming media service process, from device access, streaming media storage service calling the underlying cloud storage interface storage, to video data, image data and other data reading, is carried by the data node 111 inside the resource pool 11, that is, all is completed within the pool, and there is no traffic interaction between pools, which can significantly reduce the bandwidth pressure of the streaming media service on the core backbone network.

[0045] In an implementation scenario, please continue to refer to Figure 1 , the data node 111 may also include a streaming media storage service. When the streaming media storage service calls the cloud storage interface to write data, the storage management platform 13 may select the data node 111 where the cloud storage interface that issues the data write request is located as the third target node (not shown), and apply for and allocate storage space in the resource pool 11 where the third target node is located. The cloud storage client then slices the data to be written and stores each fragment to be written in each storage space. In the above manner, the streaming media storage service calls the cloud storage interface to write data inside the resource pool 11, which can realize self-completeness inside the resource pool 11, and then realize persistent storage of data generated by the front-end sensing device 20.

[0046] In a specific implementation scenario, as a possible example, the cloud storage interface may be an SDK interface exposed by the cloud storage client. It should be noted that the SDK interface may define protocols such as: data writing related protocols, data query related protocols, etc., and the specific content of the SDK interface is not limited here.

[0047] In a specific implementation scenario, as a possible example, the data write request may include a health care node address, and the node address may specifically be an IP address of a third target node.

[0048] In a specific implementation scenario, as mentioned above, the storage management platform 13 may include PaaS management services and cloud storage management services, and the cloud storage management may load and identify the aforementioned topology configuration file, and make unified load and space allocation for the data nodes 111 in the resource pool 11. When the streaming media storage service calls the cloud storage interface to write data, the cloud storage management service may apply for and allocate storage space in the resource pool 11 where the data node 111 is located, based on the IP address of the data node 111 where the cloud storage interface of the data write request is located, and the cloud storage client then stores the data slices to be written to the data nodes 111 in the pool respectively, so as to realize the slice storage in the business data pool, and the storage traffic does not cross the pool, which helps to reduce the bandwidth pressure of the core backbone network as much as possible.

[0049] In one implementation scenario, the storage management platform 13 is also used to respond to a failure of any data node 111, select the failed data node 111 as the fourth target node (not shown), and based on the fifth target node in the resource pool 11 where the fourth target node is located, perform pool content disaster recovery for each front-end perception device 20 that preferentially transmits data to the fourth target node, and the fifth target node is the currently normal data node 111. In other words, when a node failure occurs in the resource pool 11, the original logical binding between the front-end perception device 20 and the resource pool 11 is not changed, and the streaming media service pressure of the failed node is shared by other normal data nodes 111 in the resource pool 11, and cross-pool disaster recovery is performed. In the above manner, when a node failure occurs, the storage management platform 13 is used to implement pool content disaster recovery inside the resource pool 11 where the failed node is located, which can avoid inter-pool traffic interaction as much as possible, and help reduce the bandwidth pressure of the core trunk network. It should be noted that, unlike the pool content disaster of this application, if a node failure occurs in the multi-domain mode, the business network is transmitted across domains, resulting in increased pressure on the core trunk network.

[0050] In a specific implementation scenario, still taking the resource pool 11 containing three data nodes 111 as an example, after the storage management platform 13 finds that "node 01" in the resource pool 11 fails, it can be used as the fourth target node, and the fifth target node is selected from the currently normal "node 02" and "node 03" in the resource pool 11 where it is located, so as to change the front-end perception device 20 that originally transparently transmits data to the fourth target node (i.e., "node 01") to transparently transmit data to the fifth target node. It should be noted that the present application does not limit the specific number of the fifth target node. Exemplarily, still taking the above example as an example, if there are two front-end perception devices 20 that preferentially transparently transmit data to the fourth target node, and the loads of the currently normal "node 02" and "node 03" in the resource pool 11 are equivalent, then "node 02" can be selected as one of the front-end perception devices 20 to perform pool content disaster, and "node 03" can be selected as the other front-end perception device 20 to perform pool content disaster; or, if there are two front-end perception devices 20 that preferentially transparently transmit data to the fourth target node, and the load of the currently normal "node 02" and "node 03" in the resource pool 11 is significantly higher than that of the latter, then "node 03" can be selected as the two front-end perception devices 20 to perform pool content disaster. Of course, the above examples are only several possible examples of pool content disasters in actual applications, and other situations can be deduced by analogy, and no examples are given one by one here.

[0051] In a specific implementation scenario, after the abnormality is recovered, the in-pool migration of the streaming media service can also be triggered. Since it is limited to the inside of the resource pool 11, cross-pool transmission of the service traffic will not occur, which helps to reduce the bandwidth pressure of the core backbone network as much as possible.

[0052] In a specific implementation scenario, the storage management platform 13 may determine the resource pool 11 where the fourth target node is located based on the topology configuration file. Figure 2 For example, the storage management platform 13 may first obtain the IP address of the fourth target node, such as 192.168.2.10, and then Figure 2 The topology configuration file shown can determine that the resource pool 11 where the fourth target node is located is the resource pool 11 with "pool_name" being "BeijingPool". Of course, the above example is only a possible example of the actual application process, and other situations can be deduced by analogy, and no examples are given here one by one.

[0053] In one implementation scenario, the storage management platform 13 is also used to respond to a failure of any data node 111, select the failed data node 111 as the fourth target node (not shown), and select the data node 111 as the sixth target node (not shown) based on the current load among the currently normal data nodes 111 in the resource pool 11 where the fourth target node is located, and perform data recovery tasks for the fourth target node at the sixth target node. In the above manner, the storage management platform 13 selects the data node 111 as the failed fourth target node based on the current load to perform data recovery tasks, and can achieve load balancing as much as possible under the premise of achieving data recovery within the resource pool 11. It should be noted that, unlike the in-pool recovery of the present application, in the multi-domain mode, if a node failure occurs, the use of global recovery will result in cross-domain transmission of business traffic, and the bandwidth pressure of the core trunk network will be greatly increased.

[0054] In a specific implementation scenario, similar to the aforementioned pool content disaster, the storage management platform 13 can determine the resource pool 11 where the fourth target node is located based on the topology configuration file. For details, please refer to the aforementioned related description, which will not be repeated here.

[0055] In a specific implementation scenario, when a failure occurs in the pool, the IP address of the failed node (i.e., the fourth target node) and Figure 2 The topology configuration file shown finds the resource pool 11 where it is located, and then preferentially selects the target recovery node (i.e., the sixth target node) in the resource pool 11, and uses recovery algorithms such as EC (Erasure Coding) to perform data recovery tasks, thereby realizing abnormal data recovery. Moreover, since the recovery within the pool does not require cross-pool transmission, the bandwidth pressure of the core backbone network can be reduced as much as possible.

[0056] In the above scheme, the cloud storage system 10 includes a plurality of resource pools 11, a business management platform 12 and a storage management platform 13. The resource pool 11 includes a plurality of data nodes 111. The resource pool 11 is used to be logically bound to the front-end sensing device 20. The output data of the front-end sensing device 20 is stored in the resource pool 11 logically bound to the front-end sensing device 20, and there is no traffic interaction between different resource pools 11. The business management platform 12 is respectively connected to each resource pool 11 for communicating at least to respond to business requests, and the business requests include at least one of intra-pool access and intra-pool forwarding. The storage management platform 13 is respectively connected to each resource pool 11 for communicating at least to perform load balancing and dispatching in each resource pool 11 based on the topology configuration file. The topology configuration file defines each resource pool 11 and the data nodes 111 contained in the resource pool 11. Therefore, on the one hand, since multiple resource pools 11 are constructed in a decentralized manner, it helps to reduce the difficulty and cost of building the cloud storage system 10 compared to centralized construction. On the other hand, since storage, access, forwarding and other services are realized in the resource pool 11, and there is no traffic interaction between the resource pools 11, it can avoid storage, business and other traffic passing through the core trunk network as much as possible, which helps to reduce the load pressure of the core trunk network. On the other hand, since multiple resource pools 11 are managed together through a set of management platforms (that is, including the business management platform 12 and the storage management platform 13), unified management can be achieved compared to the multi-domain mode. Therefore, under the premise of reducing the difficulty and cost of building the cloud storage system 10, the load pressure of the core trunk network can be reduced as much as possible, and the convenience of system management can be improved.

[0057] See also Figure 3 , Figure 3 It is a flowchart of an embodiment of the cloud storage management method of the present application. It should be noted that the process steps in the embodiment of the present disclosure can be executed by the storage management platform in the cloud storage system. In addition, the embodiment of the present disclosure focuses on the process links executed by the storage management platform. The process links or other related links participated by the storage management platform can be combined with the relevant description in the aforementioned cloud storage system embodiment, and will not be repeated here. Specifically, the embodiment of the present disclosure may include the following steps:

[0058] Step S31: Acquire a topology configuration file and establish a communication connection with each resource pool in the cloud storage system.

[0059] In the disclosed embodiment, the topology configuration file defines various resource pools and data nodes contained in the resource pools. The resource pools are used to logically bind with the front-end sensing devices. The output data of the front-end sensing devices are stored in the resource pools logically bound with the front-end sensing devices. There is no traffic interaction between different resource pools. The cloud storage system also includes a business management platform that is communicatively connected with each resource pool, and is used to receive and respond to business requests, and the business requests include at least one of intra-pool access and intra-pool forwarding. For details, please refer to the aforementioned cloud storage system embodiment, which will not be repeated here.

[0060] Step S32: Based on the topology configuration file, load balancing and scheduling management are performed in each resource pool.

[0061] In one implementation scenario, in response to the fact that the front-end sensing device has been logically bound to the resource pool, the resource pool that is logically bound to the front-end sensing device can be selected as the first target pool, and then based on the topology configuration file and the load balancing of each data node in the first target pool, a data node is selected in the first target pool as the first target node for the front-end sensing device to preferentially transmit outgoing data. For details, please refer to the aforementioned cloud storage system embodiment, which will not be repeated here.

[0062] In one implementation scenario, in response to the current load of the first target node not satisfying a preset condition, a data node other than the first target node can be selected in the first target pool based on the topology configuration file and the load balancing of each data node in the first target pool as the second target node for the front-end sensing device to transparently transmit the current output data. It should be noted that the first target pool is a resource pool that is logically bound to the front-end sensing device, and the first target node is a data node in the first target pool that preferentially transparently transmits outgoing data. For details, please refer to the aforementioned cloud storage system embodiment, which will not be repeated here.

[0063] In one implementation scenario, when the business request is forwarded within a pool representing an image retrieval request, a mapping configuration file can also be obtained. It should be noted that the mapping configuration file defines a mapping relationship of logical binding between the front-end perception device and the resource pool. On this basis, the mapping configuration file is queried based on the image retrieval request to determine the resource pool that performs logical binding with the target perception device as the second target pool, so that the business management platform forwards the image retrieval request to the second target pool, and the second target pool returns the corresponding target image in response to the image retrieval request. It should be noted that the target perception device is the front-end perception device from which the image requested by the image retrieval request comes. For details, please refer to the aforementioned cloud storage system embodiment, which will not be repeated here.

[0064] In one implementation scenario, the data node may also include a streaming media storage service. In response to the streaming media storage service calling the cloud storage interface to write data, the data node where the cloud storage interface that issued the data write request is located is selected as the third target node. Based on the topology configuration file, storage space is applied for and allocated in the resource pool where the third target node is located, so that the cloud storage client can store the various to-be-written fragments obtained by slicing the to-be-written data in various storage spaces. For details, please refer to the aforementioned cloud storage system embodiment, which will not be repeated here.

[0065] In a specific implementation scenario, the cloud storage interface is an SDK interface exposed by the cloud storage client.

[0066] In a specific implementation scenario, the data write request includes a node address, and the node address is the IP address of the third target node.

[0067] In one implementation scenario, in response to a failure of any data node, the failed data node can be selected as the fourth target node, and based on the topology configuration file, a currently normal data node is selected as the fifth target node in the resource pool where the fourth target node is located. Based on the fifth target node, each front-end sensing device that preferentially transmits data to the fourth target node performs a pool content disaster. For details, please refer to the aforementioned cloud storage system embodiment, which will not be repeated here.

[0068] In one implementation scenario, in response to a failure of any data node, the failed data node can be selected as the fourth target node, and based on the topology configuration file, the resource pool where the fourth target node is located is determined, and then among the currently normal data nodes in the determined resource pool, a data node is selected as the sixth target node based on the current load, and at the sixth target node, a data recovery task is performed for the fourth target node. For details, please refer to the aforementioned cloud storage system embodiment, which will not be repeated here.

[0069] The above scheme obtains a topology configuration file and establishes a communication connection with each resource pool in the cloud storage system; wherein the topology configuration file defines each resource pool and the data nodes contained in the resource pool, the resource pool is used to be logically bound to the front-end sensing device, the output data of the front-end sensing device is stored in the resource pool logically bound to the front-end sensing device, and there is no traffic interaction between different resource pools. The cloud storage system also includes a business management platform that is communicatively connected to each resource pool, and is used to receive and respond to business requests, and the business request includes at least one of access within the pool and forwarding within the pool. Then, based on the topology configuration file, load balancing and scheduling management are performed in each resource pool. Therefore, on the one hand, due to the decentralized construction of multiple resource pools, it is helpful to reduce the difficulty and cost of cloud storage system construction compared with centralized construction. On the other hand, due to the realization of storage, access, forwarding and other services in the resource pool, and there is no traffic interaction between resource pools, it is possible to avoid storage, business and other traffic passing through the core trunk network as much as possible, which helps to reduce the load pressure of the core trunk network. On the other hand, due to the common management of multiple resource pools through a set of management platforms (i.e., including a business management platform and a storage management platform), unified management can be achieved compared with the multi-domain mode. Therefore, it is possible to reduce the load pressure on the core backbone network as much as possible and improve the convenience of system management while reducing the difficulty and cost of building a cloud storage system.

[0070] See also Figure 4 , Figure 4 It is a flowchart of another embodiment of the cloud storage management method of the present application. It should be noted that the process steps in the embodiment of the present disclosure are executed by the business management platform in the cloud storage system. In addition, the embodiment of the present disclosure focuses on the process links executed by the business management platform. The process links or other related links participated by the business management platform can be combined with the relevant description in the aforementioned cloud storage system embodiment, and will not be repeated here. Specifically, the embodiment of the present disclosure may include the following steps:

[0071] Step S41: Establishing communication connections with various resource pools in the cloud storage system.

[0072] In the disclosed embodiment, the resource pool includes several data nodes, the resource pool is used to logically bind with the front-end sensing device, the output data of the front-end sensing device is stored in the resource pool logically bound with the front-end sensing device, there is no traffic interaction between different resource pools, and the cloud storage system also includes a storage management platform that is connected to each resource pool in communication, and is used to perform load balancing and scheduling management in each resource pool based on a topology configuration file, and the topology configuration file defines each resource pool and the data nodes contained in the resource pool. For details, please refer to the aforementioned cloud storage system embodiment, which will not be repeated here.

[0073] Step S42: Receive and respond to the service request.

[0074] In the embodiment of the present disclosure, the service request includes at least one of in-pool access and in-pool forwarding. For details, please refer to the aforementioned cloud storage system embodiment, which will not be described in detail here.

[0075] In one implementation scenario, in response to a service request for in-pool access, the relative distances between the front-end sensing device and each resource pool can be obtained based on the physical location of the front-end sensing device, and then based on the relative distances between the front-end sensing device and each resource pool, a resource pool is selected to be logically bound to the front-end sensing device. For details, please refer to the aforementioned cloud storage system embodiment, which will not be repeated here.

[0076] In one implementation scenario, in response to a business request for intra-pool forwarding representing a video playback request, based on the first request identifier in the video playback request, the video playback request is forwarded to the first target pool, so that the first target pool returns the target video corresponding to the video playback request in the first target pool in response to the video playback request. It should be noted that the first request identifier includes a first device identifier, a first pool identifier, and a video acquisition period, the first device identifier is used to uniquely represent a front-end perception device, the first pool identifier is used to uniquely represent a resource pool, the first target pool is a resource pool uniquely represented by the first pool identifier, and the resource pool uniquely represented by the first pool identifier is logically bound to the front-end perception device uniquely represented by the first device identifier. For details, please refer to the aforementioned cloud storage system embodiment, which will not be repeated here.

[0077] In one implementation scenario, in response to a business request for an intra-pool forwarding representing an image retrieval request, the image retrieval request is forwarded to a second target pool based on a second request identifier in the image retrieval request, so that the second target pool returns the corresponding target image in response to the image retrieval request. It should be noted that the second request identifier includes at least a second device identifier, which is used to uniquely represent a front-end sensing device, and the second target pool is a resource pool logically bound to the front-end sensing device uniquely represented by the second device identifier. For details, please refer to the aforementioned cloud storage system embodiment, which will not be repeated here.

[0078] The above scheme establishes a communication connection with each resource pool in the cloud storage system. The resource pool includes a number of data nodes. The resource pool is used to logically bind with the front-end sensing device. The output data of the front-end sensing device is stored in the resource pool logically bound with the front-end sensing device. There is no traffic interaction between different resource pools. The cloud storage system also includes a storage management platform that is communicatively connected with each resource pool, and is used to perform load balancing and scheduling management in each resource pool based on a topology configuration file. The topology configuration file defines each resource pool and the data nodes contained in the resource pool, and then receives and responds to service requests, and the service request includes at least one of access within the pool and forwarding within the pool. Therefore, on the one hand, due to the decentralized construction of multiple resource pools, it is helpful to reduce the difficulty and cost of cloud storage system construction compared with centralized construction. On the other hand, since storage, access, forwarding and other services are realized in the resource pool, and there is no traffic interaction between resource pools, it can avoid storage, business and other traffic passing through the core trunk network as much as possible, which helps to reduce the load pressure of the core trunk network. On the other hand, since multiple resource pools are managed together through a set of management platforms (i.e., including a business management platform and a storage management platform), unified management can be achieved compared with the multi-domain mode. Therefore, it is possible to reduce the load pressure on the core backbone network as much as possible and improve the convenience of system management while reducing the difficulty and cost of building a cloud storage system.

[0079] See also Figure 5 , Figure 5 1 is a schematic diagram of a framework of an embodiment of an electronic device. The electronic device 50 at least includes a communication circuit 51, a memory 52 and a processor 53. The communication circuit 51 is coupled to the memory 52 and the processor 53 respectively. The memory 52 stores program instructions. The processor 53 is used to execute the program instructions to implement the steps in the aforementioned cloud storage management method embodiment. Specifically, the electronic device 50 may include but is not limited to a server, etc., which is not limited here.

[0080] Specifically, the processor 53 may also be referred to as a CPU (Central Processing Unit). The processor 53 may be an integrated circuit chip having the ability to process signals. The processor 53 may also be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. In addition, the processor 53 may be implemented by a plurality of integrated circuit chips.

[0081] In the above scheme, the electronic device 50 implements the steps in any of the above cloud storage management method embodiments. On the one hand, since multiple resource pools are constructed in a decentralized manner, it helps to reduce the difficulty and cost of cloud storage system construction compared to centralized construction. On the other hand, since storage, access, forwarding and other services are implemented in the resource pool, and there is no traffic interaction between resource pools, it can avoid storage, business and other traffic passing through the core trunk network as much as possible, which helps to reduce the load pressure of the core trunk network. On the other hand, since multiple resource pools are managed together through a set of management platforms (i.e., including business management platforms and storage management platforms), unified management can be achieved compared to the multi-domain mode. Therefore, under the premise of reducing the difficulty and cost of cloud storage system construction, the load pressure of the core trunk network can be reduced as much as possible, and the convenience of system management can be improved.

[0082] See also Figure 6 , Figure 6 The computer-readable storage medium 60 stores program instructions 61 executable by a processor, and the program instructions 61 can be executed to implement the steps in any of the above-mentioned cloud storage management method embodiments.

[0083] In the above scheme, the computer-readable storage medium 60 implements the steps in any of the above cloud storage management method embodiments. On the one hand, since multiple resource pools are constructed in a decentralized manner, it helps to reduce the difficulty and cost of cloud storage system construction compared to centralized construction. On the other hand, since storage, access, forwarding and other services are implemented in the resource pool, and there is no traffic interaction between resource pools, it can avoid storage, business and other traffic passing through the core trunk network as much as possible, which helps to reduce the load pressure of the core trunk network. On the other hand, since multiple resource pools are managed together through a set of management platforms (i.e., including business management platforms and storage management platforms), unified management can be achieved compared to the multi-domain mode. Therefore, under the premise of reducing the difficulty and cost of cloud storage system construction, the load pressure of the core trunk network can be reduced as much as possible, and the convenience of system management can be improved.

[0084] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0085] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0086] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0087] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0088] If the technical solution of this application involves personal information, the product using the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using the technical solution of this application has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to the collection of his or her personal information; or on the device that processes personal information, the personal information processing rules are notified by obvious signs / information, and the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

Claims

1. A cloud storage system, It is characterized in that include: A plurality of resource pools, each resource pool comprising a plurality of data nodes, the resource pool being used to be logically bound to a front-end sensing device, the output data of the front-end sensing device being stored in the resource pool logically bound to the front-end sensing device, the output data being preferentially transparently transmitted to a first target node predetermined in the resource pool logically bound to the front-end sensing device, and no traffic interaction between different resource pools, the data nodes in the same resource pool being located at the same geographical location, and the data nodes belonging to different resource pools being located at different geographical locations; A service management platform is respectively connected to each of the resource pools for communication, and is at least used to respond to service requests, wherein the service requests include at least one of intra-pool access and intra-pool forwarding. The service management platform is specifically used to respond to the service request for intra-pool access, obtain the relative distance between the front-end sensing device and each of the resource pools based on the physical location of the front-end sensing device, and select one of the resource pools to logically bind with the front-end sensing device based on the relative distance between the front-end sensing device and each of the resource pools; The storage management platform is respectively connected to the respective resource pools for at least performing load balancing and scheduling management in each resource pool based on a topology configuration file, wherein the topology configuration file defines the respective resource pools and the data nodes contained in the resource pools.

2. The system according to claim 1, It is characterized in that The storage management platform is also used to select the data node as the first target node for the front-end perception device to preferentially transmit the output data based on the load balancing of each data node in the resource pool after the front-end perception device and the resource pool perform the logical binding.

3. The system according to claim 1, It is characterized in that The storage management platform is also used to select a data node other than the first target node as a second target node for transparently transmitting the current output data of the front-end perception device based on the load balancing of each data node in the resource pool logically bound to the front-end perception device when the current load of the first target node corresponding to the front-end perception device does not meet the preset condition; Among them, the first target node is the data node that is determined to preferentially transmit the output data after the front-end perception device and the resource pool perform the logical binding, and the first target node is located in the resource pool that is logically bound to the front-end perception device.

4. The system according to claim 1, It is characterized in that When the service request is forwarded within a pool representing a video playback request, the service management platform is specifically used to forward the video playback request to a first target pool based on a first request identifier in the video playback request, and the first target pool returns a target video corresponding to the video playback request in the first target pool in response to the video playback request; Among them, the first request identifier includes a first device identifier, a first pool identifier and a video acquisition period, the first device identifier is used to uniquely represent the front-end perception device, the first pool identifier is used to uniquely represent the resource pool, the first target pool is the resource pool uniquely represented by the first pool identifier, and the resource pool uniquely represented by the first pool identifier is logically bound to the front-end perception device uniquely represented by the first device identifier.

5. The system according to claim 1, It is characterized in that When the service request is an intra-pool forwarding representing an image access request, the service management platform is specifically configured to forward the image access request to a second target pool based on a second request identifier in the image access request, and the second target pool returns a corresponding target image in response to the image access request; Among them, the second request identifier at least includes a second device identifier, and the second device identifier is used to uniquely identify the front-end perception device, and the second target pool is a resource pool logically bound to the front-end perception device uniquely identified by the second device identifier.

6. The system according to claim 1, It is characterized in that The data node also includes a streaming media storage service. When the streaming media storage service calls a cloud storage interface to write data, the storage management platform selects the data node where the cloud storage interface that issues the data write request is located as the third target node, and applies for and allocates storage space in the resource pool where the third target node is located. The cloud storage client then stores each to-be-written fragment obtained by slicing the to-be-written data in each of the storage spaces.

7. The system according to claim 1, It is characterized in that The storage management platform is further configured to, in response to a failure of any of the data nodes, select the failed data node as a fourth target node, and perform at least one of the following: Based on a fifth target node in the resource pool where the fourth target node is located, performing pool content disaster prevention for each of the front-end sensing devices that preferentially transparently transmit the output data to the fourth target node, and the fifth target node is a currently normal data node; Among the currently normal data nodes in the resource pool where the fourth target node is located, the data node is selected as the sixth target node based on the current load, and the data recovery task is performed for the fourth target node at the sixth target node.

8. A cloud storage management method, It is characterized in that A storage management platform applied to the cloud storage system according to any one of claims 1 to 7, comprising: Obtain a topology configuration file, and establish a communication connection with each resource pool in the cloud storage system; wherein the topology configuration file defines each resource pool and the data nodes contained in the resource pool, the resource pool is used to be logically bound to the front-end sensing device, the output data of the front-end sensing device is stored in the resource pool logically bound to the front-end sensing device, and there is no traffic interaction between different resource pools, the cloud storage system also includes a business management platform that is communicatively connected to each resource pool, and is used to receive and respond to business requests, and the business requests include at least one of intra-pool access and intra-pool forwarding; Based on the topology configuration file, load balancing and scheduling management are performed in each resource pool.

9. The method according to claim 8, It is characterized in that The method further comprises: In response to the front-end sensing device and the resource pool having performed the logical binding, selecting the resource pool that performs the logical binding with the front-end sensing device as the first target pool; Based on the topology configuration file and the load balancing of each of the data nodes in the first target pool, the data node is selected in the first target pool as the first target node to which the front-end perception device preferentially transparently transmits the output data.

10. The method according to claim 8, It is characterized in that The method further comprises: In response to the current load of the first target node not satisfying a preset condition, based on the topology configuration file and the load balancing of each of the data nodes in the first target pool, selecting a data node other than the first target node in the first target pool as a second target node for the front-end sensing device to transparently transmit the current output data; Among them, the first target pool is a resource pool that executes the logical binding with the front-end sensing device, and the first target node is a data node in the first target pool that preferentially transmits the output data.

11. The method according to claim 8, It is characterized in that When the service request is an intra-pool forwarding representing an image access request, the method further includes: Obtain a mapping configuration file; wherein the mapping configuration file defines a mapping relationship of the logical binding between the front-end sensing device and the resource pool; Based on the image retrieval request, a query is performed in the mapping configuration file to determine the resource pool that executes the logical binding with the target perception device, as the second target pool, so that the business management platform forwards the image retrieval request to the second target pool, and the second target pool returns the corresponding target image in response to the image retrieval request; wherein the target perception device is the front-end perception device from which the image requested by the image retrieval request comes.

12. The method according to claim 8, It is characterized in that The data node also includes a streaming storage service, and the method further includes: In response to the streaming media storage service calling the cloud storage interface to write data, selecting a data node where the cloud storage interface that issues the data write request is located as a third target node; Based on the topology configuration file, storage space is applied for and allocated in the resource pool where the third target node is located, so that the cloud storage client can store the various to-be-written fragments obtained by slicing the to-be-written data into each of the storage spaces respectively.

13. The method according to claim 8, It is characterized in that The method further comprises: In response to a failure of any of the data nodes, selecting the failed data node as the fourth target node, and performing at least one of the following: Based on the topology configuration file, in the resource pool where the fourth target node is located, a currently normal data node is selected as a fifth target node, and based on the fifth target node, pool content disaster prevention is performed for each of the front-end sensing devices that preferentially transparently transmit the output data to the fourth target node; Based on the topology configuration file, the resource pool where the fourth target node is located is determined, and among the currently normal data nodes in the determined resource pool, the data node is selected as the sixth target node based on the current load, and at the sixth target node, a data recovery task is performed for the fourth target node.

14. A cloud storage management method, It is characterized in that A service management platform applied to a cloud storage system as claimed in any one of claims 1 to 7, comprising: Establishing a communication connection with each resource pool in the cloud storage system; wherein the resource pool includes a plurality of data nodes, the resource pool is used to be logically bound to the front-end sensing device, the output data of the front-end sensing device is stored in the resource pool logically bound to the front-end sensing device, there is no traffic interaction between different resource pools, and the cloud storage system also includes a storage management platform that is communicatively connected to each resource pool, and is used to perform load balancing and scheduling management in each resource pool based on a topology configuration file, wherein the topology configuration file defines each resource pool and the data nodes contained in the resource pool; Receive and respond to a service request; wherein the service request includes at least one of intra-pool access and intra-pool forwarding.

15. The method according to claim 14, It is characterized in that The method further comprises: In response to the service request being an intra-pool forwarding representing a video playback request, based on a first request identifier in the video playback request, forwarding the video playback request to a first target pool, so that the first target pool returns a target video corresponding to the video playback request in the first target pool in response to the video playback request; Among them, the first request identifier includes a first device identifier, a first pool identifier and a video acquisition period, the first device identifier is used to uniquely represent the front-end perception device, the first pool identifier is used to uniquely represent the resource pool, the first target pool is the resource pool uniquely represented by the first pool identifier, and the resource pool uniquely represented by the first pool identifier is logically bound to the front-end perception device uniquely represented by the first device identifier.

16. The method according to claim 14, It is characterized in that The method further comprises: In response to the service request being an intra-pool forwarding representing an image access request, forwarding the image access request to a second target pool based on a second request identifier in the image access request, so that the second target pool returns a corresponding target image in response to the image access request; Among them, the second request identifier at least includes a second device identifier, and the second device identifier is used to uniquely identify the front-end perception device, and the second target pool is a resource pool logically bound to the front-end perception device uniquely identified by the second device identifier.

Citation Information

Patent Citations

  • Data writing method and apparatus, data reading method and apparatus, and cloud storage system

    CN107092437A

  • Front-end equipment access method and device, server and storage medium

    CN107800694A

  • Data storing, reading and cleaning methods and devices and cloud storage system

    CN107967117A

  • Cloud storage system, media data storage method and system

    CN108737476A

  • Multi-data-center online management system and management method based on Internet of Things

    CN110798362A