Data writing method and device for monitoring scenario, storage cluster and medium

Abstract

This invention provides a data writing method, apparatus, storage cluster, and medium for monitoring scenarios, applied in the field of data storage technology. Specifically, the method is applied to a storage cluster, which includes a first data storage pool for storing data prior to changes in data protection policies. The method includes: upon receiving a write request from the business layer, detecting whether a second data storage pool exists in the storage cluster; if a second data storage pool exists, writing the data to be written to the second data storage pool according to the write request; the second data storage pool is used to store data after the data protection policy change, and the write request includes the data to be written, which includes data from the next period after the current monitoring period has completed its full storage or data that has not been fully stored within the current monitoring period. This invention supports online continuous changes to data protection policies without requiring data migration, effectively reducing resource consumption and not affecting upper-layer real-time services.

Description

Technical Field

[0001] This invention relates to the field of data storage technology, and in particular to a data writing method, apparatus, storage cluster, and medium for monitoring scenarios. Background Technology

[0002] In monitoring scenarios, if the actual situation changes, the protection priority of the original data in the monitoring system needs to be adjusted, which means changing the data protection strategy (switching the strategy pool). For data whose importance increases, erasure replication protection needs to be strengthened, while for data whose importance decreases, data protection needs to be reduced. Switching data protection strategies involves extremely heavy data migration work and has a significant impact on upper-layer real-time services.

[0003] When data protection policies need to be changed in real-world scenarios, existing technologies add a data protection pool to the storage cluster. The storage cluster then migrates old data from the old data pool to the data protection pool. Simultaneously, the business layer requires additional tools to read and write new / old data. This method struggles to achieve online changes to data protection policies. When switching policy pools, the business layer needs to reconfigure, which impacts data, does not support continuous changes, and data migration consumes significant resources while affecting upper-layer real-time services. Summary of the Invention

[0004] This invention provides a data writing method, apparatus, storage cluster, and medium for monitoring scenarios, which addresses the shortcomings of existing technologies where data migration requires significant resources and can impact upper-layer real-time services. It enables the change of erasure replication policies at the business layer; that is, when the data protection policy changes, the business layer data is stored in a new second data storage pool. Throughout the process, the storage cluster supports continuous online changes to the data protection policy without requiring data migration, effectively reducing resource consumption and avoiding impact on upper-layer real-time services.

[0005] This invention provides a data writing method for monitoring scenarios, applied to a storage cluster, wherein the storage cluster includes a first data storage pool, the first data storage pool being used to store data prior to changes in data protection policies, and the method includes:

[0006] Upon receiving a write request from the business layer, the system checks whether a second data storage pool exists in the storage cluster. The second data storage pool is used to store data after the data protection policy has been changed. The write request includes data to be written, which includes data in the next cycle after the current monitoring cycle has been fully stored or data that has not been fully stored within the current monitoring cycle.

[0007] If a second data storage pool exists, the data to be written is written to the second data storage pool according to the write request.

[0008] According to a data writing method for monitoring scenarios provided by the present invention, the write request includes a write object corresponding to the data to be written, and the writing of the data to be written to a second data storage pool according to the write request includes:

[0009] If there is an object to be written in the first data storage pool, then delete the data corresponding to the object to be written from the first data storage pool.

[0010] Create the objects to be written in the second data storage pool and establish a mapping relationship between the objects to be written in the first data storage pool and the objects to be written in the second data storage pool;

[0011] Based on the mapping relationship, the data to be written in the write request is written to the storage space corresponding to the object to be written in the second data storage pool.

[0012] According to a data writing method for monitoring scenarios provided by the present invention, the method further includes:

[0013] If the object to be written does not exist in the first data storage pool, then the object to be written will be created in the first data storage pool.

[0014] Create the objects to be written in the second data storage pool and establish a mapping relationship between the objects to be written in the first data storage pool and the objects to be written in the second data storage pool;

[0015] Based on the mapping relationship, the data to be written in the write request is written to the storage space corresponding to the object to be written in the second data storage pool.

[0016] According to a data writing method for monitoring scenarios provided by the present invention, the write request includes a write object corresponding to the data to be written, and the method further includes:

[0017] If a second data storage pool does not exist, then determine whether there is an object to be written in the first data storage pool;

[0018] If an object to be written exists, the data to be written is overwritten into the storage space corresponding to the object in the first data storage pool according to the write request.

[0019] According to a data writing method for monitoring scenarios provided by the present invention, the storage cluster further includes a metadata storage pool, the metadata storage pool storing a first command, the first command including a first identifier of the first data storage pool, and the method further includes:

[0020] If all data in the first data storage pool has been deleted, and all objects in the mapping relationship are mapped to the same second data storage pool, then the first data storage pool is released.

[0021] Obtain the second identifier of the second data storage pool and replace the first identifier in the first command with the second identifier.

[0022] According to a data writing method for monitoring scenarios provided by the present invention, the storage cluster further includes a second data storage pool and a metadata storage pool. The metadata storage pool stores a second command, which includes a first identifier of the first data storage pool. Before detecting whether the second data storage pool exists in the storage cluster, the method further includes:

[0023] Obtain the data protection policy change instruction sent by the business layer; the data protection policy change instruction includes the target protection level corresponding to the changed data protection policy.

[0024] A second data storage pool matching the target protection level is determined within the storage cluster; the aforementioned storage cluster includes data storage pools with different protection levels.

[0025] Obtain the second identifier of the second data storage pool and replace the first identifier in the second command with the second identifier.

[0026] According to a data writing method for monitoring scenarios provided by the present invention, the method further includes:

[0027] Obtain the read request sent by the business layer; the read request includes the object to be read corresponding to the data to be read.

[0028] If the data corresponding to the object to be read in the first data storage pool is not empty, then read the data stored in the storage space of the object to be read in the first data storage pool.

[0029] If the data corresponding to the object to be read in the first data storage pool is empty, then the object to be read is found in the second data storage pool according to the mapping relationship, and the data stored in the storage space of the object to be read is read in the second data storage pool.

[0030] The present invention also provides a data writing device for monitoring scenarios, applied to a storage cluster, wherein the storage cluster includes a first data storage pool, the first data storage pool being used to store data prior to changes in data protection policies, and the device includes the following modules:

[0031] The detection module is used to detect whether a second data storage pool exists in the storage cluster when a write request is received from the business layer. The second data storage pool is used to store data after the data protection policy has been changed. The write request includes data to be written, which includes data in the next cycle after the full cycle storage of the current monitoring cycle or data that has not been stored in the current monitoring cycle.

[0032] The write data module is used to write the data to be written to the second data storage pool according to the write request if a second data storage pool exists.

[0033] The present invention also provides a storage cluster, including a first data storage pool, a memory, a processor, and a computer program stored on the memory and executable on the processor. The first data storage pool is used to store data before the data protection policy is changed. When the processor executes the computer program, it implements the data writing method for monitoring scenarios as described above.

[0034] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the data writing method for monitoring scenarios as described above.

[0035] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the data writing method for monitoring scenarios as described above.

[0036] This invention provides a data writing method, apparatus, storage cluster, and medium for monitoring scenarios. Applied to a storage cluster, the storage cluster includes a first data storage pool for storing data prior to changes in data protection policies. The method, upon receiving a write request from the business layer, detects the existence of a second data storage pool within the storage cluster. If a second data storage pool exists, it writes the data to be written to the second data storage pool according to the write request. The second data storage pool stores data after the data protection policy change. The write request includes the data to be written, which includes data from the next period after the current monitoring period's full-cycle storage or data not yet stored within the current monitoring period. In this method, when the erasure replication policy changes at the business layer (i.e., the data protection policy changes), the write request from the business layer can detect the existence of a new data storage pool. If a new data storage pool exists, the changed data is stored in that new data storage pool. This supports continuous online changes to data protection policies without migrating data from the old data storage pool to the new one, effectively reducing resource consumption and without impacting upper-layer real-time services. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0038] Figure 1 This is one of the flowcharts illustrating the data writing method for monitoring scenarios provided by this invention.

[0039] Figure 2 This is the second flowchart of the data writing method for monitoring scenarios provided by the present invention.

[0040] Figure 3 This is the third flowchart of the data writing method for monitoring scenarios provided by the present invention.

[0041] Figure 4 This is one of the schematic diagrams of data storage pool changes provided by the present invention.

[0042] Figure 5 This is the second schematic diagram of the data storage pool change provided by the present invention.

[0043] Figure 6 This is the third schematic diagram of the data storage pool change provided by the present invention.

[0044] Figure 7 This is the fourth schematic diagram of the data storage pool change provided by the present invention.

[0045] Figure 8 This is the fifth schematic diagram of the data storage pool change provided by the present invention.

[0046] Figure 9 This is a schematic diagram of the data writing device for monitoring scenarios provided by the present invention.

[0047] Figure 10 This is a schematic diagram of the storage cluster structure provided by the present invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0049] Currently, in monitoring scenarios, changes in the actual environment necessitate adjustments to the protection priority of existing data. This may involve enhanced erasure and copy protection for some data, while reducing protection for others. All these situations require switching protection strategies, which is extremely labor-intensive and significantly impacts real-time operations. In related technologies, the mainstream approach for changing data protection strategies in the field involves adding a new data protection pool, migrating storage to the new pool, and using tools to ensure access to old data, monitor retention periods, and clean up data, or importing data from the old pool to the new pool. However, this technology has the following problems: online changes to data protection strategies are not possible; switching strategy pools requires reconfiguration of upper-layer services, impacting data, and it does not support continuous changes; additional tools are needed for services to read and write both old and new data, preventing upper-layer services from managing retention periods as needed; data migration consumes significant resources and impacts real-time operations. Therefore, this invention provides a data writing method, apparatus, storage cluster, and medium for monitoring scenarios, which can solve the above-mentioned technical problems.

[0050] The following is combined Figures 1-8 This invention describes a data writing method for monitoring scenarios according to an embodiment of the present invention.

[0051] It should be noted that the execution subject of the embodiments of the present invention may be a data writing device for monitoring scenarios, a storage cluster including a data writing device for monitoring scenarios, or other devices or equipment, which are not specifically limited here.

[0052] The following examples use a storage cluster as the execution subject for illustration.

[0053] Figure 1 This is one of the flowcharts illustrating the data writing method for monitoring scenarios provided by this invention, such as... Figure 1 As shown, the method includes the following steps:

[0054] S102, upon receiving a write request from the business layer, detect whether a second data storage pool exists in the storage cluster; the second data storage pool is used to store data after the data protection policy has been changed, and the write request includes data to be written, which includes data in the next cycle after the full cycle storage of the current monitoring cycle or data that has not been stored within the current monitoring cycle.

[0055] In this embodiment, the method is applied to a storage cluster, which includes a first data storage pool. The first data storage pool stores data prior to a change in the data protection policy. That is, the first data storage pool refers to the data storage pool configured by the storage cluster for the then-current data protection policy before the policy change, and can also be called the old data storage pool. The second data storage pool refers to the data storage pool newly configured by the storage cluster based on the received erasure replication policy change information when the data protection policy changes; it can also be called the new data storage pool, and can store data after the data protection policy change.

[0056] The data protection strategies mentioned above refer to erasure replication strategies, such as 6+1 erasure replication strategies, 6+2 erasure replication strategies, etc. For example, the 6+1 erasure replication strategy means storing one data copy in the data storage pool for subsequent data verification, while the 6+2 erasure replication strategy means storing two data copies in the data storage pool for subsequent data verification.

[0057] In this step, during the continuous writing / storage of monitoring data at the business layer, assuming the storage cluster becomes aware of a change in the current erasure replication policy / data protection policy, it can be determined that the protection policy for the current monitoring data needs to be changed. Generally, the current monitoring data needs to be stored in another storage pool. In this step, it is possible to detect whether there is a second data storage pool in the storage cluster and obtain the detection result.

[0058] It is understandable that this second data storage pool is a different data storage pool from the first data storage pool, and the erasure replication strategy / data protection strategy for the data stored in the two are different.

[0059] In addition, for the write request received from the business layer mentioned above, the write request may include the data to be written, and may also include the object to be written corresponding to the data to be written. The object to be written may be the identifier of the data to be written, such as a date identifier, data name, etc. For example, if the data included in the write request is data from a certain day, then the date of that day can be considered as the object to be written.

[0060] Regarding the data to be written, since this embodiment stores data from a monitoring scenario, to save storage space, monitoring data is typically stored periodically according to a storage cycle, such as days, weeks, or months. For example, if each week is a monitoring cycle, then the monitoring data of the next week will overwrite the monitoring data of the previous week. Therefore, the data to be written can include data from the new monitoring cycle after all the data from the current monitoring cycle has been stored in the first data storage pool, or it can include data from the current monitoring cycle that has not yet been stored. For example, if the monitoring cycle is weekly, then the monitoring cycle is Monday to Sunday. The data to be written could be the monitoring data from Monday of the second week after the first week, or it could be the monitoring data from Wednesday of the first week that has not yet been stored.

[0061] S104, if a second data storage pool exists, the data to be written is written to the second data storage pool according to the write request.

[0062] In this step, as mentioned above, it's possible to detect whether a second data storage pool exists in the storage cluster, and obtain the detection result. If the detection result indicates the existence of a second data storage pool in the storage cluster—that is, a data storage pool capable of storing data after the data protection policy change—then the data to be written in the write request can be written to the second data storage pool. Specifically, when writing data, if the data to be written includes data from a new monitoring period after all data from the current monitoring period has been stored in the first data storage pool, then the data to be written can be stored sequentially in the second data storage pool according to attributes such as the identifier of the new monitoring period. For example, if the first data storage pool stores data from data0 to data7 for a full coverage period, then the data to be written can continue to be written starting from data0 in the second data storage pool. If the data to be written includes data from the current monitoring period that has not been fully stored, then the data to be written can be stored sequentially in the second data storage pool starting from attributes such as the identifier of the data that has not been fully stored for the current monitoring period. For example, if the first data storage pool does not store data from a full coverage period, but rather data0 to data1, then the data to be written can continue to be written starting from data2 in the second data storage pool.

[0063] In addition, as mentioned above, the write request may also include the object to be written corresponding to the data to be written. If the detection result is that there is no second data storage pool in the storage cluster, then optionally, if there is no second data storage pool, it is determined whether there is an object to be written in the first data storage pool; if there is an object to be written, then according to the write request, the data to be written is overwritten to the storage space corresponding to the object to be written in the first data storage pool.

[0064] In other words, if a second data storage pool does not exist in the storage cluster, and data needs to be written / stored in the original first data storage pool, the process begins by determining whether the object to be written exists in that first data storage pool. This involves checking for an identifier for the data to be written (e.g., data0). If such an identifier exists, the data in the storage space corresponding to the object under the original data protection policy needs to be updated to the data after the change in the data protection policy. In other words, the data to be written can replace / overwrite the previously stored data in the storage space corresponding to the object. If the object to be written does not exist in the first data storage pool, a new object can be created directly using its attribute information, and a corresponding storage space can be allocated. The data to be written is then stored in the storage space corresponding to the object.

[0065] Since there is no second data storage pool in the storage cluster, the data after the data protection policy change can continue to be stored in the original first data storage pool. This ensures business continuity, enables continuous online changes to the data protection policy, and does not affect the normal operation of the business.

[0066] Understandably, after the data protection strategy was changed, all new data at the business layer was stored using the changed data protection strategy.

[0067] This embodiment of the data writing method for monitoring scenarios is applied to a storage cluster, which includes a first data storage pool. The first data storage pool stores data before the data protection policy changes. The method, upon receiving a write request from the business layer, detects whether a second data storage pool exists in the storage cluster. If a second data storage pool exists, it writes the data to be written to the second data storage pool according to the write request. The second data storage pool stores data after the data protection policy changes. The write request includes the data to be written, which includes data from the next period after the current monitoring period has completed its storage or data that has not been fully stored within the current monitoring period. In this method, since the erasure replication policy changes at the business layer (i.e., the data protection policy changes), the write request at the business layer can detect whether a new data storage pool exists. If a new data storage pool exists, the data after the data protection policy change is stored in the new data storage pool. This supports continuous online changes to the data protection policy without migrating data from the old data storage pool to the new one, effectively reducing resource consumption and without affecting upper-layer real-time services.

[0068] Figure 2 This is the second flowchart of the data writing method for monitoring scenarios provided by the present invention, as shown below. Figure 2As shown, the write request includes the object to be written corresponding to the data to be written. The step S104 above, "writing the data to be written to the second data storage pool according to the write request", may include the following steps:

[0069] S202, if there is an object to be written in the first data storage pool, then delete the data corresponding to the object to be written from the first data storage pool.

[0070] S204, create the object to be written in the second data storage pool, and establish a mapping relationship between the object to be written in the first data storage pool and the object to be written in the second data storage pool.

[0071] S206, according to the mapping relationship, write the data to be written in the write request to the storage space corresponding to the object to be written in the second data storage pool.

[0072] In the aforementioned scenario, when a second data storage pool exists, before writing data to the second data storage pool, it can be determined whether a corresponding object exists in the first data storage pool. Specifically, it can be checked whether an object with the same attributes as the object to be written exists in the first data storage pool. If the object exists in the first data storage pool, then its storage space also contains data. The data stored in the first data storage pool can then be deleted, resulting in an empty object. A new object can then be created in the second data storage pool based on its attributes. This newly created object is initially empty, and a mapping relationship can be established between the empty object in the first data storage pool and the corresponding object in the second data storage pool. This mapping relationship can also be called a link. Thus, if the business layer needs to access data during a data protection policy change, it can use this mapping relationship to link from the object in the first data storage pool to the corresponding object in the new data storage pool (e.g., the second data storage pool) to access the data, ensuring business continuity.

[0073] After establishing the mapping relationship, the newly transmitted data to be written from the business layer can be written to the storage space of the object to be written in the second data storage pool.

[0074] Alternatively, as an optional embodiment, if there is no object to be written in the first data storage pool, the object to be written is created in the first data storage pool; the object to be written is created in the second data storage pool, and a mapping relationship is established between the object to be written in the first data storage pool and the object to be written in the second data storage pool; according to the mapping relationship, the data to be written in the write request is written to the storage space corresponding to the object to be written in the second data storage pool.

[0075] In other words, if the object to be written does not exist in the first data storage pool, it means that the first data storage pool does not have the corresponding data for that object. To facilitate access to the data of the object by the intermediate business layer, a new object to be written can be created in the first data storage pool based on its attributes. This newly created object is an empty object without stored data. Simultaneously, a new object to be written can also be created in the second data storage pool based on its attributes. This newly created object is also initially empty. A mapping relationship can then be established between the empty object to be written in the first data storage pool and the corresponding object in the second data storage pool; this mapping relationship can also be called a link. Thus, if the business layer needs to access data during a data protection policy change, it can use this mapping relationship to link from the object in the first data storage pool to the corresponding object in the new data storage pool to access the data of the corresponding object, ensuring business continuity. After establishing the mapping relationship, newly transmitted data to be written by the business layer can also be written to the storage space of the object to be written in the second data storage pool.

[0076] In addition, it should be noted that each object in the first data storage pool can establish its own data storage pool mapping relationship, which represents the specific data storage pool in which the object's data is stored.

[0077] In this embodiment, by deleting the data of the object to be written when it exists in the first data storage pool, creating a new object in the second data storage pool, and establishing a mapping relationship between the objects in the two data storage pools, it is possible to ensure business continuity if the business layer needs to access data during data protection policy changes. This is achieved by linking the object in the first data storage pool to the corresponding object in the new data storage pool (the second data storage pool). Furthermore, by creating a new object in both the first and second data storage pools when it does not exist, and simultaneously establishing a mapping relationship between the objects in the two data storage pools, business continuity can also be ensured even when the object is absent in the first data storage pool.

[0078] Furthermore, for the aforementioned storage cluster, a data index pool can be pre-configured. This data index pool includes multiple objects and their mapping relationships. It primarily links objects accessed by the business layer to the actual data storage pool where those objects' data is stored. Initially, this data index pool is the first data storage pool, meaning both are identical. As data protection strategies change, both the data storage pool and the data index pool can be continuously replaced based on data storage conditions. Therefore, when detecting the existence of a second data storage pool in the storage cluster, the actual detection involves checking if the data index pool is consistent with the first data storage pool. If they are consistent, it indicates the absence of a second data storage pool; otherwise, it indicates the presence of a second data storage pool.

[0079] In addition, the aforementioned storage cluster may also include a metadata storage pool. This metadata storage pool stores a first command, which includes a first identifier of the first data storage pool. That is, the first command actually stores the identifier of the data index pool, and is a permanent command that is only updated after all data in the old data storage pool has been deleted. Since the above example uses the first data storage pool as the initial data storage pool, the identifier of this first data storage pool is the identifier of the data index pool, denoted as the first identifier, such as Data1. Therefore, as the data protection strategy changes, the first identifier of the data index pool in the metadata storage pool can be updated according to the data storage status. The following embodiment illustrates the process of updating the first identifier of the data index pool.

[0080] In some embodiments, the above method may further include the following steps:

[0081] If all data in the first data storage pool has been deleted, and all objects in the mapping relationship are mapped to the same second data storage pool, then the first data storage pool is released; the second identifier of the second data storage pool is obtained, and the first identifier in the first command is replaced with the second identifier.

[0082] As mentioned in the above embodiments, when a second data storage pool exists, data in the object storage space of the first data storage pool can be continuously deleted as the data protection strategy changes. After all the data in the first data storage pool is deleted, what remains in the first data storage pool are all empty objects with no data stored. In the case that all the empty objects in the first data storage pool are linked to the same second data storage pool, the first data storage pool can be released in order to save storage space.

[0083] Then, the identifier corresponding to the new second data storage pool can be obtained, denoted as the second identifier. This second data storage pool can be used as a new data index pool. Therefore, the identifier of the data index pool in the metadata storage pool can be updated, specifically by replacing the first identifier with the second identifier, that is, the data index pool is now the second data storage pool.

[0084] In this embodiment, the first data storage pool is released after all data in the old first data storage pool is deleted, which saves storage space. At the same time, by updating the identifier of the data index pool to the identifier of the new second data storage pool, the data protection strategy can be continuously changed online, while ensuring that the business can continue normally without being affected during this period.

[0085] Furthermore, the aforementioned metadata storage pool can also store a second command. This second command includes a first identifier of the first data storage pool. This second command actually stores the identifier of the data storage pool currently storing the data; it is a temporary command and can be updated as new data storage pools are added. Since the above example uses the first data storage pool as the initial data storage pool, initially, the identifier stored in the second command is the identifier of the first data storage pool, and it can be updated continuously as new data storage pools appear. The following embodiment will explain how to update the identifier of the data storage pool currently storing the data in the second command.

[0086] In some embodiments, before detecting whether a second data storage pool exists in the storage cluster in S102, the method may further include the following steps:

[0087] Obtain the data protection policy change instruction sent by the business layer; the data protection policy change instruction includes the target protection level corresponding to the changed data protection policy.

[0088] A second data storage pool matching the target protection level is determined within the storage cluster; the aforementioned storage cluster includes data storage pools with different protection levels.

[0089] Obtain the second identifier of the second data storage pool and replace the first identifier in the second command with the second identifier.

[0090] As mentioned above, when a data protection policy changes, the storage cluster can be notified of the change. Specifically, the application layer can send a data protection policy change command to the storage cluster, instructing it to modify the data protection policy. This command can not only indicate a change in the data protection policy but also include the target protection level, which can be higher or lower than the original protection level. Generally, a higher protection level requires more storage space.

[0091] The aforementioned storage cluster can include multiple different data storage pools, each with its own protection level. In other words, the storage cluster can contain data storage pools with different protection levels. After receiving the data protection policy change instruction from the business layer, the target protection level can be parsed out. Then, the storage cluster is searched for a data storage pool with a protection level (generally equal to or greater than) that matches the target protection level. This matching data storage pool can be designated as the second data storage pool. The identifier corresponding to this second data storage pool can then be obtained from the storage cluster, designated as the second identifier, such as Data2. Finally, the first identifier in the second command can be replaced with the second identifier, updating the data storage pool currently storing the data from the first data storage pool to the second data storage pool.

[0092] After the steps of this embodiment, the actual detection of whether the data index pool detected above is consistent with the first data storage pool is whether the identifier in the first command and the identifier in the second command are consistent. If they are consistent, it means that there is no new data storage pool; if they are inconsistent, it means that there is a new data storage pool.

[0093] In this embodiment, when the data protection policy changes, the business layer informs the storage cluster of the protection level of the changed protection policy. This allows the storage cluster to find a second data storage pool that matches the protection level and update the identifier of the data storage pool currently storing data in the second command accordingly. This enables the storage cluster to quickly and accurately detect the new data storage pool to store data when the business layer needs to write data, thereby improving the efficiency and accuracy of data writing / storage.

[0094] The above embodiments illustrate the process of writing data from the business layer to the data storage pool. The following embodiments illustrate the process of the business layer reading data from the data storage pool.

[0095] Figure 3This is the third flowchart of the data writing method for monitoring scenarios provided by the present invention, as shown below. Figure 3 As shown, the above method may further include the following steps:

[0096] S302, Obtain the read request sent by the business layer; the read request includes the object to be read corresponding to the data to be read.

[0097] When the business layer needs to read data from the data storage pool, it can first determine which object's data needs to be read, that is, determine the object to be read. Then, the object to be read can be encapsulated in a read request and sent to the storage cluster. In this way, the storage cluster can obtain the object that the business layer wants to read, that is, obtain the object to be read.

[0098] S304, if the data corresponding to the object to be read in the first data storage pool is not empty, then read the data stored in the storage space of the object to be read in the first data storage pool.

[0099] In this step, the first data storage pool can be the current data index pool. The storage cluster can first find the object to be read in the first data storage pool and determine whether there is data stored in the storage space of the object to be read. If there is data, that is, the object to be read in the first data storage pool is not an empty object, then the corresponding data can be read from the storage space of the object to be read in the first data storage pool and the read data can be sent to the business layer.

[0100] S306, if the data corresponding to the object to be read in the first data storage pool is empty, then the object to be read is found in the second data storage pool according to the mapping relationship, and the data stored in the storage space of the object to be read is read in the second data storage pool.

[0101] In this step, if the storage cluster finds the object to be read in the first data storage pool and determines that there is no data stored in the storage space of the object to be read, that is, the object to be read in the first data storage pool is an empty object, then the object to be read can be linked to the same object in the second data storage pool through the mapping / linking relationship between the objects in the first data storage pool and the same objects in the new second data storage pool. The corresponding data can then be read from the storage space of the object to be read in the second data storage pool, and the read data can be sent to the business layer.

[0102] In this embodiment, when the business layer needs to read data, if the data of the object to be read exists in the old first data storage pool, it can be read directly. If the data of the object to be read does not exist in the old first data storage pool, the read request can be linked to the new second data storage pool through the link relationship to read the corresponding data. In this way, even if the protection strategy changes continuously, the read request of the business layer can be executed smoothly without affecting the normal data reading of the business layer.

[0103] To facilitate a more detailed explanation of the technical solution of the present invention, an example scenario is given below for illustration.

[0104] The scenario is as follows: The original data was stored in the erasure 6+1 policy pool Data1, which was later upgraded to the erasure 6+2 policy. Subsequently, due to business needs, a new three-replica data pool was created, and the data protection level was upgraded again to the 6+3 policy.

[0105] See Figure 4 The diagram illustrates one of the data storage pool changes. Initially, the data storage pool was configured as Data1, but it was switched to the erasure 6+2 data storage pool Data2 midway through the process. Here, `meta` represents the metadata stored in the metadata storage pool. `{"dataIdxPol":"DataX"}` is the first command, representing information about the data index pool. `DataX` represents the identifier of the data index pool, and `X` represents the specific identifier number. Initially, the identifier of the data index pool was the identifier of the first data storage pool, which was Data1. `{"dataPol":"DataY"}` is the second command, representing information about the data storage pool. `DataY` represents the identifier of the data storage pool, and `Y` represents the specific identifier number. Initially, it was the identifier of the first data storage pool, Data1, but it is updated to the identifier of the new second data storage pool, Data2, when a new second data storage pool exists. `dataP{"link_pol":"DataZ"}` represents link information or mapping relationships, where `dataP` represents the mapped object, and `DataZ` represents the identifier of the data storage pool that the mapped object maps to / links to. `P` and `Z` represent the specific identifier numbers, respectively.

[0106] The specific procedures for changing the data storage pool and data index pool are as follows:

[0107] 1. The initial configuration of the storage cluster is Data1. At this time, the identifier of the data storage pool and the identifier of the data index pool recorded in the metadata storage pool are both Data1, that is, the first command {"dataIdxPol":"Data1"}, the second command {"dataPol":"Data1"}, and the erasure policy is 6+1.

[0108] 2. After monitoring and storing data for a period of time, see [link / reference]. Figure 4One of the data storage pool change diagrams shown is to obtain the data protection policy change instruction sent by the business layer, switch the data protection policy from erasure policy 6+1 to erasure policy 6+2, and switch the current data storage pool from the data storage pool Data1 corresponding to erasure policy 6+1 to the data storage pool Data2 corresponding to erasure policy 6+2.

[0109] 3. At this point, update the identifier of the data storage pool recorded in the metadata storage pool to Data2, that is, update the second command {"dataPol":"Data1"} to {"dataPol":"Data2"}. Determine whether Data1 meets the cleanup conditions (i.e., all data in Data1 has been deleted). If the cleanup conditions are not met, continue to use Data1 as the data index pool.

[0110] 4. When writing data at the business layer, the information of the data index pool and data storage pool is sent to Data1. If the storage cluster detects that the current data storage pool is not Data1 but Data2, an empty object is created in Data1, and the attribute information of the empty object, i.e. the mapping / association / link information, is recorded as {"link_pol":"Data2"}, and data is written to Data2.

[0111] 5. After data has been written for a period of time, the data protection policy switches from erasure policy 6+2 to erasure policy 6+3, and the data storage pool switches to the data storage pool Data3 corresponding to erasure policy 6+3. See [link / reference] Figure 5 The second diagram illustrating the data storage pool change shows that after a period of writing, the data protection strategy continues to change to erasure policy 6+3. The identifier update strategy of the data storage pool in the metadata storage pool is the same as the initial change, changing to {“dataPol”:“Data3”}. The data index pool remains Data1. When writing data to the new data storage pool, the corresponding empty object is still created in the data index pool and the mapping relationship / association relationship / link relationship is recorded.

[0112] 6. At any time, see Figure 6 The third diagram of the data storage pool change shows that the data index pool is checked to see if it meets the update conditions, that is, whether the data index pool is full of empty objects. If the data index pool does not meet the update conditions, empty objects are still created and the mapping / association / linking relationships of the empty objects are recorded.

[0113] 7. Starting from the current data storage pool, calculate the data distribution after one monitoring cycle of full coverage, referring to [reference needed]. Figure 7The fourth diagram illustrating the data storage pool change shows that after the full coverage is complete, all data is stored in data storage pool Data3. The original data index pool Data1 now only contains empty objects and mapping / association / link data. It should be noted that the storage space occupied by mapping / association / link data is negligible compared to the actual data storage space.

[0114] 8. When it is detected that the data index pool no longer contains objects storing data (i.e., all are empty objects), and there are no associated data storage pools (i.e., inconsistent data storage pool information for empty object mappings / associations / links) (used to handle scenarios with multiple changes), clean up the mapping / association / link data in the current data index pool and update the identification information in the metadata storage pool. See also Figure 8 The fifth diagram illustrating the data storage pool change specifically involves determining that the original data index pool Data1 has no abnormal items (i.e., Data1 pool contains only empty objects, and the data storage pools mapped / associated / linked to the empty objects are consistent, such as all being mapped / associated / linked to Data3). If this is done, the metadata storage pool is notified to update both the data storage pool identifier and the data index pool identifier to Data3. Specifically, the first command in the metadata storage pool is updated from the identifier {"dataIdxPol":"Data1"} of the first data storage pool to the identifier {"dataIdxPol":"Data3"} of the third data storage pool. The original data storage pool Data1 and the intermediate data storage pool Data2 are then released. Thus, both the original data storage pool Data1 and the intermediate data storage pool Data2 are released from the business logic, thereby completing the continuous online change of the data protection strategy without affecting real-time business operations.

[0115] As can be seen from the description of the above embodiments, the technical solution of the present invention has the following technical effects:

[0116] 1. It can provide continuous online change services for data protection policies.

[0117] 2. The database space occupied by mapping / association / linking relationship data is negligible relative to the actual data space. Since the actual data space has been cleaned up, if it can be assessed that the amount of data written does not exceed the amount of data being migrated, the original data storage pool can provide complete storage services even before the data protection policy change is completed.

[0118] 3. During the data protection strategy change, all new data in the business layer adopts the new protection strategy. It can be combined with object attributes as secondary index association to realize transparent access of cross-pool data in the cluster by the business layer. In other words, the old data storage pool can also support business access, ensuring business continuity and transparency to upper-layer business.

[0119] 4. In actual monitoring systems, the data in the original data storage pool will be written by the business layer and gradually cleaned up and replaced by the full coverage process. There is no additional data cleaning and migration work, the resource consumption is low, and it does not affect real-time business.

[0120] 5. Streamlined configuration can free up the actual space occupied by the original data storage pool and intermediate data storage pool, saving storage space.

[0121] The data writing device for monitoring scenarios provided by the present invention is described below. The data writing device for monitoring scenarios described below and the data writing method for monitoring scenarios described above can be referred to in correspondence.

[0122] Figure 9 This is a schematic diagram of the data writing device for monitoring scenarios provided by the present invention. See also... Figure 9 As shown, this device can be applied to a storage cluster, which includes a first data storage pool for storing data prior to changes in data protection policies. The device may include:

[0123] The detection module 410 is used to detect whether a second data storage pool exists in the storage cluster when a write request is received from the business layer. The second data storage pool is used to store data after the data protection policy has been changed. The write request includes data to be written, which includes data in the next cycle after the full cycle storage of the current monitoring cycle or data that has not been stored in the current monitoring cycle.

[0124] The write data module 420 is used to write the data to be written to the second data storage pool according to the write request if a second data storage pool exists.

[0125] In some embodiments, the write request includes a write object corresponding to the data to be written, and the write data module 420 is specifically used for...

[0126] If an object to be written exists in the first data storage pool, the data corresponding to the object to be written is deleted from the first data storage pool; an object to be written is created in the second data storage pool, and a mapping relationship is established between the object to be written in the first data storage pool and the object to be written in the second data storage pool; according to the mapping relationship, the data to be written in the write request is written to the storage space corresponding to the object to be written in the second data storage pool.

[0127] Optionally, the aforementioned write data module 420 is also used for

[0128] If the object to be written does not exist in the first data storage pool, the object to be written is created in the first data storage pool; the object to be written is created in the second data storage pool, and a mapping relationship is established between the object to be written in the first data storage pool and the object to be written in the second data storage pool; according to the mapping relationship, the data to be written in the write request is written to the storage space corresponding to the object to be written in the second data storage pool.

[0129] In some embodiments, the write request includes the object to be written corresponding to the data to be written, and the write data module 420 is further used for

[0130] If the second data storage pool does not exist, then determine whether there is an object to be written in the first data storage pool; if there is an object to be written, then according to the write request, overwrite the data to be written into the storage space corresponding to the object to be written in the first data storage pool.

[0131] In some embodiments, the storage cluster further includes a metadata storage pool storing a first command, the first command including a first identifier of the first data storage pool, and the apparatus further includes an update module for...

[0132] If all data in the first data storage pool has been deleted, and all objects in the mapping relationship are mapped to the same second data storage pool, then the first data storage pool is released; the second identifier of the second data storage pool is obtained, and the first identifier in the first command is replaced with the second identifier.

[0133] In some embodiments, the storage cluster further includes a second data storage pool and a metadata storage pool. The metadata storage pool stores a second command, which includes a first identifier of the first data storage pool. Before detecting whether the second data storage pool exists in the storage cluster, the update module is further configured to...

[0134] Obtain the data protection policy change instruction sent by the business layer; the data protection policy change instruction includes the target protection level corresponding to the changed data protection policy; determine the second data storage pool that matches the target protection level in the storage cluster according to the target protection level; the storage cluster includes data storage pools with different protection levels; obtain the second identifier of the second data storage pool, and replace the first identifier in the second command with the second identifier.

[0135] In some embodiments, the above-described apparatus further includes:

[0136] The read request acquisition module is used to acquire read requests sent by the business layer; the read requests include the object to be read corresponding to the data to be read.

[0137] The data reading module is used to read the data stored in the storage space of the object to be read from the first data storage pool if the data corresponding to the object to be read in the first data storage pool is not empty.

[0138] The data reading module is also used to find the object to be read in the second data storage pool according to the mapping relationship if the data corresponding to the object to be read in the first data storage pool is empty, and to read the data stored in the storage space of the object to be read in the second data storage pool.

[0139] It should be noted that the apparatus provided in this embodiment of the invention can implement all the method steps implemented in the above method embodiment and can achieve the same technical effect. Therefore, the parts and beneficial effects that are the same as those in the method embodiment will not be described in detail here.

[0140] Figure 10 An example is a schematic diagram of the physical structure of a storage cluster, such as... Figure 10 As shown, the storage cluster may include: a first data storage pool, a processor 510, a communications interface 520, a memory 530, and a communication bus 540. The first data storage pool stores data prior to changes in the data protection policy. The processor 510, communications interface 520, and memory 530 communicate with each other via the communication bus 540. The processor 510 can invoke logical instructions in the memory 530 to execute a data writing method for a monitoring scenario. This method includes: upon receiving a write request from the business layer, detecting whether a second data storage pool exists in the storage cluster; the second data storage pool stores data after the data protection policy change; the write request includes data to be written, which includes data from the next cycle after the current monitoring cycle has been fully stored, or data not yet stored within the current monitoring cycle; if a second data storage pool exists, the data to be written is written to the second data storage pool according to the write request.

[0141] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0142] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the data writing method for monitoring scenarios provided by the above methods. The method includes: upon receiving a write request sent by the business layer, detecting whether a second data storage pool exists in the storage cluster; the second data storage pool is used to store data after the data protection policy has been changed; the write request includes data to be written, which includes data in the next cycle after the full cycle storage of the current monitoring cycle or data that has not been stored in the current monitoring cycle; if a second data storage pool exists, then writing the data to be written to the second data storage pool according to the write request.

[0143] In another aspect, the present invention also provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program implements a data writing method for monitoring scenarios provided by the above methods. The method includes: upon receiving a write request sent by the business layer, detecting whether a second data storage pool exists in the storage cluster; the second data storage pool is used to store data after a change in data protection policy; the write request includes data to be written, which includes data in the next cycle after the current monitoring cycle has been fully stored or data that has not been fully stored within the current monitoring cycle; if a second data storage pool exists, writing the data to be written to the second data storage pool according to the write request.

[0144] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0145] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0146] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data writing method for monitoring scenarios, characterized in that, Applied to a storage cluster, the storage cluster including a first data storage pool, the first data storage pool being used to store data prior to a change in data protection policy, the method includes: Upon receiving a write request from the business layer, the system checks whether a second data storage pool exists in the storage cluster. The second data storage pool is used to store data after the data protection policy has been changed. The write request includes data to be written, which includes data from the next cycle after the current monitoring cycle has been fully stored or data that has not been fully stored within the current monitoring cycle. If the second data storage pool exists, the data to be written is written to the second data storage pool according to the write request; The write request also includes the write object corresponding to the data to be written, and the step of writing the data to be written to the second data storage pool according to the write request includes: If the object to be written does not exist in the first data storage pool, the object to be written is created in the first data storage pool; the object to be written is also created in the second data storage pool, and a mapping relationship is established between the object to be written in the first data storage pool and the object to be written in the second data storage pool; according to the mapping relationship, the data to be written in the write request is written to the storage space corresponding to the object to be written in the second data storage pool. If the object to be written exists in the first data storage pool, the data corresponding to the object to be written is deleted from the first data storage pool; the object to be written is created in the second data storage pool, and a mapping relationship is established between the object to be written in the first data storage pool and the object to be written in the second data storage pool; according to the mapping relationship, the data to be written in the write request is written to the storage space corresponding to the object to be written in the second data storage pool.

2. The data write-in method for a monitoring scenario according to claim 1, wherein, The write request includes the object to be written corresponding to the data to be written, and the method further includes: If the second data storage pool does not exist, then determine whether the object to be written exists in the first data storage pool; If the object to be written exists, then according to the write request, the data to be written is overwritten into the storage space corresponding to the object to be written in the first data storage pool.

3. The data write method for a surveillance scenario according to any one of claims 1 to 2, characterized in that, The storage cluster further includes a metadata storage pool, which stores a first command, the first command including a first identifier of the first data storage pool, and the method further includes: If all data in the first data storage pool has been deleted, and all objects in the mapping relationship are mapped to the same second data storage pool, then the first data storage pool is released. Obtain the second identifier of the second data storage pool, and replace the first identifier in the first command with the second identifier.

4. The data write method for a surveillance scenario according to any one of claims 1 to 2, characterized in that, The storage cluster further includes a second data storage pool and a metadata storage pool. The metadata storage pool stores a second command, which includes a first identifier of the first data storage pool. Before detecting whether the second data storage pool exists in the storage cluster, the method further includes: Obtain the data protection policy change instruction sent by the service layer; the data protection policy change instruction includes the target protection level corresponding to the changed data protection policy; A second data storage pool matching the target protection level is determined in the storage cluster; the storage cluster includes data storage pools with different protection levels. Obtain the second identifier of the second data storage pool, and replace the first identifier in the second command with the second identifier.

5. The data write-in method for a surveillance scenario according to claim 1, wherein, The method further includes: Obtain the read request sent by the business layer; the read request includes the object to be read corresponding to the data to be read; If the data corresponding to the object to be read in the first data storage pool is not empty, then the data stored in the storage space of the object to be read is read from the first data storage pool; If the data corresponding to the object to be read in the first data storage pool is empty, then the object to be read is found in the second data storage pool according to the mapping relationship, and the data stored in the storage space of the object to be read is read in the second data storage pool.

6. A data writing device for monitoring scenarios, characterized in that, Applied to a storage cluster, the storage cluster including a first data storage pool, the first data storage pool being used to store data prior to a change in data protection policy, the apparatus comprising: The detection module is used to detect whether a second data storage pool exists in the storage cluster when a write request is received from the business layer; the second data storage pool is used to store data after the data protection policy has been changed, and the write request includes data to be written, which includes data in the next cycle after the full cycle storage of the current monitoring cycle or data that has not been stored in the current monitoring cycle. The write data module is used to write the data to be written to the second data storage pool according to the write request if the second data storage pool exists. The write request also includes the write object corresponding to the data to be written. The write data module is specifically configured to: if the write object does not exist in the first data storage pool, create the write object in the first data storage pool; create the write object in the second data storage pool and establish a mapping relationship between the write object in the first data storage pool and the write object in the second data storage pool; write the data to be written in the write request to the storage space corresponding to the write object in the second data storage pool according to the mapping relationship; if the write object exists in the first data storage pool, delete the data corresponding to the write object in the first data storage pool; create the write object in the second data storage pool and establish a mapping relationship between the write object in the first data storage pool and the write object in the second data storage pool; and write the data to be written in the write request to the storage space corresponding to the write object in the second data storage pool according to the mapping relationship.

7. A storage cluster comprising a first data storage pool, a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The first data storage pool is used to store data prior to the change of data protection policy, and when the processor executes the program, it implements the data writing method for monitoring scenarios as described in any one of claims 1 to 5.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by the processor, it implements the data writing method for monitoring scenarios as described in any one of claims 1 to 5.

Images