Method for storing data, computing device, storage medium and program product

Abstract

The invention relates to a method for storing data, a computing device, a storage medium and a program product. The method comprises the following steps: generating a data acquisition instruction in response to a predetermined condition; the method further comprises the steps that in response to the data collection instruction, data collection is carried out so as to obtain target data. And determining a storage strategy corresponding to the target data based on the data attribute of the target data and the storage state corresponding to the memory. In addition, the method further comprises the step of storing the target data based on the storage strategy. Through the mode, the optimal storage strategy can be selected for the to-be-stored data according to the real-time data characteristics and the state of the memory, so that the performance and the capacity of the memory can be fully utilized, and the storage efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and more particularly to a method, a computing device, a computer storage medium and a computer program product for storing data. Background Art

[0002] In industrial scenarios, smart home scenarios, video surveillance scenarios or other scenarios, sensors or other devices will collect a large amount of data, which is large in volume and complex in type. How to store and transmit this data is crucial. For example, in industrial scenarios, efficient collection, processing and analysis of data can monitor the operating status of equipment. On this basis, the industrial system can make reasonable responses or decisions in a timely manner to improve its automation and intelligence.

[0003] Traditional methods for storing data mainly include storing the collected data in the same storage device, such as a solid-state drive or memory. During the storage process, all collected data is stored in the same storage location without distinguishing the data type, which may lead to a waste of storage resources and performance bottlenecks.

[0004] In summary, the traditional method for storing data has the following shortcomings: it is difficult to fully utilize the performance and capacity of the memory, and the storage efficiency is low. Summary of the invention

[0005] The embodiments of the present invention provide a method for storing data, a computing device, a computer storage medium and a computer program product, which can fully utilize the performance and capacity of a memory and improve storage efficiency.

[0006] In a first aspect of the present invention, a method for storing data is provided. The method includes generating a data acquisition instruction in response to a predetermined condition being satisfied. The method also includes acquiring data in response to the data acquisition instruction so as to obtain target data. The method also includes determining a storage strategy corresponding to the target data based on data attributes of the target data and a storage state corresponding to the memory, wherein the storage state includes at least one or more of the storage capacity, read / write speed, and storage reliability of the memory. In addition, the method also includes storing the target data based on the storage location and storage method indicated by the storage strategy.

[0007] In a second aspect of the present invention, an electronic device is provided, which includes one or more processors and a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method provided according to the first aspect of the present invention.

[0008] In a third aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer-executable instructions, wherein the computer-executable instructions are executed by a processor to implement the method provided according to the first aspect of the present invention.

[0009] According to a fourth aspect of the present invention, there is provided a computer program product, comprising machine executable instructions, which, when executed, enable a machine to perform the method provided by the first aspect of the present invention.

[0010] It should be understood that the contents described in the summary of the invention are not intended to limit the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and other features, advantages and aspects of the embodiments of the present invention will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements.

[0012] Figure 1 A schematic diagram is shown of an example environment in which various embodiments of the invention may be implemented.

[0013] Figure 2 A flow chart of a method for storing data according to some embodiments of the present invention is shown.

[0014] Figure 3 A schematic diagram of a method for collecting data according to some embodiments of the present invention is shown.

[0015] Figure 4 A schematic diagram of a method for adjusting a storage policy according to some embodiments of the present invention is shown.

[0016] Figure 5 A schematic diagram of a data migration method according to some embodiments of the present invention is shown.

[0017] Figure 6 A schematic diagram of backing up data according to some embodiments of the present invention is shown.

[0018] Figure 7 A schematic diagram showing multiple buffers storing data according to some embodiments of the present invention.

[0019] Figure 8 A block diagram of an apparatus for storing data according to some embodiments of the present invention is shown.

[0020] Fig. 9 A block diagram is shown of a device in which various embodiments of the present invention may be implemented. DETAILED DESCRIPTION

[0021] Embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present invention. It should be understood that the drawings and embodiments of the present invention are only for exemplary purposes and are not intended to limit the scope of protection of the present invention.

[0022] In the description of the embodiments of the present invention, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0023] As mentioned above, due to the limited storage space in the storage system, in order to ensure the storage performance of the storage system, it is necessary to manage the large amount of stored data. The collected data usually has high real-time and accuracy requirements. Therefore, how to use efficient and reliable storage solutions to store the collected data is crucial.

[0024] To this end, an embodiment of the present invention provides a method for storing data. The method includes generating a data acquisition instruction in response to a predetermined condition being met. The method also includes acquiring data in response to the data acquisition instruction so as to obtain target data. The method also includes determining a storage strategy corresponding to the target data based on data attributes of the target data and a storage state corresponding to the memory, wherein the storage state includes at least one or more of the storage capacity, read / write speed, and storage reliability of the memory. In addition, the method also includes storing the target data based on the storage location and storage method indicated by the storage strategy.

[0025] In this way, the optimal storage strategy can be selected for the data to be stored according to the real-time data characteristics and memory status, so that the performance and capacity of the memory can be fully utilized to avoid the waste of storage resources. On the other hand, when the data changes or the memory changes, the storage strategy can be adjusted flexibly and accurately, thereby enhancing data security on the basis of improving data storage efficiency. In addition, data for storage can be flexibly and accurately collected according to changes in conditions, thereby further improving the efficiency and accuracy of data storage.

[0026] Figure 11 is a schematic diagram of an example environment 100 in which various embodiments of the present invention may be implemented. Figure 1 As shown, in the environment 100, a data acquisition system 102, a processing device 104 and a storage system 106 are included. The data acquisition system 102 and the controller 104, and the processing device 104 and the storage system 106 are connected in communication, for example, by wireless connection, wired connection, etc. The processing device 104 can be a user terminal, a mobile device, a computer, etc., and can also be a computing system, a single server, a distributed server, or a cloud-based server.

[0027] In some embodiments of the present invention, the data acquisition system 102 may include multiple acquisition devices (e.g., acquisition device 102-1, acquisition device 102-2, acquisition device 102-3, acquisition device 102-4). The storage system 106 may include multiple memories (e.g., memory 106-1, memory 106-2). It should be understood that the architecture and functions in the example environment 100 are described only for exemplary purposes, and do not imply any limitation on the scope of the present invention. The embodiments of the present invention may also be applied to other environments with different structures and / or functions.

[0028] In some embodiments of the present invention, the types of multiple acquisition devices may be the same or different. The acquisition device may be used to acquire relevant parameters of industrial equipment or other equipment, such as but not limited to temperature, pressure, voltage, current, humidity, and flow. For example, the acquisition device may be a variety of types of sensors, such as a temperature sensor, a pressure sensor, etc., or may be an image acquisition device, an audio acquisition device, etc.

[0029] In some embodiments of the present invention, the data collection system 102 may start collecting data after receiving a data collection instruction. The data collection instruction is used to indicate the time to collect data and the conditions for starting data collection. The data collection instruction may be generated based on user input information, or generated based on satisfying a predetermined condition, or generated by an external trigger mechanism. For example, multiple data points may be collected within a preset collection time T according to a preset collection frequency f, and the multiple data points may constitute target data ( Figure 1 not shown).

[0030] like Figure 1As shown, after the target data is acquired, the target data can be sent to the processing device 104, and the processing device 104 determines the storage strategy corresponding to the target data based on the data attributes of the target data and the storage status of multiple memories in the storage system 106. In this way, the storage system 106 can store the received target data according to the storage strategy, for example, storing the target data in a specified location (for example, in the storage area A of the memory 106-1) according to the storage method indicated by the storage strategy.

[0031] In some embodiments of the present invention, the storage state of the memory may include the storage capacity, remaining capacity, storage performance (such as read and write speed, response time, etc.), storage reliability, etc. of the memory. The data attributes of the target data may include, but are not limited to, the size of the data, the data type, the access frequency of the data, and the life cycle of the data. For example, small data can be stored in a high-speed storage memory to increase the access speed; for large data, the data can be split and stored in multiple memories respectively to balance the storage performance and storage capacity. In addition, for key data that is more important and needs to be accessed frequently, the data can be stored in a fast-access memory and the key data can be backed up. The backup data can be stored in another memory to improve the reliability of the data.

[0032] In this way, the optimal storage strategy can be made according to the characteristics of the data and the storage resources, thereby ensuring the integrity, availability and performance of the data. In addition, the data for storage can be collected flexibly and accurately according to the changes in conditions, thereby further improving the efficiency and accuracy of data storage.

[0033] Figure 2 FIG. 2 is a flowchart of a method 200 for storing data according to some embodiments of the present invention. The method 200 may be Figure 1 The processing device 104 shown in FIG. Figure 2As shown, in box 202, method 200 includes generating a data acquisition instruction in response to the predetermined condition being met. It can be understood that the acquisition device may continuously acquire a large amount of data, and some of these data need to be stored and some do not need to be stored. On this basis, it can be determined at what time to start collecting data according to the predetermined conditions, so as to determine the data that needs to be stored and improve the efficiency of data storage. Among them, the predetermined condition can be a condition set by the user, and the condition can be a condition related to the data or related to the trigger device. For example, the predetermined condition can be that the rate of change of the data is greater than a certain threshold or the volume of the data is greater than a certain threshold. In other embodiments, the predetermined condition can also be a predetermined state of the trigger device. For example, if the state of the trigger device reaches the predetermined state, it indicates that the predetermined condition is met and a data acquisition instruction can be generated. On this basis, the data acquisition instruction can also include multiple types, for example, it can include but is not limited to automatic acquisition instructions, manual acquisition instructions, and pull-off condition acquisition instructions.

[0034] At block 204, method 200 may include collecting data in response to the data collection instruction to obtain target data. The data collection instruction may be used to indicate a trigger node and a trigger condition for data collection. For example, the data collection instruction may be used to indicate what kind of data to collect or under what conditions to collect data, and may also indicate a collection cycle, a collection frequency, or a collection duration. For example, multiple data points may be collected within a preset collection time T and at a preset collection frequency, and the multiple data points may constitute the target data. In some embodiments of the present invention, the target data may be data that needs to be stored and is associated with a target scene. The target scene may be an industrial scene, an autonomous driving scene, a video surveillance scene, and the like. For example, the target data may be part or all of the business data in a business scene in an industrial system; or the target data may also be business data in a business scene of a vehicle (for example, it may include but is not limited to data collected by sensors of the vehicle or the speed, acceleration, etc. of the vehicle) or environmental data during the driving process of the vehicle. In some embodiments of the present invention, the target data may be data collected by sensors, terminal devices, or other devices. For example, it may include but is not limited to temperature data, pressure data, image data, video data, voice data, and the like.

[0035] In box 206, method 200 may include determining a storage strategy corresponding to the target data based on the data attributes of the target data and the storage state corresponding to the memory, wherein the storage state includes at least one or more of the storage capacity, read / write speed, and storage reliability of the memory. The data attributes of the data may include, but are not limited to, data type, data size, confidentiality level of the data, importance level of the data, physical meaning of the data (e.g., temperature, flow, pressure, etc.), data format (e.g., integer, floating point number, string, etc.), and data acquisition frequency, acquisition cycle, etc. The data types may include, but are not limited to, real-time data (e.g., data that needs to be accessed or updated at a high frequency), temporary data (e.g., data with a short life cycle), historical data (e.g., data collected in the previous cycle or the previous cycle), key data, etc. Key data may be data related to the current scenario or current business. In some embodiments of the present invention, the importance level of the data may be divided according to the degree of impact caused by data loss or damage, for example, it may be divided into very important, generally important, unimportant, etc. Alternatively, the importance level of the data may also be represented by a numerical value, such as 20% important, 50% important, 100% important, etc.

[0036] In some embodiments of the present invention, a storage system or storage device may include one memory or multiple memories. The types of memories may be the same or different, for example, including but not limited to static random access memory (SRAM), FLASH memory, and may also include a mechanical hard disk (HDD), a solid state drive (SSD), etc. In other embodiments, the memory may also include a cache memory.

[0037] It can be understood that different memories correspond to different storage states. The storage state may include, but is not limited to, the storage space of the memory (for example, the total capacity and remaining capacity of the memory), resource utilization (for example, resource utilization and I / O utilization), read and write speed (for example, sequential read and write speed or random read and write speed), storage reliability (failure rate of the memory and data recovery capability), power consumption or life (for example, the expected time that the memory can work normally under specific conditions), etc. The storage reliability can be determined based on the bad block information, erase times, alarm information and other storage-related information of the memory. For example, different memories have different storage capacities, some memories have a storage capacity of 100GB, and some memories have a storage capacity of 100TB, that is; or the sizes of blocks in different memories are different or the redundant backup mechanisms used are different.

[0038] In some embodiments of the present invention, the storage strategy of the target data can be determined according to the data attributes and the storage status of the memory. The storage strategy can be used to indicate the storage location of the target data, for example, it can be used to indicate in which memory the target data is stored, or at which position of the memory (such as the starting position). In other embodiments, the storage strategy can also be used to indicate the storage method of the data, such as whether backup is required, whether the target data needs to be stored in a specific cache area first, whether hierarchical storage is required, etc. For example, when the target data collected is large in volume, the target data can be stored in a memory with a large remaining capacity; or when the type of the target data is an integer or a floating point number, the target data can be stored in a memory suitable for sequential reading and writing to improve storage efficiency. Alternatively, when the importance level of the target data is high, the target data can be backed up, or the target data that needs to be read later can be backed up.

[0039] At block 206, method 200 may include storing the target data based on the storage location and storage mode indicated by the storage policy. For example, the target data may be stored at the storage location according to the storage requirements based on the storage location and storage requirements indicated by the storage policy. For example, real-time data with a high degree of importance may be stored in SRAM. Alternatively, target data with a high access frequency may be stored in a storage medium with a high read / write speed, and data with a low access frequency may be stored in a storage medium with a low read / write speed.

[0040] In this way, the optimal storage strategy can be selected for the data to be stored according to the real-time data characteristics and memory status, so that the performance and capacity of the memory can be fully utilized to avoid the waste of storage resources. On the other hand, when the data changes or the memory changes, the storage strategy can be adjusted flexibly and accurately, thereby enhancing data security on the basis of improving data storage efficiency. In addition, data for storage can be flexibly and accurately collected according to changes in conditions, thereby further improving the efficiency and accuracy of data storage.

[0041] In some embodiments of the present invention, data collection instructions may be divided into multiple types according to the type of preset conditions, for example, they may be divided into automatic collection instructions, triggered collection instructions, and manual collection instructions, etc. Among them, the automatic collection instruction may be to start formally collecting data when the collected data meets the requirements. For example, the collection instruction may be automatically triggered when the rate of change of the collected data is greater than a preset slope threshold, or the collection instruction may be automatically triggered when the format of the collected data changes. In some embodiments of the present invention, the manual collection instruction may be a collection instruction triggered by a user (such as an administrator) according to actual needs. For example, a user may trigger a collection instruction through a user interface or other interactive methods according to specific task requirements.

[0042] In some embodiments, the break condition acquisition instruction may be an acquisition instruction triggered when an external device or system meets a predetermined condition. That is, when the external device or system detects that a predetermined condition is met, it will send a trigger signal to the data acquisition system, and the system will start collecting data after receiving the signal. In one example, when the sensor detects that parameters such as temperature, pressure, or humidity are greater than or less than a preset threshold, the acquisition instruction can be triggered; or when the parameters corresponding to the environment (such as light intensity, air quality) are within a preset range or greater than a certain threshold, the acquisition instruction can be triggered. It should be noted that this is equivalent to tying a "rope" to a device in the system, and when the "rope" is broken, the acquisition instruction can be triggered.

[0043] It is understandable that different data acquisition instruction types have different characteristics and are suitable for different application scenarios. In practical applications, the appropriate instruction type can be selected according to specific needs to achieve efficient and accurate data acquisition.

[0044] In actual applications, during the data collection process, multiple data collection instructions may overlap or conflict in the same time period. For example, when the collection data corresponding to the last data collection instruction has not been collected yet, a new data collection instruction is received. In this case, it is necessary to determine a new collection strategy (for example, whether to discard the data that has been collected and re-collect new data or continue to collect data according to the previous progress) according to the priority of multiple data collection instructions. Among them, the priority of the data collection instruction can be used to characterize the importance of the data collection instruction. In one example, the priority of the manual collection instruction is less than the priority of the automatic collection instruction and the priority of the pull-off condition collection instruction. If a pull-off condition collection instruction is received in the process of collecting data in response to the automatic collection instruction, since the priority of the pull-off condition collection instruction is greater than the priority of the automatic collection instruction, the data that has been collected can be discarded and new data can be collected again (for example, new data can be collected according to the collection time indicated by the pull-off condition collection instruction). If a manual collection instruction is received in the process of collecting data in response to the automatic collection instruction, since the priority of the manual collection instruction is higher than the priority of the automatic collection instruction, data can continue to be collected.

[0045] In some embodiments of the present invention, after the raw data is collected according to a preset time period or a preset cycle, the raw data may be preprocessed to extract valuable information from the raw data to obtain target data. For example, the raw data may be filled, filtered, or reduced in dimension to determine the target data.

[0046] Figure 3 FIG. 3 is a schematic diagram showing a method 300 for collecting data according to some embodiments of the present invention. Figure 3 As shown, in response to the data collection instruction, the corresponding data is collected within the collection time T. If a new data collection instruction is received at time t (within time T), it is necessary to determine the comparison result between the priority of the new data collection instruction and the priority of the original data collection instruction, and determine the progress of data collection according to the comparison result. For example, if the priority of the new data collection instruction is higher than the priority of the original data collection instruction, the data collected within the (0-t) time period will be discarded, and new data will be collected again. If the priority of the new data collection instruction is lower than the priority of the original data collection instruction, new data will continue to be collected.

[0047] In this way, when multiple data collection instructions overlap or conflict within the same time period, an appropriate collection strategy can be adopted to collect data, thereby improving the accuracy and flexibility of data collection.

[0048] In some embodiments of the present invention, the raw data can be filtered by a threshold method. For example, a data list containing n data points can be collected within a preset time T. The first m data points in the data list are regarded as noise points, and the first n data points are deleted from the raw data list. It can be understood that by filtering out the initial noise data, the impact of abnormal values ​​caused by equipment startup, sensor calibration or other factors on subsequent analysis can be reduced. Then, the average value corresponding to the first k data points in the data list after filtering out the noise points can be determined, and the average value can be used as a reference point or threshold point. In this way, by traversing the remaining data points, the data points within the reference point range can be used as valid data. Among them, the reference point range can be a certain percentage before and after the reference point or a fixed value range. For example, the reference point range can be (reference point - 20% Base point, base point + 20% reference point).

[0049] In this way, after preprocessing the collected raw data, the number of data points that need to be processed can be reduced, which can reduce the computing resources required for data analysis and processing. In addition, the filtered valid data set is purer, which helps to reduce noise interference in the data analysis process, thereby improving the accuracy and efficiency of data analysis.

[0050] In some embodiments of the present invention, different types of data have different life cycles or importance levels, and have different impacts on storage strategies. For example, target data may be classified according to the nature of the data (such as importance, access frequency, sensitivity, etc.) to determine the storage priority and method corresponding to different types of data. In one example, different priorities may be assigned to different types of data based on the life cycle and importance of the data. For example, the priority of real-time data is higher than that of temporary data. The priority of real-time data is also higher than that of historical data, and the priority of critical data is also higher than that of historical data.

[0051] In some embodiments of the present invention, in order to clearly and accurately determine the memory corresponding to the target data, the storage scores corresponding to different memories can be determined based on the attributes of the data and the state of the memory. Among them, the storage score can be used to characterize the suitability of the target data stored in the memory or the performance of the memory in storing the target data. It can be understood that the score corresponding to the memory can be determined from multiple dimensions. For example, factors that may affect the storage score can be taken into account one by one, and all influencing factors can be comprehensively considered to determine the final storage score.

[0052] In some embodiments of the present invention, different factors have different degrees of influence on the final storage score, or different business scenarios and data types may have different requirements for the memory. Based on this, different weight values ​​can be set for different influencing factors. The weight value can be used to characterize the importance of the influencing factor or the degree of influence of the influencing factor on the storage score. For example, the weight value corresponding to the data type can be a, the weight value corresponding to the storage capacity can be b, and the weight value of the importance level of the data can be c. After determining the weight values ​​corresponding to different influencing factors, the storage score corresponding to the memory can be determined based on all the influencing factors and all the weight values. For example, the influence corresponding to each influencing factor is weighted and summed to obtain a comprehensive score. An optimal storage strategy is selected based on the comprehensive score. For example, data with a higher comprehensive score can be stored in SRAM (because SRAM usually has a faster read and write speed), while data with a lower comprehensive score can be stored in FLASH (because FLASH has a larger storage capacity).

[0053] In some embodiments of the present invention, the sub-score corresponding to each factor (such as the size of the data, the available storage space of the memory, or the priority of the data, etc.) can be determined based on the attributes of the data or the storage state of the memory. The storage score corresponding to the memory is determined based on the weighted sum of the sub-scores of each factor. For example, a score function can be designed for each factor based on the importance of each factor to the selection of the storage strategy. For example, a corresponding logarithmic function or exponential function can be set according to the access frequency of the data to strengthen the importance of frequently accessed data. Alternatively, the priority function of the data can be set to a simple linear function, and the higher the priority, the higher the corresponding sub-score.

[0054] It is understandable that the scoring mechanisms corresponding to different memories may be different. For example, the storage score of memory A (such as SRAM) can be determined based on the type of data, the read and write speed of SRAM, and the available space of SRAM; the storage score of memory B (such as FLASH) can be determined based on the type of data, the security requirements of the data, and the available space of FLASH. In one example, the storage score of SRAM = real-time data priority × weight 1 + read and write speed requirements × weight 2 - available space × weight 3; the storage score of FLASH = historical data priority × weight 4 + data security requirements × weight 5 - available space × weight 3.

[0055] In some embodiments, after determining the storage scores corresponding to different memories, the target memory for storing the target data can be determined based on the comparison result of the storage scores. For example, when the storage score of SRAM is greater than the storage score of FLASH, SRAM can be used as the target memory for storing the target data.

[0056] It should be noted that since the properties of storage and data may change over time (for example, the capacity of the storage decreases, the performance decreases, or the importance level of the data changes), the storage score of the storage will also change continuously. On this basis, it is necessary to regularly update the storage scores corresponding to multiple storages and dynamically adjust the storage strategy to ensure that the storage used to store data is still the optimal storage.

[0057] Figure 4 FIG. 4 is a schematic diagram showing a method 400 for adjusting a storage policy according to some embodiments of the present invention. Figure 4 As shown, the original target data 402 can be stored in the memory 404. As the capacity of the memory decreases or the attributes of the target data 402 change, the storage score of the memory 404 may be less than the storage score of the memory 406 after a period of time. On this basis, it is necessary to switch the storage location from the memory 404 to the memory 406, so as to maximize the use of storage resources and improve the efficiency of data storage.

[0058] Figure 5 FIG. 5 is a schematic diagram showing a method 500 for data migration according to some embodiments of the present invention. Figure 5 As shown, in the process of storing the target data 502 in the memory 504, the storage capacity of the memory 504 will continue to decrease as the stored target data 502 increases. On this basis, the remaining storage space (or remaining storage capacity) of the memory 504 can be monitored in real time. If the remaining storage space is less than the data capacity corresponding to the remaining target data that has not been stored, it is necessary to migrate the data 506 that has been stored in the memory 504 to ensure that important data or key data can be stored, thereby improving the rationality of data storage. For example, it can be determined whether the priority of the data 506 that has been stored in the memory 504 is higher than the preset priority. If the priority of the data 506 that has been stored in the memory 504 is lower than the preset priority, the data 506 is migrated from the memory 504 to another memory 508.

[0059] In this way, a reasonable data migration strategy can be used to prevent the risk of being forced to delete data due to insufficient storage space, thereby protecting the integrity and security of the data.

[0060] Figure 6 FIG. 6 is a schematic diagram showing a method 600 for backing up data according to some embodiments of the present invention. Figure 6As shown, in order to ensure the security of data storage, the target data 602 can be backed up when the target data 602 is stored to obtain the backup data 604 corresponding to the target data 602. In some embodiments of the present invention, the target data 602 can be stored in the memory 606, and the backup data 604 can be stored in another memory 608. It can be understood that the backup data 604 ensures redundant storage of data, and even if one of the memories (such as the memory 606) fails or the data is damaged, the target data 602 can still be restored from another memory (memory 608). This redundancy mechanism significantly improves the fault tolerance and availability of data. In addition, the integrity and consistency of the backup data 604 can also be verified regularly to ensure that the backup data is reliable and effective when the data needs to be restored.

[0061] In some embodiments of the present invention, whether to back up the target data and the method or quantity of backup can be determined based on the life cycle or other attributes of the target data. For example, one-time data (such as temporary files or intermediate calculation results) may not need to be backed up; while for data that needs to be read later or data that needs to be accessed frequently, in order to ensure the security and reliability of the data, the data can be backed up. It can be understood that the amount and redundancy of backup data will directly affect the storage cost. Therefore, when formulating a backup strategy, it is necessary to weigh the relationship between data security and storage cost. For example, for data that only needs to be saved for a short period of time, only a shorter backup cycle can be backed up, and the data can be deleted or archived after it expires.

[0062] In actual applications, in order to improve storage efficiency and reduce latency, the target data can be first stored in a buffer, and when the buffer is full, the data in the buffer is stored in the target storage. That is, the target data is pre-cached in a specific cache medium, so as to ensure that the system can obtain the data to be read in time. In addition, in order to further improve the efficiency of data storage and the security of storage, multiple buffers can be set, and different buffers are used to store part of the target data. For example, the first half of the target data can be cached in buffer A, and the second half of the target data can be cached in buffer B.

[0063] Figure 7 FIG. 2 is a schematic diagram showing a method for storing data in multiple buffers according to some embodiments of the present invention. Figure 7As shown, the target data 702 to be stored can be divided into two parts of data such as target data A and target data B. In some embodiments of the present invention, the buffer 704 corresponds to the memory 706, and the buffer 708 corresponds to the memory 710. The reading and writing of the buffer 704 and the buffer 708 do not interfere with each other. For example, the target data A can be written into the buffer 704 in sequence. In this process, the target data B can be written into the buffer 708. It can be understood that when the buffer 704 is full (for example, the remaining capacity of the buffer 704 is 0), the data stored therein can be stored in the memory 706, and then the buffer 704 can be cleared. Similarly, when the buffer 708 is full, the data stored therein can be stored in the memory 710, and the buffer 708 can be cleared.

[0064] like Figure 7 As shown, after clearing the buffer 706, the identities of the buffer 704 and the buffer 708 can be swapped. For example, the remaining data of the target data A can be written into the buffer 708 in sequence, and the remaining data of the target data B can be written into the buffer 704 in sequence. When the buffer 704 is full again, the data stored therein can be stored in the memory 710; when the buffer 708 is full again, the data stored therein can be stored in the memory 706. Through the double buffer mechanism, data can be written into one buffer while data in another buffer is being transferred to the memory, thereby reducing the waiting time and improving the efficiency of data storage.

[0065] It can be understood that the sizes of multiple buffers can be dynamically adjusted according to storage requirements, and the buffer status can be monitored in real time. If an abnormality occurs in a buffer, it can automatically switch to another buffer to ensure data security and stability.

[0066] Figure 8 FIG. 8 is a block diagram of an apparatus 800 for storing data according to some embodiments of the present invention. Figure 8 As shown, the device 800 includes a data acquisition instruction generation unit 802, which is configured to generate a data acquisition instruction in response to the predetermined condition being met. The device 800 also includes a target data acquisition unit 804, which is configured to acquire the target data to be stored in response to the data acquisition instruction. The device 800 also includes a storage strategy determination unit 806, which is configured to determine the storage strategy corresponding to the target data based on the data attributes of the target data and the storage state corresponding to the memory, and the storage state at least includes at least one or more of the storage capacity, read and write speed, and storage reliability of the memory. The device 800 also includes a storage unit 808, which is configured to store the target data based on the storage location and storage method indicated by the storage strategy.

[0067] It can be understood that by using the apparatus 800 of the present invention, at least one of the advantages that can be achieved by the method or process described above can be achieved.

[0068] In some embodiments, the predetermined condition includes any one of the following: confirming that the data change rate is greater than a preset change rate threshold; determining that the state of the monitored trigger device is a preset state; and detecting user input information associated with data collection.

[0069] In some embodiments, the data collection instruction includes a first data collection instruction and a second data collection instruction, and the target data acquisition unit 804 is further configured to: in response to the first data collection instruction, collect target data within a preset time period; when the second data collection instruction is received within the preset time period, obtain the priority of the first data collection instruction and the second data collection instruction; and in response to the priority of the second data collection instruction being higher than the priority of the first data collection instruction, re-collect data based on the second data collection instruction to obtain the target data.

[0070] In some embodiments, the storage strategy includes a target memory for storing target data, and the storage strategy determination unit 806 is further configured to: determine storage scores corresponding to multiple memories respectively based on data attributes of the target data and multiple storage states corresponding to the multiple memories; and determine a target memory for storing the target data from the multiple memories based on the storage scores corresponding to the multiple memories respectively.

[0071] In some embodiments, the multiple memories include a first memory and a second memory, and determining the storage scores corresponding to the multiple memories respectively includes: determining the type of target data based on data attributes of the target data; determining the priority corresponding to the target data based on the type of the target data; and determining a first storage score corresponding to the first memory based on the priority and the storage status corresponding to the first memory; and determining a second storage score corresponding to the second memory based on the priority and the storage status corresponding to the second memory.

[0072] In some embodiments, the storage state of the first memory includes available storage space and read and write speeds, and a first storage score corresponding to the first memory is determined: a first weight value corresponding to the available storage space and a second weight value corresponding to the read and write speed are determined; and based on the available storage space, the first weight value, the read and write speeds, and the second weight value, the first storage score corresponding to the first memory is determined.

[0073] In some embodiments, determining the first storage score corresponding to the first memory also includes: determining a third weight value corresponding to the priority; and determining the first storage score corresponding to the first memory based on available storage space, the first weight value, the read and write speed, the second weight value, the priority and the third weight value.

[0074] In some embodiments, where the data attributes also include a data security coefficient, the storage status of the second memory includes available storage space, and a second storage score corresponding to the second memory is determined: a fourth weight value corresponding to the priority, a fifth weight value corresponding to the data security coefficient, and a sixth weight value corresponding to the available storage space are determined; and based on the priority, the fourth weight value, the data security, the fifth weight value, the available storage space, and the sixth weight value, the second storage score corresponding to the second memory is determined.

[0075] In some embodiments, the target memory is a first memory, and the method further includes: in response to the available storage space of the first memory being less than the data capacity corresponding to the target data, determining a priority order corresponding to multiple data contained in the target data; based on the priority order, determining data in the target data having a priority lower than a preset priority; and migrating the data having a priority lower than the preset priority from the first memory to the second memory.

[0076] In some embodiments, the target memory is a first memory, and the method further includes: determining backup data corresponding to the target data; and storing the backup data in a second memory.

[0077] In some embodiments, the storage unit 808 is further configured to: write the target data into the buffer in sequence; and store the data stored in the buffer into the target memory in response to the remaining capacity of the buffer being less than a preset capacity threshold.

[0078] In some embodiments, the buffer includes a first buffer and a second buffer, the first buffer corresponds to the first memory, the second buffer corresponds to the second memory, the data stored in the buffer is stored in the target memory, and the device 800 also includes: in response to the remaining capacity of the first buffer being less than a preset capacity threshold, the data stored in the buffer is stored in the first memory; and clearing the first buffer and writing the target data to the second buffer.

[0079] In some embodiments, the apparatus 800 further includes a buffer adjustment unit configured to dynamically adjust the size of the buffer based on data attributes of the target data and the type of the data acquisition instruction.

[0080] In some embodiments, the target data acquisition unit 804 is further configured to: in response to a data collection instruction, collect candidate data within a preset time period; determine the average value of some data in the candidate data; and determine corresponding target data from the candidate data based on the average value and a preset ratio.

[0081] In some embodiments, the data attributes of the target data include at least one or more of the following attributes: data volume, encryption information, data security coefficient, data type; the storage state of the memory includes at least one or more of the following states: read and write speed, data access frequency, power consumption information, available storage space, and remaining storage space.

[0082] Fig. 9 A schematic block diagram of an example device 900 that can be used to implement an embodiment of the present invention is shown. As shown, the device 900 includes a computing unit (i.e., CPU 901), which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (i.e., ROM 902) or loaded from a storage unit 908 to a random access memory (i.e., RAM 903). In RAM 903, various programs and data required for the operation of the device 900 can also be stored. CPU 901, ROM 902, and RAM 903 are connected to each other via a bus 904. An input / output interface (i.e., I / O interface 905) is also connected to the bus 904.

[0083] A number of components in the device 900 are connected to the I / O interface 905, including: an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0084] CPU 901 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of CPU 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 901 performs the various methods and processes described above, such as method 200. For example, in some embodiments, method 200 may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 900 via ROM 902 and / or communication unit 909. When the computer program is loaded into RAM 903 and executed by CPU 901, one or more steps of method 200 described above may be performed. Alternatively, in other embodiments, CPU 901 may be configured to execute method 200 in any other appropriate manner (e.g., by means of firmware).

[0085] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip systems (SOCs), load programmable logic devices (CPLDs), and the like.

[0086] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0087] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In addition, although each operation is depicted in a specific order, this should be understood as requiring such operations to be performed in the specific order shown or in a sequential order, or requiring that all illustrated operations should be performed to obtain the desired result. In certain environments, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present invention. Certain features described in the context of a separate embodiment may also be implemented in a single implementation in combination. On the contrary, the various features described in the context of a single implementation may also be implemented in multiple implementations individually or in any suitable sub-combination.

[0088] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A method for storing data, characterized in that include: In response to a predetermined condition being met, generating a data collection instruction; In response to the data collection instruction, collecting data to obtain target data; Determine a storage strategy corresponding to the target data based on data attributes of the target data and storage states corresponding to a plurality of memories, wherein the storage state includes at least one or more of storage capacity, read / write speed, and storage reliability of the memory; as well as The target data is stored based on the storage location and storage method indicated by the storage policy.

2. The method according to claim 1, characterized in that The reservation conditions include any of the following: Confirming that the change rate of the data is greater than a preset change rate threshold; Determining that the state of the monitored trigger device is a preset state; and detecting user input information associated with data collection.

3. The method according to claim 1, characterized in that: The data collection instruction includes a first data collection instruction and a second data collection instruction, and collecting data to obtain target data includes: In response to the first data collection instruction, collecting target data within a preset time period; When the second data collection instruction is received within the preset time period, obtaining the priority of the first data collection instruction and the second data collection instruction; and In response to the priority of the second data acquisition instruction being higher than the priority of the first data acquisition instruction, data is re-acquired based on the second data acquisition instruction to obtain target data.

4. The method according to claim 1, characterized in that The storage strategy includes a target storage device for storing the target data, and determining the storage strategy corresponding to the target data includes: Determining storage scores corresponding to the plurality of memories respectively based on the data attributes of the target data and the plurality of storage states corresponding to the plurality of memories; and Based on the storage scores respectively corresponding to the plurality of memories, a target memory for storing the target data is determined from the plurality of memories.

5. The method according to claim 4, characterized in that The plurality of memories include a first memory and a second memory, and determining storage scores corresponding to the plurality of memories respectively includes: Determining the type of the target data based on the data attribute of the target data; Based on the type of the target data, determining a priority corresponding to the target data; and Determining a first storage score corresponding to the first memory based on the priority and a storage state corresponding to the first memory; and Based on the priority and a storage status corresponding to the second memory, a second storage score corresponding to the second memory is determined.

6. The method according to claim 5, characterized in that The storage state of the first memory includes available storage space and a read / write speed, and determining a first storage score corresponding to the first memory includes: Determine a first weight value corresponding to the available storage space and a second weight value corresponding to the read / write speed; and A first storage score corresponding to the first memory is determined based on the available storage space, the first weight value, the read and write speeds, and the second weight value.

7. The method according to claim 6, characterized in that Determining a first storage score corresponding to the first memory further includes: determining a third weight value corresponding to the priority; and A first storage score corresponding to the first memory is determined based on the available storage space, the first weight value, the read / write speed, the second weight value, the priority, and the third weight value.

8. The method according to claim 5, characterized in that The data attribute further includes a data security coefficient, the storage state of the second memory includes available storage space, and a second storage score corresponding to the second memory is determined: Determine a fourth weight value corresponding to the priority, a fifth weight value corresponding to the data security coefficient, and a sixth weight value corresponding to the available storage space; as well as A second storage score corresponding to the second memory is determined based on the priority, the fourth weight value, the data security, the fifth weight value, the available storage space, and the sixth weight value.

9. The method according to claim 5, characterized in that The target memory is the first memory, and the method further comprises: In response to the available storage space of the first memory being smaller than the data capacity corresponding to the target data, determining a priority order corresponding to a plurality of data included in the target data; Based on the priority order, determining data in the target data with a priority lower than a preset priority; and Migrate data with a priority lower than a preset priority from the first memory to the second memory.

10. The method according to claim 5, characterized in that The target memory is the first memory, and the method further comprises: Determining backup data corresponding to the target data; and The backup data is stored in the second memory.

11. The method according to claim 5, characterized in that Storing the target data further includes: Writing the target data into a buffer in sequence; and In response to the remaining capacity of the buffer being less than a preset capacity threshold, the data stored in the buffer is stored in the target memory.

12. The method according to claim 11, characterized in that The buffer includes a first buffer and a second buffer, the first buffer corresponds to the first memory, the second buffer corresponds to the second memory, and storing the data stored in the buffer into the target memory includes: In response to the remaining capacity of the first buffer being less than a preset capacity threshold, storing the data stored in the first buffer in the first memory; and The first buffer is cleared and the target data is written into the second buffer.

13. The method according to claim 11, characterized in that Also includes: The size of the buffer is dynamically adjusted based on the data attributes of the target data and the type of the data acquisition instruction.

14. The method according to claim 1, characterized in that Obtaining target data includes: In response to the data collection instruction, collecting candidate data within a preset time period; Determine an average value of part of the candidate data; Based on the average value and the preset ratio, corresponding target data is determined from the candidate data.

15. The method according to claim 1, characterized in that The data attributes of the target data include at least one or more of the following attributes: data volume, encryption information, data security coefficient, data type; the storage state of the memory includes at least one or more of the following states: read and write speed, data access frequency, power consumption information, available storage space, and remaining storage space.

16. A computing device, characterized in that: include: at least one processor; as well as A memory coupled to the at least one processor and having instructions stored thereon, the instructions, when executed by the at least one processor, causing the apparatus to perform the method according to any one of claims 1-15.

17. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a machine, the method according to any one of claims 1 to 15 is performed.

18. A computer program product, characterized in that The method comprises machine executable instructions which, when executed, cause a machine to perform the method according to any one of claims 1 to 15.

Images