Storage method, device, terminal equipment and storage medium
By establishing a computing allocation directory and monitoring memory and processor utilization in a high-performance computer cluster, and dynamically managing the storage of computing terminals, the problem of reduced storage capacity was solved, and efficient storage space optimization and dynamic recovery of result files were achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-31
- Publication Date
- 2026-03-31
AI Technical Summary
Existing distributed parallel storage methods result in reduced storage capacity in high-performance computer clusters, and expansion increases costs, making it difficult to effectively manage the storage of large amounts of computation results.
By establishing a computing allocation directory in the management terminal, allocating computing files to the computing terminal for computation, detecting memory usage and processor idle ratio, deleting unnecessary computing allocation directories and result files, generating script files for recovery, and dynamically storing data using the idle processor capabilities of the computing terminal.
Without increasing costs, the storage capacity of high-performance computer clusters was improved, storage space utilization was optimized, and dynamic management and recovery of result files were achieved.
Smart Images

Figure CN114924693B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, specifically to a storage method, apparatus, terminal device, and storable medium. Background Technology
[0002] As user demands for products continue to increase, the amount of computation required by R&D personnel during product design has significantly increased. This results in a large volume of calculations, which need to be stored to ensure usability. Current storage methods utilize distributed parallel storage within high-performance computing clusters. However, the mechanism of distributed parallel storage leads to reduced storage capacity, making it unsuitable for storing large amounts of data without scaling up the high-performance computing clusters. Scaling up, on the other hand, increases costs. Summary of the Invention
[0003] This application discloses a storage method, apparatus, terminal device, and storable medium that can improve the storage capacity of high-performance computer clusters without increasing costs.
[0004] The first aspect of this application discloses a storage method applied to a management terminal, the management terminal being communicatively connected to at least one computing terminal;
[0005] Establish at least one computing allocation directory, and each computing allocation directory contains at least one computing file;
[0006] The computing terminal distributes copies of the computing files contained in each of the computing allocation directories to the computing terminal, so that the computing terminal performs calculations on each of the copies to obtain result files associated with each of the computing allocation directories.
[0007] If the memory usage rate of the computing terminal is detected to be greater than the first usage rate threshold, then the target computing allocation directory to be deleted is determined, and the target computing allocation directory is deleted, as well as the result file associated with the target computing allocation directory in the computing terminal is deleted, so that the memory usage rate of the computing terminal is less than or equal to the first usage rate threshold.
[0008] Generate script files corresponding to the deleted result files in the computing terminal;
[0009] If the processor idle ratio of the computing terminal is detected to be greater than the ratio threshold, then each of the script files is executed to enable the computing terminal to recover the deleted result files.
[0010] As an optional implementation, in the first aspect of this embodiment, the step of running each of the script files to enable the computing terminal to recover the deleted result files if the detected processor idle ratio of the computing terminal is greater than a ratio threshold includes:
[0011] If the processor idle ratio of the computing terminal is detected to be greater than the ratio threshold and the memory usage rate is less than the second usage rate threshold, then each of the script files is run so that the computing terminal restores the deleted result files one by one until the processor idle ratio of the computing terminal is detected to be greater than the ratio threshold, or the memory usage rate is greater than or equal to the second usage rate threshold.
[0012] As an optional implementation, in the first aspect of this embodiment, determining the target computing allocation directory to be deleted, deleting the target computing allocation directory, and deleting the result file associated with the target computing allocation directory in the computing terminal, so that the memory occupancy rate of the computing terminal is less than or equal to the first occupancy rate threshold, includes:
[0013] The computing allocation directories are sorted according to the order in which they were created, resulting in the first directory sort.
[0014] The target computing allocation directories to be deleted are determined sequentially according to the first directory sorting, and the target computing allocation directories are deleted, as well as the result files associated with the target computing allocation directories in the computing terminal are deleted, until the memory occupancy rate of the computing terminal is less than or equal to the first occupancy rate threshold.
[0015] As an optional implementation, in the first aspect of this embodiment, after deleting the target computing allocation directory, the method further includes:
[0016] Record the deletion time corresponding to the target computational allocation directory that was deleted;
[0017] The step of running each of the script files to enable the computing terminal to recover the deleted result files includes:
[0018] The target calculation and allocation directories are sorted according to the order of their deletion times to obtain the second directory sorting.
[0019] The target computing allocation directories to be restored are determined sequentially according to the second directory sorting, and the script file corresponding to the result file associated with the target computing allocation directory to be restored is run so that the computing terminal can restore the deleted result file.
[0020] As an optional implementation, in the first aspect of this embodiment, before allocating copies of the computation files contained in each of the computation allocation directories to the computing terminal, so that the computing terminal performs computation on each of the copies to obtain a result file associated with each of the computation allocation directories, the method further includes:
[0021] Configure a preset calculation time for each of the calculation files, wherein the preset calculation time refers to the length of time the computing terminal performs calculations on the copy files corresponding to the calculation files;
[0022] The step of allocating copies of the computation files contained in each of the computation allocation directories to the computing terminal, so that the computing terminal performs computations on each of the copied files to obtain result files associated with each of the computation allocation directories, includes:
[0023] The computing terminal distributes copies of the computing files contained in each of the computing allocation directories to the computing terminal, so that the computing terminal performs calculations on each of the copied files according to the preset calculation duration corresponding to each of the copied files, and obtains result files associated with each of the computing allocation directories.
[0024] As an optional implementation, in the first aspect of this embodiment, after allocating copies of the computation files contained in each of the computation allocation directories to the computing terminal, so that the computing terminal performs computations on each of the copied files to obtain result files associated with each of the computation allocation directories, the method further includes:
[0025] Record the actual calculation time of each of the result files, where the actual calculation time is the shortest time it takes for the computing terminal to calculate the copied file to obtain the result file;
[0026] The script file that generates the corresponding result files for each deletion in the computing terminal includes:
[0027] The filename of the calculation file corresponding to the result file deleted in the computing terminal is parsed to obtain the calculation parameters of the calculation file corresponding to the deleted result file;
[0028] Create script files corresponding to each deleted result file, and write the actual calculation time of each deleted result file and the calculation parameters of the corresponding calculation file into the script files corresponding to each deleted result file.
[0029] As an optional implementation, in a first aspect of this embodiment, running each of the script files to enable the computing terminal to recover the deleted result file includes:
[0030] Each of the script files is run so that the computing terminal can recover the deleted result file based on the actual computation time of the deleted result file corresponding to the script file and the computation parameters of the computation file corresponding to the deleted result file.
[0031] A second aspect of this application discloses a storage device, comprising:
[0032] A directory building module is used to create at least one computing allocation directory, each of which contains at least one computing file;
[0033] The file allocation module is used to allocate copies of the calculation files contained in each of the calculation allocation directories to the computing terminal, so that the computing terminal can perform calculations on each of the copies to obtain result files associated with each of the calculation allocation directories;
[0034] The file deletion module is used to determine the target computing allocation directory to be deleted and delete the target computing allocation directory if the memory usage rate of the computing terminal is detected to be greater than the first usage rate threshold, and delete the result file associated with the target computing allocation directory in the computing terminal, so that the memory usage rate of the computing terminal is less than or equal to the first usage rate threshold.
[0035] A script building module is used to generate script files corresponding to each deleted result file in the computing terminal;
[0036] The file recovery module is used to run each of the script files if the processor idle ratio of the computing terminal is detected to be greater than a ratio threshold, so that the computing terminal can recover the deleted result files.
[0037] A third aspect of this application discloses a terminal device, including a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the processor implements any of the storage methods disclosed in this application.
[0038] A fourth aspect of this application discloses a computer-readable storage medium that stores a computer program, wherein the computer program, when executed by a processor, implements a storage method disclosed in this application.
[0039] Compared with related technologies, the embodiments of this application have the following beneficial effects:
[0040] In this embodiment, at least one computing allocation directory is established, each containing at least one computing file. A management terminal distributes copies of the computing files contained in each computing allocation directory to a connected computing terminal, enabling the computing terminal to perform calculations on each copy and obtain result files associated with each computing allocation directory. When the management terminal detects that the memory usage rate of the computing terminal exceeds a first usage rate threshold, it determines the target computing allocation directory to be deleted and deletes the target computing allocation directory and the result files associated with it in the computing terminal, so that the memory usage rate of the computing terminal is less than or equal to the first usage rate threshold. The management terminal generates script files corresponding to each deleted result file in the computing terminal, and runs each script file when it detects that the processor idle ratio of the computing terminal exceeds a ratio threshold, enabling the computing terminal to recover the deleted result files. Without needing to expand the high-performance computer cluster, the computing power of the idle central processing unit and graphics processor in the computing terminals of the high-performance computer cluster is used to compensate for the storage capacity of the computing terminals, achieving dynamic storage of result files and improving the storage capacity of the high-performance computer cluster. Attached Figure Description
[0041] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0042] Figure 1 This is a diagram illustrating an application scenario of the storage method in one embodiment;
[0043] Figure 2 This is a flowchart illustrating a storage method in one embodiment;
[0044] Figure 3 This is a schematic diagram illustrating the composition of the calculated allocation directory and calculated file name in one embodiment;
[0045] Figure 4 This is a schematic diagram of the method flow of another storage method disclosed in one embodiment;
[0046] Figure 5 This is a schematic diagram of the method flow of another storage method disclosed in one embodiment;
[0047] Figure 6 This is a schematic diagram of the structure of a storage device disclosed in an embodiment of this application;
[0048] Figure 7 This is a schematic diagram of the structure of a terminal device disclosed in one embodiment. Detailed Implementation
[0049] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0050] It should be noted that the terms "comprising" and "having," and any variations thereof, in the embodiments and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.
[0051] This application discloses a storage method, apparatus, terminal device, and storage method that can improve the storage capacity of high-performance computer clusters without increasing costs. These will be described in detail below.
[0052] Please see Figure 1 , Figure 1 This is a diagram illustrating an application scenario of the storage method in one embodiment. For example... Figure 1As shown, the storage system may include a management terminal 10 and a computing terminal 20, which are communicatively connected. Both the management terminal 10 and the computing terminal 20 can be a single terminal device or a cluster of terminal devices. Terminal devices can be desktop computers, tablets, laptops, and mobile phones, etc. The computing terminal 20 may include at least a central processing unit (CPU) and a graphics processing unit (GPU). The computer systems in the management terminal 10 and the computing terminal 20 can be Windows, Linux, iOS, or Unix, without specific limitations. The management terminal 10 first establishes at least one computing allocation directory, each containing at least one computing file. The management terminal 10 allocates copies of the computing files contained in each computing allocation directory to the computing terminal 20. The computing terminal 20 performs calculations on each copied file to obtain result files associated with each computing allocation directory. When the management terminal 10 detects that the memory usage rate of the computing terminal 20 is greater than a first usage rate threshold, it determines the target computing allocation directory to be deleted and deletes the target computing allocation directory, as well as the result files associated with the target computing allocation directory in the computing terminal 20, so that the memory usage rate of the computing terminal 20 is less than or equal to the first usage rate threshold. The management terminal 10 generates script files corresponding to each deleted result file in the computing terminal 20, and runs each script file when it detects that the processor idle ratio of the computing terminal 20 is greater than the ratio threshold, so that the computing terminal 20 can restore the deleted result files.
[0053] Please see Figure 2 , Figure 2 This is a flowchart illustrating a storage method in one embodiment, which can be applied to the aforementioned management terminal 10. For example... Figure 2 As shown, the method may include the following steps:
[0054] 210. Create at least one computing allocation directory, and each computing allocation directory contains at least one computing file.
[0055] In this embodiment, the management terminal first establishes one or more computing allocation directories based on multiple computing files within the management terminal. Each computing allocation directory corresponds to one or more computing files, and the one or more computing files corresponding to each computing allocation directory may be the same or different. After establishing the computing allocation directories, the management terminal can name each computing allocation directory according to a combination of home directory, administrator account name, work directory, personal user, and project number. For example, a computing allocation directory can be named / home / admin / workspace / user / m.
[0056] In this embodiment, the project content stored in the storage system can be the calculation results of different scenarios during the product design process. During product design, the product model needs to be simulated in different scenarios to determine the performance or effect of the designed product under different conditions. In this scenario, the management terminal in the storage system first needs to obtain the model file. The model file is a collection containing information such as vertex coordinates, indices, UVs, normals, node relationships, materials, textures, and animations—essentially, a model of the designed product. After obtaining the model file, the management terminal converts it into a format suitable for calculation on the computing terminal, according to project requirements; this file is the calculation file. After generating the calculation file, the management terminal can set the name of the calculation file based on the name of the associated calculation allocation directory. Please refer to [link to relevant documentation]. Figure 3 , Figure 3 This is a schematic diagram illustrating the calculation of the allocated directory and the calculation of file names in one embodiment. For example... Figure 3 As shown, the name of the computation allocation directory associated with the computation file is / home / admin / workspace / user / m. Therefore, the computation file itself is named / home / admin / workspace / user / m / a. Here, 'home' is the main directory, 'admin' is the administrator account name, 'workspace' is the working directory, 'user' is the individual user, 'm' is the project name, and 'a' represents the computation parameters of the computation file. The main directory represents the name of the folder storing all computation files, the administrator account name represents the username with permissions to configure the management terminal, the working directory represents the path where the computation files used for work are located, the individual user represents the name of the user participating in the project, and the project name is the name of the project to which the computation file belongs. This facilitates quick retrieval of computation files. The computation allocation directory can be understood as a folder, and the computation files included in the computation allocation directory can be understood as the files included in the folder.
[0057] 220. Distribute copies of the computation files contained in each computation allocation directory to the computation terminal, so that the computation terminal can perform computations on each copy file and obtain result files associated with each computation allocation directory.
[0058] In this embodiment, the management terminal copies the computational files included in each computational allocation directory, so that for each computational allocation directory, another identical computational allocation directory and the same computational files included in that directory are obtained, i.e., copied files. The management terminal distributes the copied computational allocation directories and the copied files of the computational files included in the computational allocation directories to the computing terminal. After receiving each computational allocation directory and the copied files of the computational files included in each computational allocation directory, the computing terminal can perform calculations on the copied files of the computational files included in each computational allocation directory, and obtain result files. The result file obtained by calculating the copied files of the computational files included in a computational allocation directory is associated with that computational allocation directory.
[0059] In some embodiments, after allocating copies of computing files included in each computing allocation directory to computing terminals, the management terminal can generate a computing file resource allocation record table. This record table records the identifiers of each computing allocation directory and the computing terminals to which the copies of the computing files included in that directory are allocated. This allows the management terminal to quickly retrieve the computing allocation directories allocated to computing terminals and the copies of the computing files included in those directories based on the computing file resource allocation record table.
[0060] In some embodiments, the total file size of the computational files included in a computational allocation directory may be positively correlated with the total size of the result files associated with the computational allocation directory obtained by performing calculations on the computational files included in that computational allocation directory. Therefore, the management terminal can first determine the total size of the computational files included in each computational allocation directory, and then detect the memory usage of each computing terminal to determine its memory usage rate. Based on the total size of the computational files included in each computational allocation directory and the memory usage rate of each computing terminal, copies of the computational files contained in each computational allocation directory are allocated to the computing terminals. For example, copies of the computational files contained in computational allocation directories with larger total sizes are allocated to computing terminals with lower memory usage rates. This allows the computing terminals to store the result files for a longer period after computation, achieving reasonable allocation of copied files.
[0061] 230. If the memory usage rate of the computing terminal is detected to be greater than the first usage rate threshold, then the target computing allocation directory to be deleted is determined, and the target computing allocation directory and the result file associated with the target computing allocation directory in the computing terminal are deleted, so that the memory usage rate of the computing terminal is less than or equal to the first usage rate threshold.
[0062] In this embodiment, the management terminal can monitor and detect the memory usage of each connected computing terminal. The memory usage rate is the ratio between the total memory usage of each computing terminal and the total maximum memory of each computing terminal. After detecting the memory usage rate of a computing terminal, the management terminal determines the relationship between the memory usage rate and a preset first memory usage rate. When the detected memory usage rate is greater than the first memory usage rate threshold, a target computing allocation directory to be deleted is determined. The target computing allocation directory is the allocated computing allocation directory stored in the computing terminal. After determining the target computing allocation directory, the management terminal deletes the target computing allocation directory and the associated result files in the computing terminal, so that the memory usage rate of the computing terminal is less than or equal to the first memory usage rate threshold. Specifically, the management terminal first determines a target computing allocation directory in the computing terminal and deletes this target computing allocation directory and the result files associated with it. If the memory usage rate of the computing terminal is less than or equal to a first usage rate threshold after deletion, then there is no need to continue determining computing allocation directories. If the memory usage rate of the computing terminal is still greater than the first usage rate threshold after deletion, then the management terminal continues to determine the next target computing allocation directory and deletes the determined target computing allocation directory and the result files associated with it. This process is repeated until the memory usage rate of the computing terminal is less than or equal to the first usage rate threshold.
[0063] In some embodiments, the management terminal can sort the computing allocation directories in descending order of total size based on the total size of the result files included in each computing allocation directory in the computing terminal. Each computing allocation directory is then identified as a target computing allocation directory according to the sorting order, and the target computing allocation directories and their associated result files are deleted until the memory usage rate of the computing terminal is less than or equal to a first usage rate threshold. Deleting computing allocation directories in descending order allows the memory usage rate of the computing terminal to quickly meet the first usage rate threshold.
[0064] In some embodiments, the management terminal may receive a deletion instruction, which is used to indicate the computing allocation directory to be deleted;
[0065] Delete the computing allocation directory indicated by the deletion command in the management terminal, delete the computing allocation directory indicated by the deletion command in the computing terminal, and delete the result files associated with the indicated computing allocation directory.
[0066] In this embodiment, the deletion command can be input by the user into the management terminal. Upon receiving the deletion command, the management terminal can enter manual deletion mode. In manual deletion mode, the management terminal reads the computing allocation directory indicated by the deletion command, as well as the result files associated with that directory. Then, it deletes the computing allocation directory indicated by the deletion command and the associated result files from both the computing terminal and the management terminal. This allows for the complete removal of worthless data from the high-performance computing cluster manually, thus freeing up storage space and optimizing dynamically stored content within the cluster.
[0067] In some embodiments, after step 230, the process may further include: after a set delay period, the management terminal checks the memory usage of the computing terminal again. That is, after the management terminal allocates the target computing directory and deletes the result files associated with the target computing directory in the computing terminal, causing the memory usage to be less than or equal to a first usage threshold, the management terminal needs to wait a set delay period before checking the memory usage of the computing terminal again to determine whether the operation of allocating the target computing directory and deleting the result files associated with the target computing directory in the computing terminal needs to be performed again. By setting a delay period, storage space optimization can be performed again after a time interval following an initial storage space optimization, making the dynamic storage process more rational and avoiding frequent deletion of result files.
[0068] 240. Generate script files corresponding to each deleted result file in the computing terminal.
[0069] In this embodiment of the application, after the management terminal deletes the target computing allocation directory and the result files associated with the target computing allocation directory in the computing terminal, it can generate script files corresponding to each deleted result file.
[0070] 250. If the processor idle ratio of the computing terminal is detected to be greater than the ratio threshold, then run each script file to enable the computing terminal to recover the deleted result files.
[0071] In this embodiment, the management terminal can detect the usage of processors in the computing terminal, specifically the idle and active ratios of the processors. The idle ratio is the ratio between the total idle computing power of processors in all computing terminals and the total available computing power of processors in all computing terminals. When the management terminal detects that the idle ratio of processors in the computing terminal is greater than a threshold, it can run various script files. At this time, the computing terminal can recover deleted result files. Specifically, when the management terminal runs various script files, it sends instructions and the corresponding calculation files for the deleted result files to the computing terminal. After receiving the instructions, the computing terminal recalculates the calculation files corresponding to the deleted result files to obtain the deleted result files again, thus realizing the recovery of the deleted result files.
[0072] In one embodiment, if step 250 detects that the processor idle ratio of the computing terminal is greater than a ratio threshold, then running each script file to enable the computing terminal to recover the deleted result file may include the following steps:
[0073] If the processor idle ratio of the computing terminal is detected to be greater than the ratio threshold and the memory usage rate is less than the second usage rate threshold, then each script file is run to enable the computing terminal to restore the deleted result files one by one until the processor idle ratio of the computing terminal is detected to be greater than the ratio threshold, or the memory usage rate is greater than or equal to the second usage rate threshold.
[0074] In this embodiment, the management terminal can detect the processor usage in the computing terminal and monitor the memory usage of each connected computing terminal. The management terminal first detects the memory usage of the computing terminal. If the memory usage is less than a preset second threshold, the management terminal then detects the processor idle ratio. When the processor idle ratio is greater than a threshold, the management terminal runs various script files, allowing the computing terminal to restore deleted files one by one. After each restored file, the management terminal detects the processor idle ratio and memory usage. If the processor idle ratio is greater than a threshold, or the memory usage is greater than or equal to the second threshold, the management terminal does not need to continue running the script files, and the computing terminal stops restoring the deleted files. If the processor idle ratio of the computing terminal is still greater than the ratio threshold, and the memory utilization rate is still less than the second utilization rate threshold, then the management terminal continues to run various script files, enabling the computing terminal to continue recovering the deleted result files. The management terminal continuously runs different script files, enabling the computing terminal to continuously recover different deleted result files, until the management terminal detects that the processor idle ratio of the computing terminal is greater than the ratio threshold, or the memory utilization rate is greater than or equal to the second utilization rate threshold. This approach can comprehensively consider the storage space and computing power of the computing terminal during the result file recovery process, better realizing dynamic storage of high-performance computer clusters.
[0075] By adopting the above embodiments, the computing power of the idle central processing units and image processors of the computing terminals in the high-performance computer cluster can be used to compensate for the storage capacity of the computing terminals without expanding the capacity of the high-performance computer cluster, thereby realizing dynamic storage of result files and improving the storage capacity of the high-performance computer cluster.
[0076] In one embodiment, see Figure 4 , Figure 4 This is a flowchart illustrating another storage method disclosed in one embodiment. This method can be applied to the aforementioned management terminal 10. As shown in Figure 4, the method may include the following steps:
[0077] 410. Establish at least one computing allocation directory, and each computing allocation directory contains at least one computing file.
[0078] In this embodiment of the application, after the management terminal establishes each computing allocation directory, it can record the directory establishment time corresponding to each computing allocation directory.
[0079] 420. Distribute copies of the computation files contained in each computation allocation directory to the computation terminal, so that the computation terminal can perform computations on each copy file and obtain result files associated with each computation allocation directory.
[0080] 430. If the memory usage rate of the computing terminal is detected to be greater than the first usage rate threshold, then sort the computing allocation directories according to the order of their creation time to obtain the first directory sorting.
[0081] In this embodiment of the application, when the management terminal detects the memory usage rate of the computing terminal and determines that the detected memory usage rate is greater than the first usage rate threshold, the management terminal can sort the computing allocation directories according to the chronological order of the directory creation time, based on the recorded directory creation time, to obtain a first directory sort, which is a sequence containing the computing allocation directories.
[0082] In some embodiments, the management terminal can sort the various computing allocation directories recorded in the computing file resource allocation record table to quickly obtain the first directory sorting.
[0083] 440. Determine the target computing allocation directories to be deleted in order of sorting according to the first directory, delete the target computing allocation directories, and delete the result files associated with the target computing allocation directories in the computing terminal until the memory usage rate of the computing terminal is less than or equal to the first usage rate threshold.
[0084] In this embodiment, the management terminal sequentially determines each computing allocation directory as a target computing allocation directory according to its position in the obtained first directory sorting, and deletes the target computing allocation directory and its associated result files until the memory usage of the computing terminal is less than or equal to a first usage threshold. Specifically, the management terminal first determines the computing allocation directory ranked first in the first directory sorting, that is, the computing allocation directory with the earliest creation time, as a target computing allocation directory in the computing terminal. Then, the management terminal deletes this target computing allocation directory and its associated result files. If the memory usage of the computing terminal is less than or equal to the first usage threshold after deletion, then there is no need to continue determining computing allocation directories; if the memory usage of the computing terminal is still greater than the first usage threshold after deletion, then the management terminal determines the computing allocation directory ranked second in the first directory sorting, that is, the computing allocation directory with the second earliest creation time, as the next target computing allocation directory. Then, the management terminal deletes the determined target computing allocation directory and its associated result files, continuously repeating the process of determining, deleting, and deleting the target computing allocation directory and its associated result files until the memory usage of the computing terminal is less than or equal to a first usage threshold. Target computing allocation directories are determined one by one based on their creation time, and the target computing allocation directories and their associated result files are deleted to free up storage space on the computing terminal. This process preserves the latest result files, facilitating access to result files during project processing and enabling more efficient management of result files.
[0085] In some embodiments, after deleting each target computing allocation directory and the result files associated with each target computing allocation directory, the management terminal can establish a pending response directory record table. The pending response directory record table is used to record each deleted target computing allocation directory and the result files associated with each deleted target computing allocation directory, which can quickly determine the deleted computing allocation directories and result files.
[0086] 450. Record the deletion time corresponding to the target calculation and allocation directory.
[0087] In this embodiment, when the management terminal deletes the target computing allocation directory and the associated result files, it can record the deletion time of the target computing allocation directory. The deletion time can be either the start time of deleting the target computing allocation directory and the associated result files, or the completion time of deleting the target computing allocation directory and the associated result files. No specific limitation is made here.
[0088] 460. Generate script files corresponding to each deleted result file in the computing terminal.
[0089] 470. If the processor idle ratio of the computing terminal is detected to be greater than the ratio threshold, the target computing allocation directories to be deleted are sorted according to the order of their deletion time to obtain the second directory sorting.
[0090] In this embodiment of the application, when the management terminal detects the processor idle ratio of the computing terminal and determines that the detected processor idle ratio is greater than the ratio threshold, the management terminal can sort the deleted computing allocation directories according to the deletion time of each deleted computing allocation directory and obtain a second directory sort, which is a sequence containing each deleted computing allocation directory.
[0091] In some embodiments, the management terminal can sort the various deleted calculation allocation directories recorded in the directory record table to be replied to, so as to quickly obtain the second directory sort.
[0092] 480. Determine the target computing allocation directories to be restored in the order of the second directory, and run the script file corresponding to the result file associated with the target computing allocation directory to be restored, so that the computing terminal can restore the deleted result file.
[0093] In this embodiment of the application, the management terminal determines each deleted computing allocation directory as the target computing allocation directory to be restored according to the position of each deleted computing allocation directory in the obtained second directory sorting, and runs the script file corresponding to each result file associated with the target computing allocation directory to be restored, so that the computing terminal restores the deleted result files associated with the target computing allocation directory to be restored until the processor idle ratio of the computing terminal is less than or equal to the ratio threshold.
[0094] Specifically, the management terminal first identifies the second-ranked compute allocation directory in the second directory sorting (i.e., the compute allocation directory deleted earliest) as a target compute allocation directory to be restored. Then, the management terminal runs the script files corresponding to the result files associated with each target compute allocation directory to be restored, and the compute terminal restores the result files corresponding to the script files run by the management terminal. After all the result files associated with the target compute allocation directories to be restored have been restored, if the processor idle ratio of the compute terminal is less than or equal to a threshold, then there is no need to continue identifying target compute allocation directories to be restored. If, after deletion, the processor idle ratio of the compute terminal is still greater than the threshold, then the management terminal identifies the second-ranked compute allocation directory in the second directory sorting (i.e., the compute allocation directory deleted second earliest) as the next target compute allocation directory to be restored. The management terminal then runs the script files corresponding to the result files associated with the identified target compute allocation directory to be restored, and the compute terminal restores the result files corresponding to the script files run by the management terminal. This process of identifying target compute allocation directories to be restored and restoring the result files associated with them is repeated continuously until the processor idle ratio of the compute terminal is less than or equal to the threshold. Based on the deletion time of the deleted computing allocation directories, the target computing allocation directories to be restored are determined one by one, and the result files associated with the target computing allocation directories to be restored are restored. This allows for dynamic storage of the result files, enabling the restoration of the result files with the earliest deletion time and avoiding the result files not being restored for a long period of time during the dynamic storage process.
[0095] In one embodiment, see Figure 5 , Figure 5 This is a flowchart illustrating another storage method disclosed in one embodiment. This method can be applied to the aforementioned management terminal 10. As shown in Figure 5, the method may include the following steps:
[0096] 510. Create at least one computing allocation directory, and each computing allocation directory contains at least one computing file.
[0097] 520. Configure the preset calculation time for each calculation file. The preset calculation time refers to the length of time the computing terminal spends calculating the corresponding copy file of the calculation file.
[0098] In this embodiment, the management terminal can receive user-inputted instructions to configure the preset calculation duration for each calculation file. The preset calculation duration for each calculation file can be a uniform preset calculation duration or the preset calculation durations can be different for each file; no specific limitation is made here. The preset calculation duration refers to the length of time from the start of the calculation of the calculation file or its corresponding copy to the end of the calculation of the calculation file or its copy by the computing terminal.
[0099] 530. Distribute the copies of the computation files contained in each computation allocation directory to the computation terminal, so that the computation terminal can perform computation on each copy file according to the preset computation time corresponding to each copy file, and obtain the result file associated with each computation allocation directory.
[0100] In this embodiment, after the management terminal distributes copies of the computation files contained in each computation allocation directory to the computing terminal, the computing terminal performs calculations on each received copy file according to the preset computation time corresponding to each copy file, obtaining a result file associated with each computation allocation directory. This allows for limiting the computation time of the computation files, avoiding unnecessary long-term calculations on the files after obtaining the result files. This makes it easier for the processor idle ratio of the computing terminal to fall below a threshold, thereby increasing the likelihood of recovering deleted result files and improving dynamic storage performance.
[0101] 540. Record the actual calculation time of each result file. The actual calculation time is the shortest time it takes for the computing terminal to perform calculations on the copied file to obtain the result file.
[0102] In this embodiment, during the process of the computing terminal calculating each copied file to obtain a result file, the management terminal can record the actual calculation time for each result file. The actual calculation time refers to the shortest time from the start of calculation on the calculated file or its copy to obtaining the result file. In other words, when the computing terminal calculates the calculated file or its copy, the result file can be obtained within the actual calculation time, at which point the computing terminal stops calculating the calculated file or its copy. The actual calculation time for the result file can be obtained by the management terminal by recording the time from the start of calculation on the calculated file or its copy to obtaining the result file. The preset calculation time must be greater than the actual calculation time to ensure that the result file is obtained.
[0103] 550. Parse the filename of the calculation file corresponding to the deleted result file in the computing terminal to obtain the calculation parameters of the calculation file corresponding to the deleted result file.
[0104] In this embodiment, after deleting the result file in the computing terminal, the management terminal can parse the filename of the corresponding computing file to obtain the calculation parameters. For example, computing file 1 is used to calculate the situation of a mobile phone in a drop scenario, so the drop height and drop posture need to be considered. In this case, the drop scenario, drop height, and drop posture are all calculation parameters of computing file 1. The filename of computing file 1 is m_v1_1p5m_back_lb2rt.inp. After the management terminal deletes the result file 1 corresponding to computing file 1, the management terminal can parse this filename to obtain the drop scenario and obtain calculation parameters such as the project name (m_v1), drop height (1p5m), and drop posture (back_lb2rt). The management terminal can use Python to parse the filename.
[0105] 560. Create script files corresponding to each deleted result file, and write the actual calculation time of each deleted result file and the calculation parameters of the corresponding calculation file into the script files corresponding to each deleted result file.
[0106] In this embodiment, when creating script files corresponding to each deleted result file, the management terminal can name the script files according to the filenames of the calculation files corresponding to each deleted result file. For example, if the filename of calculation file 1 is m_v1_1p5m_back_lb2rt.inp, after deleting result file 1 corresponding to calculation file 1, the management terminal can name script file 1 as m_v1_1p5m_back_lb2rt.shell when creating script file 1 corresponding to result file 1.
[0107] After creating each script file, the management terminal writes the actual computation time of each deleted result file, along with the computation parameters of the corresponding computation files, into the script file corresponding to each deleted result file. For example, after creating a script file named m_v1_1p5m_back_lb2rt.shell, the management terminal can write computation parameters including the fall scenario, fall height (1p5m), and fall attitude (back_lb2rt), as well as the recorded actual computation time (e.g., 1 minute), into the created script file 1. Furthermore, the management terminal can also write instructions to convert model files into computation files within the created script files. This allows for the rapid construction of script files while ensuring their validity.
[0108] 570. Run each script file so that the computing terminal can restore the deleted result file based on the actual calculation time of the deleted result file corresponding to the script file and the calculation parameters of the calculation file corresponding to the deleted result file.
[0109] In this embodiment, the management terminal runs the created script files. The computing terminal then calculates the corresponding calculation file based on the calculation parameters of the deleted result file corresponding to the script file. After the actual calculation time for the deleted result file is completed, the deleted result file is obtained again. At this point, the computing terminal stops calculating the corresponding calculation file, thus restoring the deleted result file. To illustrate the principle, when the management terminal runs the created script files, it executes an instruction to convert the model file into a calculation file. The model file is converted into calculation file 1, which includes the written calculation parameters such as the drop scenario, drop height (1p5m), and drop posture (back_lb2rt). The management terminal then transmits calculation file 1 to the computing terminal. The computing terminal calculates the calculation file 1 based on the calculation parameters in calculation file 1. After the actual calculation time is completed, the computing terminal obtains result file 1, at which point the computing terminal has completed the restoration of result file 1 and can stop calculating it. This effectively limits the time required to restore the deleted result file and ensures the accuracy of the restored result file.
[0110] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of a storage device disclosed in an embodiment of this application. This storage device can be applied to... Figure 1 The management terminal in the system. For example... Figure 6 As shown, the storage device 600 may include: a directory building module 610, a file allocation module 620, a file deletion module 630, a script building module 640, and a file recovery module 650.
[0111] The directory construction module 610 is used to create at least one computing allocation directory, and each computing allocation directory contains at least one computing file;
[0112] The file allocation module 620 is used to allocate copies of the computation files contained in each computation allocation directory to the computing terminal, so that the computing terminal can perform computation on each copy file and obtain the result file associated with each computation allocation directory.
[0113] The file deletion module 630 is used to determine the target computing allocation directory to be deleted and delete the target computing allocation directory and the result file associated with the target computing allocation directory in the computing terminal if the memory usage rate of the computing terminal is detected to be greater than the first usage rate threshold, so that the memory usage rate of the computing terminal is less than or equal to the first usage rate threshold.
[0114] Script building module 640 is used to generate script files corresponding to each deleted result file in the computing terminal;
[0115] The file recovery module 650 is used to run various script files if the processor idle ratio of the computing terminal is detected to be greater than the ratio threshold, so that the computing terminal can recover the deleted result files.
[0116] In one embodiment, the file recovery module 650 is further configured to:
[0117] If the processor idle ratio of the computing terminal is detected to be greater than the ratio threshold and the memory usage rate is less than the second usage rate threshold, then each script file is run to enable the computing terminal to restore the deleted result files one by one until the processor idle ratio of the computing terminal is detected to be greater than the ratio threshold, or the memory usage rate is greater than or equal to the second usage rate threshold.
[0118] In one embodiment, the file deletion module 630 is further configured to:
[0119] The computing allocation directories are sorted according to the order in which they were created, resulting in the first directory sort.
[0120] The target computing allocation directories to be deleted are determined sequentially according to the first directory order, and the target computing allocation directories are deleted, as well as the result files associated with the target computing allocation directories in the computing terminal, until the memory usage of the computing terminal is less than or equal to the first usage threshold.
[0121] In one embodiment, the file deletion module 630 is further configured to:
[0122] Record the deletion time corresponding to the target calculation and allocation directory to be deleted;
[0123] The file recovery module 650 is also used for:
[0124] The target calculation and allocation directories are sorted according to the order of their deletion times to obtain the second directory sorting.
[0125] The target computing allocation directories to be restored are determined sequentially according to the second directory order, and the script files corresponding to the result files associated with the target computing allocation directories to be restored are run so that the computing terminal can restore the deleted result files.
[0126] In one embodiment, the file allocation module 620 is further configured to:
[0127] Configure the preset calculation time for each calculation file. The preset calculation time refers to the length of time the computing terminal spends calculating the corresponding copy file of the calculation file.
[0128] The copy files of the computation files contained in each computation allocation directory are distributed to the computation terminal, so that the computation terminal performs computation on each copy file according to the preset computation time corresponding to each copy file, and obtains the result file associated with each computation allocation directory.
[0129] In one embodiment, the file allocation module 620 is further configured to:
[0130] Record the actual computation time of each result file. The actual computation time is the shortest time it takes for the computing terminal to perform calculations on the copied file to obtain the result file.
[0131] In one embodiment, the script building module 640 is further configured to:
[0132] The filename of the calculation file corresponding to the result file deleted in the computing terminal is parsed to obtain the calculation parameters of the calculation file corresponding to the deleted result file;
[0133] Create script files corresponding to each deleted result file, and write the actual calculation time of each deleted result file and the calculation parameters of the corresponding calculation file into the script files corresponding to each deleted result file.
[0134] In one embodiment, the file recovery module 650 is further configured to:
[0135] Run each script file so that the computing terminal can recover the deleted result file based on the actual computation time of the deleted result file corresponding to the script file and the computation parameters of the computation file corresponding to the deleted result file.
[0136] Please see Figure 7 , Figure 7This is a schematic diagram of the structure of a terminal device disclosed in one embodiment. For example... Figure 7 As shown, the terminal device 700 may include:
[0137] Memory 710 that stores executable program code.
[0138] Processor 720 coupled to memory 710.
[0139] The processor 720 calls the executable program code stored in the memory 710 to execute any of the storage methods disclosed in the embodiments of this application.
[0140] It should be noted that, Figure 7 The terminal device shown may also include components not shown, such as a power supply, input buttons, camera, speaker, screen, RF circuit, Wi-Fi module, and Bluetooth module, which will not be described in detail in this embodiment.
[0141] This application discloses a computer-readable storage medium storing a computer program, wherein the computer program causes a computer to execute any of the storage methods disclosed in this application.
[0142] This application discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute any of the storage methods disclosed in this application.
[0143] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Those skilled in the art should also recognize that the embodiments described in the specification are optional embodiments, and the actions and modules involved are not necessarily essential to this application.
[0144] In the various embodiments of this application, it should be understood that the sequence number of each process does not necessarily imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.
[0145] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they can be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0146] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0147] If the aforementioned integrated units are implemented as software functional units and sold or used as independent products, they can be stored in a computer-accessible memory. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several requests to cause a computer device (which can be a personal computer, server, or network device, specifically a processor in the computer device) to execute some or all of the steps of the methods described in the various embodiments of this application.
[0148] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.
[0149] The storage methods, apparatus, terminal devices, and storable media disclosed in the embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. A storage method characterized by comprising: The application is applied to a management terminal, which is in communication connection with at least one computing terminal; the method comprises: establishing at least one computing allocation directory, each of which comprises at least one computing file; allocating a copy file of each computing file comprised in each computing allocation directory to the computing terminal, so that the computing terminal performs computation on each copy file to obtain a result file associated with each computing allocation directory; if it is detected that the memory occupancy rate of the computing terminal is greater than a first occupancy threshold, determining a target computing allocation directory to be deleted, deleting the target computing allocation directory, and deleting a result file associated with the target computing allocation directory in the computing terminal, so that the memory occupancy rate of the computing terminal is less than or equal to the first occupancy threshold; generating a script file corresponding to each deleted result file in the computing terminal; if it is detected that the processor idle proportion of the computing terminal is greater than a proportion threshold, running each script file, so that the computing terminal restores the deleted result file.
2. The method of claim 1, wherein, The method comprises: if it is detected that the processor idle proportion of the computing terminal is greater than a proportion threshold and the memory occupancy rate is less than a second occupancy threshold, running each script file, so that the computing terminal restores the deleted result file one by one until it is detected that the processor idle proportion of the computing terminal is greater than a proportion threshold or the memory occupancy rate is greater than or equal to the second occupancy threshold.
3. The method of claim 1, wherein, The method comprises: sorting each computing allocation directory according to the directory establishment time corresponding to each computing allocation directory in chronological order to obtain a first directory sorting; determining a target computing allocation directory to be deleted according to the first directory sorting in sequence, deleting the target computing allocation directory, and deleting a result file associated with the target computing allocation directory in the computing terminal until the memory occupancy rate of the computing terminal is less than or equal to the first occupancy threshold.
4. The method of claim 3, wherein, After the target computing allocation directory is deleted, the method further comprises: recording a deletion time corresponding to the deleted target computing allocation directory; The method comprises: sorting each deleted target computing allocation directory according to the deletion time corresponding to each deleted target computing allocation directory in chronological order to obtain a second directory sorting; running each script file, so that the computing terminal restores the deleted result file. The target computing allocation directory to be recovered is determined in sequence according to the second catalog sorting, and a script file corresponding to the result file associated with the target computing allocation directory to be recovered is run, so that the computing terminal recovers the deleted result file.
5. The method of claim 1, wherein, Before the step of allocating the copy file of the computing file contained in each computing allocation directory to the computing terminal, so that the computing terminal computes each copy file to obtain the result file associated with each computing allocation directory, the method further comprises: configuring a pre-designed computing duration of each computing file, wherein the pre-designed computing duration refers to a time length for the computing terminal to compute the copy file corresponding to the computing file; the step of allocating the copy file of the computing file contained in each computing allocation directory to the computing terminal, so that the computing terminal computes each copy file to obtain the result file associated with each computing allocation directory, comprises: allocating the copy file of the computing file contained in each computing allocation directory to the computing terminal, so that the computing terminal computes each copy file according to the pre-designed computing duration corresponding to each copy file to obtain the result file associated with each computing allocation directory.
6. The method according to any one of claims 1 to 5, characterized in that, After the step of allocating the copy file of the computing file contained in each computing allocation directory to the computing terminal, so that the computing terminal computes each copy file to obtain the result file associated with each computing allocation directory, the method further comprises: recording an actual computing duration of each result file, wherein the actual computing duration is a shortest time length for the computing terminal to compute the copy file to obtain the result file; the step of generating the script file corresponding to each deleted result file in the computing terminal, comprises: parsing the file name of the computing file corresponding to the deleted result file in the computing terminal to obtain the computing parameter of the computing file corresponding to the deleted result file; creating the script file corresponding to each deleted result file, and writing the actual computing duration of each deleted result file and the computing parameter of the computing file corresponding to each deleted result file into the script file corresponding to each deleted result file, respectively.
7. The method of claim 6, wherein, the step of running each script file, so that the computing terminal recovers the deleted result file, comprises: running each script file, so that the computing terminal recovers the deleted result file according to the actual computing duration of the deleted result file corresponding to the script file and the computing parameter of the computing file corresponding to the deleted result file.
8. A memory device, comprising: comprises: a catalog construction module configured to establish at least one computing allocation directory, wherein each computing allocation directory contains at least one computing file; a file allocation module configured to allocate the copy file of the computing file contained in each computing allocation directory to the computing terminal, so that the computing terminal computes each copy file to obtain the result file associated with each computing allocation directory; and a script file generation module configured to generate the script file corresponding to each deleted result file in the computing terminal. The file deletion module is configured to determine a target computing allocation directory to be deleted if it is detected that the memory usage of the computing terminal is greater than a first usage threshold, delete the target computing allocation directory, and delete result files associated with the target computing allocation directory in the computing terminal, so that the memory usage of the computing terminal is less than or equal to the first usage threshold. The script construction module is configured to generate a script file corresponding to each deleted result file in the computing terminal. The file recovery module is configured to run each script file if it is detected that the processor idle ratio of the computing terminal is greater than a ratio threshold, so that the computing terminal recovers the deleted result files.
9. A terminal device, comprising: The terminal device comprises a memory and a processor, the memory stores a computer program, and the computer program is executed by the processor to enable the processor to implement the method of any one of claims 1 to 7.
10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the method of any one of claims 1 to 7.
Citation Information
Patent Citations
Quick cluster deployment method based on Linux operation system
CN104572269A
Cloud system, management node and data obtaining method
CN105245565A