Cluster storage resource timed cleaning method, device and storage medium

Abstract

The present application relates to a method, device and storage medium for regularly cleaning up cluster storage resources, the method comprising: an electronic device scans cluster operation statements submitted within a set time interval, identifies the table headers and target table headers used in all operation statements, and determines a table with a specific beginning; the electronic device integrates the table with the specific beginning and the subdirectory of a preset format to obtain a multi-level partitioned table and a multi-level partitioned subdirectory; the electronic device arranges the multi-level partitioned table and the multi-level partitioned subdirectory in reverse order according to storage volume, and performs a cleaning operation on the table and the subdirectory in the order of the reverse arrangement. The technical solution provided by the present application regularly cleans up cluster storage resources, thereby reducing the storage volume of historical data, and has the advantage of low cost.

Description

Technical Field

[0001] The present application relates to the field of big data technology, and particularly to a method, device and storage medium for periodically cleaning cluster storage resources. Background Art

[0002] Nowadays, there are more and more industries and technical fields demanding big data analysis systems. For example, the financial industry needs to use big data systems combined with VaR (Value at Risk) or machine learning solutions for credit risk control. The retail and catering industries need big data systems to achieve auxiliary sales decision-making. Various application scenarios require big data systems to continuously aggregate and analyze time-series data. Major technology companies need to establish big data analysis middle platforms, etc. Abstractly speaking, the analysis systems supporting these scenario requirements face roughly the same technical challenges. The data scope for business analysis spans across real-time data and historical data, and both low-latency real-time data analysis and exploratory data analysis of a large amount of historical data are required. Regarding reliability and scalability issues, users may store a large amount of historical data, and at the same time, the data scale has a continuous growth trend. Therefore, the existing data resources have a large storage capacity, which greatly increases the cost. Summary of the Invention

[0003] Embodiments of the present application provide a method, device and storage medium for periodically cleaning cluster storage resources, which can periodically clean cluster storage resources, thereby reducing the storage amount of historical data and lowering the data storage cost.

[0004] In a first aspect, a method for periodically cleaning cluster storage resources is provided, which is applied to an electronic device. The method includes:

[0005] The electronic device scans the operation statements of the cluster submitted within a set time interval, identifies the headers of the tables used in all operation statements and the headers of the target tables, and determines the tables with a specific start.

[0006] The electronic device integrates the tables with the specific start and subdirectories in a preset format to obtain a multi-level partitioned table and a multi-level partitioned subdirectory.

[0007] The electronic device sorts the multi-level partitioned table and the multi-level partitioned subdirectory in descending order of storage capacity, and performs a cleaning operation on the table and the subdirectory in the order of the descending sort.

[0008] In a second aspect, an electronic device is provided. The electronic device includes:

[0009] A scanning and identifying unit, configured to scan the operation statements of the cluster submitted within a set time interval, identify the headers of the tables used in all operation statements and the headers of the target tables, and determine the tables with a specific start.

[0010] A processing unit for integrating the table with a specific start and subdirectories in a preset format to obtain a multi-level partitioned table and multi-level partitioned subdirectories; sorting the multi-level partitioned table and multi-level partitioned subdirectories in descending order of storage capacity, and performing a cleaning operation on the table and subdirectories in the order of the reverse order

[0011] In a third aspect, an embodiment of the present application provides an electronic device, including a processor, a memory, a communication interface, and one or more programs. Among them, the above one or more programs are stored in the above memory and are configured to be executed by the above processor. The above programs include instructions for performing the steps in the first aspect of the embodiment of the present application.

[0012] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium. Among them, the above computer-readable storage medium stores a computer program for electronic data exchange. Among them, the above computer program enables a computer to execute some or all of the steps described in the first aspect of the embodiment of the present application.

[0013] In a fifth aspect, an embodiment of the present application provides a computer program product. Among them, the above computer program product includes a non-transitory computer-readable storage medium storing a computer program. The above computer program is operable to enable a computer to execute some or all of the steps described in the first aspect of the embodiment of the present application. This computer program product can be a software installation package.

[0014] Implementing the embodiments of the present application has the following beneficial effects:

[0015] It can be seen that the technical solution provided by the present application reduces redundant data and the cost of data storage by performing a regular cleaning operation on the operation statements of the cluster. Therefore, it has the advantage of reducing costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0017] Figure 1 is a schematic structural diagram of an electronic device provided by an embodiment of the present application;

[0018] Figure 2 is a schematic flowchart of a method for periodically cleaning cluster storage resources provided by an embodiment of the present application;

[0019] Figure 3It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0020] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without making creative efforts belong to the scope of protection of the present application.

[0021] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products or devices.

[0022] Referring to "embodiment" herein means that a specific feature, structure or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0023] The technical solution provided by the embodiment of the present application is executed based on the hardware structure of the electronic device. The electronic device can be a portable electronic device that also includes other functions such as a personal digital assistant and / or a music player function, such as a mobile phone, a tablet computer, a wearable electronic device with wireless communication function (such as a smart watch), etc. Of course, it can also include a fixed electronic device, such as a traffic camera, a surveillance camera, etc. Exemplary embodiments of the portable electronic device and the fixed electronic device include, but are not limited to, portable electronic devices or fixed electronic devices running the IOS system, the Android system, the Microsoft system or other operating systems (such as an embedded operating system). The above-mentioned portable electronic device can also be other portable electronic devices, such as a laptop computer. It should also be understood that in some other embodiments, the above-mentioned electronic device may not be a portable electronic device, but a desktop computer or a fixed camera.

[0024] An electronic device is a device deployed in an indoor or outdoor environment for signal transmission and reception. For example, the signal transceiver device can be an evolved Node B (eNB), a radio network controller (RNC), a Node B (NB), a Base Station Controller (BSC), a Base Transceiver Station (BTS), a home base station (e.g., Home evolved Node B, or Home Node B, HNB), an access controller (AC), a WIFI access point (Access Point, AP), etc.

[0025] The software and hardware operating environment of the technical solution disclosed in this application is introduced as follows.

[0026] Exemplarily, Figure 1 A schematic structural diagram of the electronic device 100 is shown. The electronic device can specifically be an intelligent camera. Of course, the above-mentioned electronic device can also be a wearable device, specifically, wearable portable devices such as smart watches and smart bracelets. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, antennas 1, 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone interface 170D, a sensor module 180, a compass 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0027] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent components or integrated in one or more processors. In some embodiments, the electronic device 100 may also include one or more processors 110. Among them, the controller may generate operation control signals according to the instruction operation code and timing signals to complete the control of fetching and executing instructions. In some other embodiments, a memory may also be provided in the processor 110 for storing instructions and data. Exemplarily, the memory in the processor 110 may be a cache memory. This memory may save the instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can be directly called from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the electronic device 100 in processing data or executing instructions.

[0028] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM card interface, and / or a USB interface, etc. Among them, the USB interface 130 is an interface that complies with the USB standard specification, and specifically may be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 101, and can also be used to transfer data between the electronic device 101 and peripheral devices. The USB interface 130 can also be used to connect headphones to play audio through the headphones.

[0029] It can be understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is only for illustrative purposes and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0030] Refer to Figure 2 , Figure 2 A method for periodically cleaning cluster storage resources is provided. This method is executed by an electronic device as shown in Figure 1 The method is as shown in Figure 2 and includes the following steps:

[0031] Step S201: The electronic device scans the operation statements of the cluster submitted within a set time interval, identifies the headers of the tables used in all operation statements and the headers of the target tables, and determines the tables with a specific start.

[0032] Examples of the above operation statements include query (select), create table (create), update, etc.

[0033] The tables with a specific start may specifically be tables starting with from, join, etc., and tables starting with table, into, etc.

[0034] Step S202: The electronic device obtains the file sizes of each sub-directory on the cluster storage resource and formats the information of each sub-directory into a preset format.

[0035] The above preset format may specifically include: database name, table name, 5-level partition, storage capacity (M).

[0036] Step S203: The electronic device integrates the tables with the specific start and the sub-directories in the preset format to obtain a multi-level partitioned table and multi-level partitioned sub-directories.

[0037] Step S204: The electronic device sorts the multi-level partitioned table and multi-level partitioned sub-directories in descending order of storage capacity, and performs a cleaning operation on the table and sub-directories in the order of the descending sort.

[0038] The technical solution provided by this application reduces redundant data and the cost of data storage by regularly performing a cleaning operation on the operation statements of the cluster. Therefore, it has the advantage of reducing costs.

[0039] In an alternative solution, the above-mentioned performing a cleaning operation on the table and sub-directories in the order of the descending sort may specifically include:

[0040] Extracting the status of the tables with the specific start in the table in the order of the descending sort. If the status of the tables with the specific start is unused and not updated, the deletion priority of the tables with the specific start is determined as the first priority.

[0041] Extracting the status of the tables with the specific start in the table in the order of the descending sort. If the status of the tables with the specific start is unused, the deletion priority of the tables with the specific start is determined as the second priority.

[0042] Extracting the status of the tables with the specific start in the table in the order of the descending sort. If the status of the tables with the specific start is not updated, the deletion priority of the tables with the specific start is determined as the third priority.

[0043] In an alternative solution, the above method may further include:

[0044] Notifying the tables with the first priority, the second priority, and the third priority to the person in charge corresponding to the table. The above notification methods include, but are not limited to, instant messaging software, emails, text messages, office systems, and other methods.

[0045] In an alternative solution, the above method may further include:

[0046] The electronic device receives the feedback information from the person in charge. If the feedback information contains a cleaning command, the table corresponding to the feedback information will be deleted; if the feedback information from the person in charge is not received, the electronic device deletes the table with the corresponding priority according to the order of deletion priorities.

[0047] The above-mentioned electronic device deleting the table with the corresponding priority according to the order of deletion priorities may specifically include:

[0048] The electronic device directly deletes the table with the first priority. After one week, it deletes the table with the second priority. After two weeks, it deletes the table with the third priority.

[0049] Of course, the above time is only for illustration. In actual applications, it can also be other time periods, such as one day, one week, one hour, one month, etc. Here, it only needs to ensure that the retention time of the first priority is less than that of the second priority, and the retention time of the second priority is less than that of the third priority.

[0050] Of course, in actual applications, the above-mentioned cleaning operation for the table also applies to the operation of the subdirectory.

[0051] In an alternative solution, the above-mentioned confirmation method for the update may specifically include: if the table with the specific start is updated and the update is reasonable, determine the status as updated; if the table is updated but the update is unreasonable, determine the status as not updated. Of course, if the table is not updated, directly determine the status as not updated.

[0052] The above-mentioned reasonable update may include the update of the dimension table, and the unreasonable update may include: the update task is suspended, the update is not discovered, etc.

[0053] In an alternative solution, the above-mentioned notifying the table with the first priority, the table with the second priority or the table with the third priority to the person in charge corresponding to the table may specifically include:

[0054] Determine the first identity of the person in charge, extract the first email corresponding to the first identity, and send the table with the first priority, the table with the second priority or the table with the third priority to the first email.

[0055] The above-mentioned determining the first identity of the person in charge may specifically include:

[0056] E1. Obtain the target face image of the target image of the target object (i.e., the person in charge);

[0057] E2. Verify the target face image;

[0058] E3. When the target face image is verified and passed, determine that the target object is the first identity corresponding to the preset face module.

[0059] In specific implementation, a preset face template can be pre-stored in the electronic device. The original image of the target object can be obtained through a camera. Furthermore, when the target face image matches the preset face template successfully, the electronic device can determine the first identity of the target object; otherwise, it cannot determine the first identity of the target object. In this way, the identity of the target object can be recognized, and then it can be determined whether the first identity is a reserved patient, avoiding others from initiating telemedicine.

[0060] Further, in a possible example, step E2 of verifying the target face image may include the following steps:

[0061] E21. Perform region segmentation on the target face image to obtain a target face region, where the target face region is a region image that only contains a face.

[0062] E22. Perform binarization processing on the target face region to obtain a binarized face image.

[0063] E23. Divide the binarized face image into multiple regions, where the area of each region is the same and the area size is greater than a preset area value.

[0064] E24. Extract feature points from the binarized face image to obtain multiple feature points.

[0065] E25. Determine the feature point distribution density corresponding to each region in the multiple regions according to the multiple feature points to obtain multiple feature point distribution densities.

[0066] E26. Determine the target mean square error according to the multiple feature point distribution densities.

[0067] E27. Determine the target quality evaluation value corresponding to the target mean square error according to the mapping relationship between the preset mean square error and the quality evaluation value.

[0068] E28. When the target quality evaluation value is less than the preset quality evaluation value, perform image enhancement processing on the target face image, and match the target face image after image enhancement processing with the preset face template to obtain a matching value.

[0069] E29. When the matching value is greater than the preset threshold, determine that the target face image is verified and passed.

[0070] In a specific implementation, the above-mentioned preset threshold and preset area value can both be set by the user himself or be the system default. The preset face template can be pre-stored in the electronic device. The electronic device can obtain the target face image for region segmentation to obtain the target face region. The target face region can be a region that does not include the background but only contains the face, that is, an image of the region that is only the face. Furthermore, the target face region can be binarized to obtain a binarized face image. In this way, the image complexity can be reduced. The binarized face image is divided into multiple regions, and the area size of each region is equal and greater than the preset area value. Further, the binarized face image can be subjected to feature point extraction to obtain multiple feature points. The algorithms for feature extraction can be at least one of the following: Scale Invariant Feature Transform (SIFT), SURF algorithm, pyramid algorithm, harris corner detection, etc., which are not limited here.

[0071] Furthermore, the electronic device can determine the feature point distribution density corresponding to each region in the multiple regions according to the multiple feature points to obtain multiple feature point distribution densities, and determine the target mean square error according to the multiple feature point distribution densities. The mapping relationship between the preset mean square error and the quality evaluation value can be pre-stored in the electronic device. According to the mapping relationship between the preset mean square error and the quality evaluation value, determine the target quality evaluation value corresponding to the target mean square error. Of course, the smaller the mean square error, the greater the quality evaluation value. When the target quality evaluation value is greater than the preset quality evaluation value, directly match the target face image with the preset face template. When the matching value between them is greater than the preset threshold, confirm that the target face image is verified successfully; otherwise, confirm that the target face image is verified failed.

[0072] Furthermore, when the target quality evaluation value is less than the preset quality evaluation value, the terminal can perform image enhancement processing on the target face image, match the target face image after image enhancement processing with the preset face template. When the matching value between them is greater than the preset threshold, confirm that the target face image is verified successfully; otherwise, confirm that the target face image is verified failed.

[0073] The above-mentioned method for extracting the weight data corresponding to the first identity can specifically be to preset and configure the weight data corresponding to multiple identities, and extract the weight data corresponding to the first identity.

[0074] In an alternative solution, the above-mentioned determination of the first identity of the person in charge may further include:

[0075] Obtain a picture of the target object (i.e., the person in charge), establish input data based on the picture of the target object, input the input data into the face recognition model to perform n-layer convolution operations to obtain the nth layer convolution operation result, input the nth layer convolution operation result into the fully connected layer to perform fully connected operations to obtain the fully connected calculation result, calculate the difference between the fully connected calculation result and the preset face template result. If the difference is less than the difference threshold, the UE determines that the identity of the target object is the identity of the preset face template.

[0076] In an alternative solution, the above-mentioned inputting the input data into the face recognition model to perform n-layer convolution operations to obtain the nth layer convolution operation result may specifically include:

[0077] The above UE may be provided with a separate AI chip, which is used to perform the authentication of the above target object identity. The AI chip includes: an allocation computing circuit and x computing circuits. The AI chip obtains the matrix size CI*CH of the input data. If the convolution kernel size in the n-layer convolution operation is a 3*3 convolution kernel, the allocation computing circuit divides CI*CH into CI / x data blocks in the CI direction (assuming CI is an integer multiple of x), and sequentially allocates the CI / x data blocks to the x computing circuits. The x computing circuits respectively perform the i-th layer convolution operation on the received 1 data block and the i-th layer convolution kernel to obtain the i-th convolution result (i.e., combining the x result matrices (CI / x - 2)*(CH - 2) of the x computing circuits in sequence to obtain the i-th convolution result), and send the results of the 2 columns at the edge of the i-th convolution result (the 2 columns where the results of adjacent columns are calculated by different computing circuits are determined as the edge columns) to the allocation processing circuit. The x computing circuits perform the convolution operation on the i-th layer convolution result and the (i + 1)-th layer convolution kernel to obtain the (i + 1)-th convolution result, and send the (i + 1)-th convolution result to the allocation computing circuit. The allocation computing circuit performs the i-th layer convolution operation on the (CI / x - 1) combined data blocks and the i-th layer convolution kernel to obtain the i-th combined result, splices the i-th combined result with the results of the 2 columns at the edge of the i-th convolution result (inserting the i-th combined result into the middle of the 2 edge columns according to the mathematical rules of the convolution operation) to obtain the (i + 1)-th combined data block, performs the convolution operation on the (i + 1)-th combined data block and the (i + 1)-th layer convolution kernel to obtain the (i + 1)-th combined result, and inserts the (i + 1)-th combined result between the edge columns of the (i + 1)-th convolution result (the results of adjacent columns are calculated by different computing circuits) to obtain the (i + 1)-th layer convolution result. The AI chip performs the operations of the remaining convolution layers (convolution kernels after the (i + 1)-th layer) based on the (i + 1)-th layer convolution result to obtain the n-th layer convolution operation result. The above combined data block may be a 4*CI matrix composed of 4 columns of data between two adjacent data blocks. For example, a 4*CH matrix composed of the last 2 columns of the first data block (the data block allocated to the first computing circuit) and the first 2 columns of the second data block (the data block allocated to the second computing circuit).

[0078] For the operations of the above remaining convolution layers, reference may also be made to the calculations of the i-th layer and the (i + 1)-th layer, where i is an integer greater than or equal to 1 and less than or equal to n. The above n is the total number of convolution layers of the AI model, and i is the layer number of the convolution layer. The CI is the column value of the matrix, and the CH is the row value of the matrix.

[0079] It can be understood that, in order to implement the above functions, the electronic device includes the corresponding hardware and / or software modules for executing each function. Combining the algorithm steps of each example described in the embodiments disclosed in this article, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the manner of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to exceed the scope of the present application.

[0080] In this embodiment, the electronic device can be divided into functional modules according to the above method examples. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is illustrative, only a logical function division, and there may be other division methods in actual implementation.

[0081] In the case of dividing each functional module corresponding to each function, Figure 3 The electronic device is shown, and the device includes:

[0082] A scanning and recognition unit 301, configured to scan the operation statements of the cluster submitted within a set time interval, identify the headers of the tables used in all the operation statements and the headers of the target tables, and determine the tables with a specific start;

[0083] A processing unit 302, configured to integrate the tables with the specific start and the subdirectories in a preset format to obtain a multi-level partitioned table and a multi-level partitioned subdirectory; sort the multi-level partitioned table and the multi-level partitioned subdirectory in descending order of storage capacity, and perform a cleaning operation on the table and the subdirectory in the order of the descending order.

[0084] Among them, the scanning and recognition unit 301 can be used to support the electronic device to execute the above step 201 and / or other processes of the technology described in this article, and the processing unit 302 is used to support the electronic device to execute the above step 202, step S203, step S204 and / or other processes of the technology described in this article.

[0085] It should be noted that all relevant contents of each step involved in the above method embodiment can be cited in the function description of the corresponding functional module, and will not be repeated here.

[0086] The electronic device provided in this embodiment is used to execute the above position determination method, so the same effect as the above implementation method can be achieved.

[0087] In the case of adopting an integrated unit, an electronic device may include a processing module, a storage module, and a communication module. Among them, the processing module may be used to control and manage the operations of the electronic device. For example, it may be used to support the electronic device to execute the steps performed by the above-mentioned scanning and recognition unit 301 and processing unit 302. The storage module may be used to support the electronic device to execute storing program codes and data, etc. The communication module may be used to support the communication between the electronic device and other devices.

[0088] Among them, the processing module may be a processor or a controller. It may implement or execute various exemplary logical blocks, modules, and circuits described in connection with the disclosure of the present application. The processor may also be a combination that implements computing functions, such as a combination including one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, and so on. The storage module may be a memory. The communication module may specifically be a device for interacting with other electronic devices, such as a radio frequency circuit, a Bluetooth chip, a Wi-Fi chip, etc.

[0089] In one embodiment, when the processing module is a processor and the storage module is a memory, the electronic device involved in this embodiment may be a device with Figure 1 the shown structure.

[0090] The embodiment of the present application also provides an electronic device, including a processor and a memory. The memory is used to store one or more programs and is configured to be executed by the processor. The programs include instructions for performing the following steps, and the following steps may specifically include:

[0091] Obtain the characteristic information of the agent, and determine the scope of service agents based on this characteristic information;

[0092] Obtain the accuracy of the address resolution of the customer identifier, match the agent in the area corresponding to this accuracy within the scope of service agents, and assign the customer identifier to the agent in the corresponding area.

[0093] The instructions executed by the above programs may also include the refined solutions of steps S201 and S202 as shown in Figure 2 and of course the optional solutions in the embodiments as shown in Figure 2 are not elaborated here.

[0094] The embodiment of the present application also provides a computer storage medium. Among them, this computer storage medium stores a computer program for electronic data exchange, and this computer program enables a computer to execute some or all of the steps of any method recorded in the above method embodiments.

[0095] The embodiments of the present application also provide a computer program product. The computer program product includes a non-transitory computer-readable storage medium storing a computer program. The computer program is operable to cause a computer to execute some or all of the steps of any of the methods described in the foregoing method embodiments. The computer program product may be a software installation package.

[0096] It should be noted that, for the foregoing method embodiments, for simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should understand that the present application is not limited by the described order of actions, because according to the present application, some steps may be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the present application.

[0097] In the above embodiments, the descriptions of the respective embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0098] In several embodiments provided by the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the above division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical or other form.

[0099] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0100] In addition, in each embodiment of the present application, the functional units can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0101] If the above integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the above methods in various embodiments of the present application. The aforementioned memory includes: USB flash drives, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), mobile hard disks, magnetic disks, or optical discs, etc., which are various media that can store program codes.

[0102] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program. This program can be stored in a computer-readable memory, and the memory can include: flash drives, read-only memories (English: Read-Only Memory, abbreviated: ROM), random access memories (English: Random Access Memory, abbreviated: RAM), magnetic disks, or optical discs, etc.

[0103] The above has introduced the embodiments of the present application in detail. Specific examples are used in this article to elaborate on the principles and implementation manners of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A method for regularly cleaning up cluster storage resources, characterized in that: Applied to electronic equipment, the method comprises: The electronic device scans the operation statements of the cluster submitted within a set time interval, identifies the headers of the tables used in all the operation statements and the headers of the target tables, and determines the table with a specific header; The electronic device obtains the file size of each subdirectory on the cluster storage resource, and formats the information of each subdirectory into a preset format; The electronic device integrates the table with the specific beginning and the subdirectory with the preset format to obtain a multi-level partitioned table and a multi-level partitioned subdirectory; The electronic device arranges multi-level partitioned tables and multi-level partitioned sub-directories in reverse order according to storage capacity, and performs a cleaning operation on the tables and sub-directories in the order of the reverse arrangement. The cleaning operation on the tables and sub-directories in the order of the reverse arrangement specifically includes: extracting the status of a table or sub-directory with a specific beginning in the table in the order of the reverse arrangement, and if the status of the table or sub-directory with the specific beginning is unused and not updated, determining the deletion priority of the table or sub-directory with the specific beginning as the first priority.

2. The method according to claim 1, characterized in that The cleaning operation of the table and the subdirectory according to the reverse order specifically includes: The status of a table or subdirectory with a specific beginning in the table is extracted in reverse order. If the status of the table or subdirectory with a specific beginning is unused, the deletion priority of the table or subdirectory with the specific beginning is determined to be the second priority.

3. The method according to claim 1, characterized in that The cleaning operation of the table and the subdirectory according to the reverse order specifically includes: The status of a table or subdirectory with a specific beginning in the table is extracted in reverse order. If the status of the table or subdirectory with a specific beginning is not updated, the deletion priority of the table or subdirectory with the specific beginning is determined to be the third priority.

4. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: The first priority table, the second priority table, or the third priority table is notified to a person in charge of the corresponding table.

5. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: The electronic device directly performs a deletion operation on the table of the first priority, performs a deletion operation on the table of the second priority after a first time interval, and performs a deletion operation on the table of the third priority after a second time interval; The first time interval is smaller than the second time interval.

6. The method according to claim 4, characterized in that The method further comprises: The first identity of the person in charge is determined, a first mailbox corresponding to the first identity is extracted, and the first priority form, the second priority form, or the third priority form is sent to the first mailbox.

7. An electronic device, the device being used to execute the method according to any one of claims 1 to 6, characterized in that: The electronic device comprises: A scanning and identifying unit is used to scan the operation statements of the cluster submitted within a set time interval, identify the headers of the tables used in all the operation statements and the headers of the target tables, and determine the tables with specific headers; The processing unit is used to integrate the table with the specific beginning and the sub-directory of the preset format to obtain a multi-level partitioned table and a multi-level partitioned sub-directory; arrange the multi-level partitioned table and the multi-level partitioned sub-directory in reverse order according to the storage capacity, and perform a cleaning operation on the table and the sub-directory in the order of the reverse arrangement.

8. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory is used to store one or more programs and is configured to be executed by the processor, wherein the program comprises instructions for executing the steps in the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 6.

Images