Method, device, electronic device and storage medium for executing computing tasks

Abstract

The present invention discloses a method, apparatus, electronic device, and storage medium for executing a computing task. The method includes: in response to receiving an execution instruction for a computing task, obtaining a preset file name for a file required for the computing task; determining a file block list corresponding to the preset file name, comparing a target file name corresponding to at least one file block in the file block list with the preset file name, and obtaining a first comparison result, wherein the first comparison result indicates whether the target file block required for the computing task exists in the file block list; and executing the computing task based on the first comparison result to obtain an execution result, wherein the execution result indicates whether the computing task was successfully executed. The present invention solves the technical problem of low computing task execution efficiency in related technologies.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a method, device, electronic device and storage medium for executing a computing task. Background Art

[0002] When a computing task is executed, the address information of the data file needs to be obtained. In the file system, the file is divided into a series of blocks. Each block has a fixed size and is used to store the content of the file. The data file is stored in multiple storage spaces or data units, namely data blocks or file blocks. When a computing task is executed, the address information of each data block corresponding to the data file needs to be determined.

[0003] Since the files required for a computing task may be stored in multiple file blocks, the file blocks may not be continuous in storage. Currently, the methods for obtaining the address information of the file blocks required for a computing task in related technologies are relatively complicated, resulting in a longer time to obtain the files required for the computing task and lower execution efficiency of the computing task.

[0004] To address the above-mentioned problems, no effective solutions have been proposed so far. Summary of the Invention

[0005] Embodiments of the present invention provide a method, device, electronic device, and storage medium for executing a computing task, so as to at least solve the technical problem of low execution efficiency of computing tasks in related technologies.

[0006] According to one aspect of an embodiment of the present invention, a method for executing a computing task is provided, comprising: in response to receiving an execution instruction of the computing task, obtaining a preset file name of a file required for the computing task; determining a file block list corresponding to the preset file name, comparing a target file name corresponding to at least one file block in the file block list with the preset file name, and obtaining a first comparison result, wherein the first comparison result is used to indicate whether the target file block required for the computing task exists in the file block list; executing the computing task based on the first comparison result to obtain an execution result, wherein the execution result is used to indicate whether the computing task is successfully executed.

[0007] Optionally, a computing task is executed based on the first comparison result to obtain an execution result, including: if the first comparison result is that the target file block exists in the file block list, determining the host address information of the target file block, and calling the target data in the target file block based on the host address information of the target file block to execute the computing task to obtain the execution result; if the first comparison result is that the target file block does not exist in the file block list, determining that the execution result is that the computing task was not successfully executed.

[0008] Optionally, the target file name and preset file name corresponding to at least one file block in the file block list are compared to obtain a first comparison result, including: using a Bloom filter to determine whether the preset file name exists in the target file name; if it is determined that the preset file name does not exist in the target file name, determining that the first comparison result is that the target file block required for the computing task does not exist in the file block list; if it is determined that the preset file name exists in the target file name, determining the first comparison result based on the shard information contained in the preset file name, wherein the shard information is used to indicate information about the shard in which the file block corresponding to the preset file name is located.

[0009] Optionally, determining a first comparison result based on the fragment information contained in the preset file name includes: obtaining the fragment name and fragment serial number in the fragment information; constructing character information of the preset file name based on the fragment name, wherein the character information is used to represent information of characters contained in the preset file name; constructing a preset sub-bitmap of the character information based on the fragment serial number, wherein the preset sub-bitmap is used to represent the character information through binary data; and determining the first comparison result based on the preset sub-bitmap and the target sub-bitmap corresponding to the target file name.

[0010] Optionally, determining a first comparison result based on a preset sub-bitmap and a target sub-bitmap corresponding to a target file name includes: comparing the preset sub-bitmap and the target sub-bitmap corresponding to the target file name to obtain a second comparison result, wherein the second comparison result is used to indicate whether there is a sub-bitmap identical to the preset sub-bitmap in the target sub-bitmap; if the second comparison result is that there is a sub-bitmap identical to the preset sub-bitmap in the target sub-bitmap, determining that the first comparison result is that the target file block exists in the file block list, and determining that the file block corresponding to the sub-bitmap in the file block list is the target file block; if the second comparison result is that there is no sub-bitmap identical to the preset sub-bitmap in the target sub-bitmap, determining that the first comparison result is that the target file block does not exist in the file block list.

[0011] Optionally, character information of a preset file name is constructed based on the shard name, including: if a target value contained in the shard name is greater than a preset value, an array is constructed based on the target value, and the array is determined to be character information; if the target value is less than or equal to the preset value, a string is constructed based on the target value, and the string is determined to be character information.

[0012] Optionally, the target data in the target file block is called based on the host address information of the target file block to execute the computing task and obtain the execution result, including: obtaining at least one computing node corresponding to the computing task, and constructing a host topology map based on the host address information; using the host topology map to call the target data in the target file block, and pushing the target data to at least one computing node to execute the computing task and obtain the execution result.

[0013] According to another aspect of an embodiment of the present invention, a device for executing a computing task is also provided, including: an acquisition module, used to obtain a preset file name of a file required for the computing task in response to receiving an execution instruction of the computing task; a determination module, used to determine a file block list corresponding to the preset file name, compare the target file name corresponding to at least one file block in the file block list with the preset file name, and obtain a first comparison result, wherein the first comparison result is used to indicate whether the target file block required for the computing task exists in the file block list; an execution module, used to execute the computing task based on the first comparison result, and obtain an execution result, wherein the execution result is used to indicate whether the computing task is successfully executed.

[0014] According to another aspect of an embodiment of the present invention, an electronic device is provided, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods of various embodiments of the present invention when running.

[0015] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium includes a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the methods in various embodiments of the present invention.

[0016] According to another aspect of an embodiment of the present invention, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the method in each embodiment of the present invention is implemented.

[0017] According to another aspect of an embodiment of the present invention, a computer program product is provided, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method in each embodiment of the present invention is implemented.

[0018] According to another aspect of the embodiments of the present invention, a computer program is provided. When the computer program is executed by a processor, the methods in various embodiments of the present invention are implemented.

[0019] In an embodiment of the present invention, in response to receiving an execution instruction of a computing task, a preset file name of a file required for the computing task is obtained; a file block list corresponding to the preset file name is determined, and a target file name corresponding to at least one file block in the file block list is compared with the preset file name to obtain a first comparison result, wherein the first comparison result is used to indicate whether the target file block required for the computing task exists in the file block list; the computing task is executed based on the first comparison result to obtain an execution result, wherein the execution result is used to indicate whether the computing task is successfully executed; it is easy to notice that the present application proposes to compare the preset file name of the file required for the computing task with the target file name of the corresponding file block list, and based on the comparison result, it can be confirmed that a file block list of the target file block exists, thereby realizing screening of the file block list, and then determining the address information of the target file block, simplifying the method for obtaining the address information of the target file block, so that the address information of each file block required for executing the computing task can be efficiently obtained, facilitating the smooth execution of the computing task, and thus solving the technical problem of low execution efficiency of computing tasks in related technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0021] Figure 1 is a flowchart of a method for executing a computing task according to an embodiment of the present invention;

[0022] Figure 2 is a flowchart of an optional method for executing a computing task according to an embodiment of the present invention;

[0023] Figure 3 2 is a schematic diagram of a device for executing a computing task according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0025] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] Example 1

[0027] According to an embodiment of the present invention, an embodiment of a method for executing a computing task is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0028] Figure 1 is a flow chart of a method for executing a computing task according to an embodiment of the present invention. Figure 1 As shown, the method includes the following steps:

[0029] Step S102 : in response to receiving an execution instruction of a computing task, obtaining a preset file name of a file required for the computing task.

[0030] The above-mentioned computing tasks may refer to the use of computer technology to process and analyze data. Computing tasks may include but are not limited to data analysis or data calculation. Data analysis may refer to the process of collecting, organizing, processing and analyzing large amounts of data to discover hidden patterns, trends and associations in the data, thereby drawing valuable information and conclusions. Data calculation may refer to the process of calculating and processing data using computers and related software tools, including performing basic mathematical operations, logical operations, statistical calculations and other types of computing operations on data.

[0031] The aforementioned execution instruction may refer to an instruction or command for starting a computing task.

[0032] In the application scenario of storage and computing separation, the storage address of the data file needs to be obtained when the computing task is executed. Since in the file system, the file is divided into a series of blocks, each block has a fixed size and is used to store the content of the file. The data file is stored in multiple storage spaces or data units, namely data blocks or file blocks (Block). When the computing task is executed, it is necessary to determine the address information of each data block corresponding to the data file. Among them, storage and computing separation can refer to separating computing and storage in order to better optimize resources and improve performance. In the storage and computing separation architecture, computing nodes and storage nodes are responsible for processing computing tasks and storing data respectively, which can better meet the needs of different workloads and improve the elasticity and flexibility of the system. The storage and computing separation architecture can be applied to different computing environments, including cloud computing, big data processing, and distributed systems. Through storage and computing separation, computing and storage resources can be better utilized to improve system performance and efficiency.

[0033] The above-mentioned preset file name may refer to the name or identifier of the file to be processed by the computing task. The naming format of the preset file name may be determined according to actual needs and is not limited here.

[0034] In an optional embodiment, the user can input or select a preset file name for the file required for the computing task. The system can also automatically determine the required file based on the computing task to be performed, thereby automatically reading the preset file name of the required file. The method for obtaining the preset file name here can be selected according to actual needs and is not limited here.

[0035] Step S104 : determining a file block list corresponding to a preset file name, comparing a target file name corresponding to at least one file block in the file block list with the preset file name, and obtaining a first comparison result.

[0036] The first comparison result is used to indicate whether the target file block required by the computing task exists in the file block list.

[0037] The above-mentioned file block list may refer to a list consisting of the file block where the file required for the computing task is stored and the remaining file blocks. The file required for the computing task may be stored in multiple file blocks, and the file blocks may not be continuous in storage. The file required for the computing task may correspond to multiple file block lists. Different file block lists contain at least one file block, that is, the file block list can contain one file block or multiple file blocks. The number of file blocks in the file block list can be determined according to actual conditions and is not limited here.

[0038] The target file name mentioned above may refer to the name or identifier of each file block in the file block list. The naming format of the target file name may be determined according to actual needs and is not limited here.

[0039] When a computing task is executed, it is necessary to obtain the storage address of the data file. The data file is stored in multiple storage spaces or data units, namely file blocks. When the computing task is executed, it is necessary to determine the address information of each file block corresponding to the data file. The file required for the computing task may be stored in multiple file blocks. The file blocks may not be continuous in storage. The file required for the computing task may correspond to multiple file block lists. The computing task execution method proposed in this application can at least be used to solve the problem of determining the storage address of each file block corresponding to the file required for the computing task from multiple file block lists to ensure the execution of the computing task.

[0040] The target file blocks mentioned above may refer to the file blocks corresponding to the file storage required for the computing task.

[0041] The user can enter or select a file block list corresponding to a preset file name of a file required for a computing task. The system can also automatically determine the corresponding file block list based on the computing task to be performed. For example, the metadata system of the Juice File System (Juicefs) can determine the corresponding file block list based on the files required for the computing task. Specifically, each file block can be hashed to obtain a unique hash value as the identifier of the file block, and the hash value can be mapped to the storage location of the file block. In this way, the corresponding file block list can be quickly located based on the metadata information of the file required by the computing task. The metadata system will record the hash value, storage location, and other related information of each file block, so that the required file block list can be quickly located when the target file needs to be read or written. Through such a metadata system design, Juicefs can efficiently manage large-scale file data and can quickly locate the required file block list when a file needs to be read or written, thereby improving the performance and reliability of the file system. The method for determining the file block list here can be selected according to actual needs and is not limited here.

[0042] In an optional embodiment, the system may first determine a list of all possible file blocks corresponding to the file storage required for the computing task, that is, some file block lists may contain target file blocks, and some file block lists may not contain target file blocks. It is necessary to compare whether the target file blocks exist in each file block list, and determine the storage address of each file block corresponding to the file required for the computing task to ensure the execution of the computing task. The comparison here may refer to comparing whether the target file name is completely consistent with the preset file name, or comparing whether the target file name contains the preset file name. If there are multiple target data blocks corresponding to the file required for the computing task, it is possible to compare whether the target file name contains the preset file name. All file blocks whose target file names contain the preset file name can be determined as target file blocks.

[0043] Step S106: Execute the computing task based on the first comparison result to obtain an execution result.

[0044] The execution result is used to indicate whether the computing task is successfully executed.

[0045] The above-mentioned execution result can be used to indicate whether the computing task is successfully executed. The execution result may include but is not limited to the computing task being successfully executed, the computing task being executed, the computing task being executed unsuccessfully, etc.

[0046] In an optional embodiment, all target file blocks corresponding to the file storage required for the computing task may be determined first, and then the computing task is executed to obtain an execution result indicating successful execution of the computing task; or the computing task may be started after some target file blocks corresponding to the file storage required for the computing task are determined, and then subsequent computing tasks are executed while the remaining target file blocks are determined, and then the execution result of the computing task can be obtained during execution; when the target file blocks corresponding to the file required for the computing task are not determined, an execution result indicating unsuccessful execution of the computing task can be output.

[0047] In an embodiment of the present invention, in response to receiving an execution instruction of a computing task, a preset file name of a file required for the computing task is obtained; a file block list corresponding to the preset file name is determined, and a target file name corresponding to at least one file block in the file block list is compared with the preset file name to obtain a first comparison result, wherein the first comparison result is used to indicate whether the target file block required for the computing task exists in the file block list; the computing task is executed based on the first comparison result to obtain an execution result, wherein the execution result is used to indicate whether the computing task is successfully executed; it is easy to notice that the present application proposes to compare the preset file name of the file required for the computing task with the target file name of the corresponding file block list, and based on the comparison result, it can be confirmed that a file block list of the target file block exists, thereby realizing screening of the file block list, and then determining the address information of the target file block, simplifying the method for obtaining the address information of the target file block, so that the address information of each file block required for executing the computing task can be efficiently obtained, facilitating the smooth execution of the computing task, and thus solving the technical problem of low execution efficiency of computing tasks in related technologies.

[0048] Optionally, a computing task is executed based on the first comparison result to obtain an execution result, including: if the first comparison result is that the target file block exists in the file block list, determining the host address information of the target file block, and calling the target data in the target file block based on the host address information of the target file block to execute the computing task to obtain the execution result; if the first comparison result is that the target file block does not exist in the file block list, determining that the execution result is that the computing task was not successfully executed.

[0049] The above-mentioned host address information may refer to the storage address information of the target file block. Based on the host address information, the data information stored in the target file block may be called. The type and format of the host address information may be determined according to actual needs and is not limited here.

[0050] In the case where the first comparison result is used to indicate that the target file block exists in the current file block list, the storage address information of the target file block can be determined based on the target file block. Based on the storage address information of the target file block, the data information in the target file block, that is, the target data, can be called, so that the computing task can be executed based on the data information in the target file block; if the first comparison result is used to indicate that the target file block does not exist in the current file block list, that is, the target file block has not been determined, the execution result of the unsuccessful execution of the computing task can be output.

[0051] In an optional embodiment, a method for determining the host address information of a target file block may include: determining a specific storage engine for data storage based on the type of storage medium, then determining a specific location where the file block is stored on the storage medium, and finally, calculating the specific storage address information of the file block on the storage medium based on the storage medium and the storage location information. The above-mentioned storage engine may be a file system, a database management system, etc., the above-mentioned specific location may be a file path, a database table name, a memory address, etc., and the above-mentioned storage address information may be a file offset, a row and column index of a database table, a starting address of a memory, etc. Other methods may also be selected to determine the host address information of the target file block, which are not limited here.

[0052] Optionally, the target file name and preset file name corresponding to at least one file block in the file block list are compared to obtain a first comparison result, including: using a Bloom filter to determine whether the preset file name exists in the target file name; if it is determined that the preset file name does not exist in the target file name, determining that the first comparison result is that the target file block required for the computing task does not exist in the file block list; if it is determined that the preset file name exists in the target file name, determining the first comparison result based on the shard information contained in the preset file name, wherein the shard information is used to indicate information about the shard in which the file block corresponding to the preset file name is located.

[0053] The above-mentioned Bloom filter can refer to a data structure used to determine whether an element exists in a set. The Bloom filter can be implemented by using multiple hash functions and a bit array. The elements can be mapped to the bit array, and then the judgment is made based on the state of the element mapped to the bit array. The Bloom filter does not store the elements themselves and has strong confidentiality. It also has the advantages of low query operation time complexity and small storage space. It is easy to notice that the Bloom filter may produce false positive examples, that is, it may judge that the element is in the set, but it is actually not in the set, but the Bloom filter will not produce false negative examples, that is, if the Bloom filter judges that the element is not in the set, then it is definitely not in the set.

[0054] The above-mentioned slicing may refer to a structural slicing (Slice) for processing and managing data, which may divide data into multiple parts, each part being called a slicing. Slicing may be used for data transmission, storage and processing, and is sometimes also used for data encryption and secure transmission. The size and number of slicing may be determined based on specific needs, and may be adjusted based on network bandwidth, storage space and processing power, which are not limited here. In this embodiment, a slicing may contain at least one file block, that is, a slicing may contain one file block or multiple file blocks. There is no limitation on the data of the file blocks contained in a slicing, and the host address information of the target file block contains information on the slicing where the target file block is located.

[0055] The above-mentioned shard information may refer to the address information of the shard where the target file block is located. Based on the shard information, the address of the shard where the target file block is located may be determined.

[0056] In an optional embodiment, a Bloom filter can be used to determine whether the target file names of each file block in the current file block list contain a preset file name of the file required for the computing task. Based on the feature that the Bloom filter will not generate false counterexamples, if the preset file name does not exist in the target file names of each file block in the current file block list, it can be quickly determined that the target data block does not exist in the current list file blocks, that is, the first comparison result can be determined as the target file block required for the computing task does not exist in the current file block list. At this time, it is determined that the target file block does not exist in the current file block list, and the next file block list can be judged until the target file blocks of the file required for the computing task are determined, that is, the judgment of the file block list can be stopped. Here, the characteristics of high space efficiency and fast query speed of the Bloom filter are utilized to realize the search operation of large-scale data, effectively improving the efficiency of screening out the file block list that does not contain the target file block, so as to facilitate the subsequent further determination of the file block list containing the target file block.

[0057] If it is determined based on the Bloom filter that the target file name exists / contains a preset file name, that is, the target file block may exist in the current file block list, the address of the shard where the target file block is located can be determined based on the shard information contained in the preset file name, thereby facilitating further determination of the host address information of the target file block.

[0058] Optionally, determining a first comparison result based on the fragment information contained in the preset file name includes: obtaining the fragment name and fragment serial number in the fragment information; constructing character information of the preset file name based on the fragment name, wherein the character information is used to represent information of characters contained in the preset file name; constructing a preset sub-bitmap of the character information based on the fragment serial number, wherein the preset sub-bitmap is used to represent the character information through binary data; and determining the first comparison result based on the preset sub-bitmap and the target sub-bitmap corresponding to the target file name.

[0059] The above-mentioned shard name may refer to an identifier of the shard where the file block is located, which is used to identify a specific shard. The shard name may be SliceID, etc. The shard name may also be determined according to actual needs and is not limited here.

[0060] The above-mentioned shard serial number can refer to the sequential number of the file block in the shard, which is used to identify the position of the file block in the shard. Through the shard name and shard serial number, the position of the data block in the shard can be accurately located and accessed.

[0061] The above-mentioned preset sub-bitmap can refer to a bitmap of the shard name of the shard where the target file block is located, which is recorded at the preset file name end. In the computer field, storage devices can divide data into multiple blocks and store them in different shards. In order to record the information of the shard where each file block is located, the storage device will use a sub-bitmap to represent the status of each shard name, such as whether it is used or idle. Through the sub-bitmap, the system can quickly find and manage the shard where the file block is located, thereby improving the efficiency of data storage and access.

[0062] The target sub-bitmap mentioned above may refer to a bitmap of the slice name of the slice where each file block in each file block list is located.

[0063] The corresponding character information can be constructed based on the shard name of the shard where the file block is located. Specifically, the shard name can be converted into clear character information using a hash function. For example, the shard name can be hashed using a hash function, and then the hash value can be converted into a hexadecimal string as character information. This ensures that the character information corresponding to each shard name is clear and can be irreversibly converted back to the shard name. In addition, a custom algorithm can be designed to construct the corresponding character information based on specific needs and application scenarios. It is necessary to ensure that the converted character information can accurately represent the corresponding shard name and that there will be no conflict. Here, the corresponding character information can be constructed from the shard name of the shard where the file block is located in other ways, which are not limited here.

[0064] A corresponding sub-bitmap can be constructed based on the slice serial number of the slice where the file block is located. Specifically, the slice serial number of the slice where the data block is located can be determined, and the starting position and ending position of the corresponding sub-bitmap can be calculated based on the slice serial number. For example, if the slice serial number is 3, the starting position of the sub-bitmap is 3*sub-bitmap size, and the ending position is (3+1)*sub-bitmap size. According to the starting position and the ending position, the corresponding sub-bitmap data block is extracted from the entire bitmap, and the extracted sub-bitmap data block is used for subsequent calculation or processing. Here, other methods can be used to construct the corresponding sub-bitmap based on the slice serial number of the slice where the file block is located, which are not limited here, so that a preset sub-bitmap of character information and a target sub-bitmap corresponding to the target file name can be constructed based on the slice serial number.

[0065] In an optional embodiment, the sub-bitmap of the shard where the target file block is located can be compared bit by bit with each target sub-bitmap to check for inconsistent bits. If the comparison result shows inconsistent bits, it indicates that the file block corresponding to the current target sub-bitmap is not the target file block. If the comparison result shows no inconsistent bits, it indicates that the file block corresponding to the current target sub-bitmap is the target file block. By comparing the sub-bitmap of the shard where the target file block is located with each target sub-bitmap, the target file block can be quickly determined. Since the sub-bitmap is a data structure used to represent the storage status of a file block, the sub-bitmap can help quickly determine the storage address information of the file block. By comparing the sub-bitmaps, the storage status of the file block can be directly viewed without having to query the storage address information of each file block one by one, thereby improving query efficiency. In addition, the sub-bitmap uses bit operations to represent the status of the file block, which allows efficient bit operations to be performed in memory, further improving query efficiency. Therefore, by comparing the sub-bitmaps, the storage address information of the target file block can be obtained more quickly.

[0066] Optionally, determining a first comparison result based on a preset sub-bitmap and a target sub-bitmap corresponding to a target file name includes: comparing the preset sub-bitmap and the target sub-bitmap corresponding to the target file name to obtain a second comparison result, wherein the second comparison result is used to indicate whether there is a sub-bitmap identical to the preset sub-bitmap in the target sub-bitmap; if the second comparison result is that there is a sub-bitmap identical to the preset sub-bitmap in the target sub-bitmap, determining that the first comparison result is that the target file block exists in the file block list, and determining that the file block corresponding to the sub-bitmap in the file block list is the target file block; if the second comparison result is that there is no sub-bitmap identical to the preset sub-bitmap in the target sub-bitmap, determining that the first comparison result is that the target file block does not exist in the file block list.

[0067] The second comparison result may refer to the comparison result between the preset sub-bitmap and each target sub-bitmap. The second comparison result may indicate that the comparison result between the current target sub-bitmap and the preset sub-bitmap is consistent or inconsistent.

[0068] In an optional embodiment, the sub-bitmap of the shard where the target file block is located can be compared bit by bit with each target sub-bitmap to find out whether there are inconsistent bits. If the comparison result shows inconsistent bits, it means that the file block corresponding to the current target sub-bitmap is not the target file block. At this time, the next target sub-bitmap can be selected for comparison with the preset sub-bitmap. If the comparison result shows no inconsistent bits, it means that the file block corresponding to the current target sub-bitmap is the target file block. By comparing the sub-bitmap of the shard where the target file block is located with each target sub-bitmap, the target file block can be quickly determined to ensure the smooth execution of the computing task.

[0069] Optionally, character information of a preset file name is constructed based on the shard name, including: if a target value contained in the shard name is greater than a preset value, an array is constructed based on the target value, and the array is determined to be character information; if the target value is less than or equal to the preset value, a string is constructed based on the target value, and the string is determined to be character information.

[0070] The above-mentioned preset value can be a value pre-set by the user, which is used to judge the size of the fragment name in order to determine the data structure / type of the character information. The size of the preset value can be determined according to actual needs and is not limited here.

[0071] The above-mentioned array can refer to a data structure used to store elements of the same type. The array consists of a series of consecutive memory cells. Each element has a clear index in the array, and the elements in the array can be accessed through the index. The array can be one-dimensional, two-dimensional or even higher-dimensional, and can store basic data types, objects or other arrays.

[0072] The above-mentioned character string may refer to a sequence of characters, and the character types in the character string may include but are not limited to letters, numbers, punctuation marks or other symbols.

[0073] In an optional embodiment, when constructing corresponding character information based on the shard name of the shard where the file block is located, if the shard name of the shard where the file block is located is greater than a preset value, corresponding array character information can be constructed based on the shard name. Specifically, the shard name can be converted into array character information, and a hash function or encoding algorithm can be used to convert the shard name into an array index value as part of the array character information. The shard name where the target file block is located is then associated with the array character information. The shard name can be used as an element of the array, and the array character information can be used as an index of the array to construct the corresponding array character information. Here, a method can also be used to construct the corresponding array character information based on the shard name, which is not limited here.

[0074] If the shard name of the shard where the file block is located is less than or equal to a preset value, the corresponding string character information can be constructed based on the shard name. Specifically, the index value of the shard name can be used as the character information. For each shard, its index value can be used as part of the character information. For example, if the shard name is "chunk_001", "001" can be used as part of the character information. The file name and the shard index value are then used to construct the character information. The file name of the target file and the index value of the shard can be combined as the character information. For example, if the target file name is "example.txt" and the shard name is "chunk_001", "example.txt_001" can be used as part of the character information. Finally, the character information is constructed using the hash value. The shard name can be hashed to obtain a clear hash value as the character information. This ensures that each shard has a clear character information, thereby constructing the corresponding string character information. Here, a method can also be used to construct the corresponding string character information based on the shard name, which is not limited here.

[0075] In an optional embodiment, since arrays can more efficiently store large amounts of character information and can be more easily indexed and accessed, when the shard name of the shard where the target file block is located is greater than a preset value, you can choose to construct array-type character information, which can improve the program's operating efficiency and performance; and when the shard name is less than or equal to the preset value, you can choose to construct string-type character information. Since strings can more flexibly store small amounts of character information and are also easier to understand and maintain, in addition, for small-scale character information, using string-type character information can reduce memory usage, that is, depending on the size of the shard name and the preset value, you can better meet the query requirements and improve the system's performance and maintainability by selecting an appropriate character information type.

[0076] Optionally, the target data in the target file block is called based on the host address information of the target file block to execute the computing task and obtain the execution result, including: obtaining at least one computing node corresponding to the computing task, and constructing a host topology map based on the host address information; using the host topology map to call the target data in the target file block, and pushing the target data to at least one computing node to execute the computing task and obtain the execution result.

[0077] The above-mentioned computing nodes can refer to the nodes participating in the computing task. The computing task can include at least one computing node, that is, the computing task can include one computing node or multiple computing nodes. The computing node can be determined according to actual needs and is not limited here.

[0078] The above-mentioned host topology map may refer to the sorting order of host address information of each target file block executing the computing task. Based on the host topology map, each target file block may be called in sequence to ensure the smooth execution of the computing task.

[0079] In an optional embodiment, all target file blocks corresponding to the file storage required for the computing task can be determined first and then the computing task can be executed to obtain an execution result of successful execution of the computing task; it is also possible to determine some target file blocks corresponding to the file storage required for the computing task before starting to execute the computing task, and then determine the remaining target file blocks while executing subsequent computing tasks, at this time, the execution result of the computing task can be obtained; based on the execution order of each computing node corresponding to the computing task, the host address information of each target file block can be determined in turn, and the host address information of each target file block can be composed into a host topology map. Based on the host topology map, the system can determine the host address information of each target file block, so that each target file block can be called in sequence to ensure the smooth execution of the computing task.

[0080] In a big data architecture with separated storage and computing, cloud object storage or remote object storage can be used for storage. When the data analysis layer or data computing layer needs to use data, it can pull and read data remotely through network resources. To improve efficiency, caching technology can be used at the data application end or computing end.

[0081] In a big data architecture with storage and computing separated, storage can use cloud object storage or remote object storage. When the data analysis layer or data computing layer needs to use data, it needs to pull the read data remotely through network resources. To improve efficiency, a cache technology can be used on the data application side. In this embodiment, the Juice File System (Juicefs) can be used as a cache layer for the Yarn Package Manager (Yarn) and object storage. The cache layer can also be selected according to actual needs and is not limited here. Juicefs splits large files when writing files, and can default to 4MB as a block. Therefore, blocks are cached in the cache layer. These 4MB blocks are conducive to random reads by the computing layer. However, because the computing layer cannot identify the specific location of the blocks, the data localization read rate is low. For example, computing node 1 needs Block 1 of the file src.txt, but Block 1 is on computing node 2.

[0082] For Juicefs, each file is composed of one or more data blocks (Chunks), and each Chunk is up to 64M. No matter how large the file is, all reads and writes will be located to the corresponding Chunk according to its offset (that is, the file location where the read and write operations are generated). The main difference between Chunk and Block is that they describe different storage media. Chunk describes the data block storage in the memory, while Block describes the data block storage on the disk or other permanent storage media. Chunk exists to optimize search positioning. The actual file writing is performed on the slice. In Juicefs, a Slice represents a continuous write. It belongs to a certain Chunk and cannot cross the Chunk boundary, so the Slice length will not exceed 64M. For example, if a file is generated by a coherent sequential write, then each Chunk will only contain one Slice. Sequentially writing a 160M file will eventually generate 3 Chunks, and each Chunk contains only one Slice. The Slice can be further split into multiple Blocks (the default maximum is 4M), and multi-threaded concurrent writing can improve write performance. The Chunks and Slices introduced above are actually logical data structures, and Blocks are the final physical storage form, which is the smallest storage unit of object storage and disk cache.

[0083] The technical solution proposed in this application can query the fragmented distribution of file blocks, and can solve the problem that in the absence of a metadata storage service similar to Name Node under a storage and computing separation architecture, the computing program cannot determine whether the locally cached blocks have the data required for computing. The main idea is to use the file name and file block information stored in the metadata service to calculate in advance which host the blocks required by the computing node are located on when the computing program starts. The implementation method is to use the algorithm to form a host list of the data required for computing through the file name required for computing, thereby constructing a topology structure table to ensure the locality of the data.

[0084] For example, there are two files, src1.txt and src2.txt, both stored in a remote object storage cluster. A computing task needs to read these two files. When the computing task is started for the first time, each computing node in the distributed computing system will randomly read a portion of the data from the two files to perform the calculation and cache the random data it read on its own node. This data is cached to the local node at the block granularity. However, there is no central metadata node like the NameNode. This data is stored locally and does not know which file it belongs to. If the computing task is started again, in order to fully utilize the cached data, subsequent data parsing algorithms are required. Subsequent data parsing is completed before the task is started. The task only needs to be started on the machine node that has the required data. An example block name is: jfstest / chunks / 0 / 0 / 10_0_4194304. Among them, 10 in 10_0_4194304 is the slice ID, which is a monotonically increasing ID, and 0 is the slice sequence number. For each slice ID, the slice sequence number is also monotonically increasing. Each block is randomly distributed to a computing node.

[0085] Figure 2 is a flow chart of an optional method for executing a computing task according to an embodiment of the present invention, such as Figure 2 As shown, the steps of the method for executing a computing task may include:

[0086] S202, obtaining a preset file name of a file required for the computing task;

[0087] S204, determining a list of file blocks corresponding to a preset file name;

[0088] S206, parsing the preset file name and filtering each file block list based on the Bloom filter;

[0089] S208, constructing a preset sub-bitmap based on the preset file name, and performing comparison to obtain the host address information of the target file block;

[0090] S210: construct a host topology map based on the acquired host address information, and execute a computing task based on the host topology map.

[0091] The main modules of the system may include: a mapping and parsing module, a comparison module, and a host topology construction module. Among them, the mapping and parsing module can obtain a list of all file blocks of the file in the remote file system according to the file name input by the computing task, scan the Block file name in the cache directory, and obtain the SliceID and slice sequence number in the file name; the comparison module: the comparison process can be divided into two stages. The first stage can extract all the Block names of the files required for calculation from the metadata. The second stage can compare them with the Block in the cache path through this method to find the data required by the computing node. The comparison can use a bit to mark the Value corresponding to an element, and the Key is the element. Since data is stored in bits, storage space can be greatly saved; the host topology construction module can build a host topology map in advance before the task starts based on the algorithm comparison results, and the computing node can start the computing process according to the host topology map.

[0092] File block data is randomly cached to random nodes. The distribution of file block data is sparse. If the data interval is too large or there are individual SliceIDs that are too large, unnecessary space will be wasted. When the integer data is larger than 32 bits or the data interval is more than 16 bits, it can be converted into a string for comparison. The first parsing occurs on each computing node client. After the first layer of parsing, the data will be organized into a name similar to 1-2; this means that the SliceID of this block is 1 and the sequence number is 2. The Block name can be used to construct a Bloom filter. At the same time, the size of the SliceID can be used to decide whether to construct a string or an array. The sequence number 2 can be used to construct a sub-bitmap of the SliceID. The array or string constructed with the SliceID can be used as the first layer, and the sequence number can be used as the second layer.

[0093] Host topology construction module: A Bloom filter can be constructed based on the SliceID of the Block to determine whether the Block is likely to exist on this node. If so, the above-mentioned two-layer structure is searched. If the search is completed, the data is recorded to construct the host topology map, and the data is organized in a 1-1-1 manner, that is, Block 1-1 is cached on the first node. By analogy, the computing node can start the computing task on the node where the cached data exists according to the data it needs, and can make full use of the cached data. The host topology map can be pushed to each computing node at regular intervals, and the computing node starts the computing process based on the host topology map.

[0094] The specific process of the comparison algorithm may include: the distributed computing system requires src.txt, src.txt, there are 5 computing nodes, the file is stored on the remote object storage, and the src.txt data blocks include the following examples: jfstest / chunks / 0 / 0 / 1_0_4194304, jfstest / chunks / 0 / 0 / 2_1_4194304, jfstest / chunks / 0 / 0 / 100_2_4194304, jfstes t / chunks / 0 / 0 / 200_3_4194304, jfstest / chunks / 0 / 0 / 300_15_4194304, 5 blocks are randomly cached to 5 computing nodes; a Bloom filter can be built based on each node. The construction process is to first use multiple hash functions to obtain the hash value of the block name. Each hash function will map the element to a position on the bitmap, which can be set to 1. When querying, the corresponding position will be found through all hash functions. If all positions are 1, it is judged that the element may exist. Each bitmap can be set to contain 32 bits, and bit operations can be used to quickly set or get bits. Finally, the host topology map can be constructed based on the algorithm comparison search results. Block jfstest / chunks / 0 / 0 / 1_0_4194304 is on node 1. The computing node can start the computing task on the node where the cached data exists according to the data it needs, so that the cached data can be fully used. The host topology map will be pushed to each computing node regularly, and the computing node will start the computing process based on the host topology map.

[0095] In the storage and computing separation scenario, the execution method of computing tasks proposed in this application is compared with related technologies. There is no central metadata service and no single-point metadata bottleneck. Using this method, a host topology map can be constructed. The computing node starts the computing task according to the host topology, and the judgment search speed is fast, the resource usage is small, and the data location is accurately located. In the absence of distributed cache or central metadata, the data location can also be determined to achieve cache data localization. At the same time, Bloom filters and sub-bitmaps are used to determine the data location. The data distribution can be calculated in advance when the computing task is started. According to the data situation, the SliceID of the Block can be automatically converted into an array or string. By constructing a host topology structure, the calculation can use local data as much as possible, and the local cache data of the client can be fully utilized to enhance the utilization rate of the local disk, avoid data transmission across the network, improve resource utilization, and achieve cost reduction and efficiency improvement.

[0096] Example 2

[0097] According to another aspect of an embodiment of the present invention, a device for executing a computing task is also provided. The device can execute a method for executing a computing task in the above embodiment. The specific implementation method and preferred application scenario are the same as those in the above embodiment and will not be repeated here.

[0098] Figure 3 is a schematic diagram of a device for executing a computing task according to an embodiment of the present invention, such as Figure 3 As shown, the device includes the following: an acquisition module 302 , a determination module 304 , and an execution module 306 .

[0099] Among them, the acquisition module 302 is used to obtain the preset file name of the file required for the computing task in response to receiving the execution instruction of the computing task; the determination module 304 is used to determine the file block list corresponding to the preset file name, compare the target file name and the preset file name corresponding to at least one file block in the file block list, and obtain a first comparison result, wherein the first comparison result is used to indicate whether the target file block required for the computing task exists in the file block list; the execution module 306 is used to execute the computing task based on the first comparison result to obtain an execution result, wherein the execution result is used to indicate whether the computing task is successfully executed.

[0100] In the above embodiment of the present application, the execution module includes: a calling unit and a first determining unit.

[0101] Among them, the calling unit is used to determine the host address information of the target file block if the first comparison result is that the target file block exists in the file block list, and call the target data in the target file block based on the host address information of the target file block to execute the computing task and obtain the execution result; the first determining unit is used to determine that the execution result is that the computing task is not successfully executed if the first comparison result is that the target file block does not exist in the file block list.

[0102] In the above embodiment of the present application, the determination module includes: a judgment unit, a second determination unit, and a third determination unit.

[0103] Among them, the judgment unit is used to use the Bloom filter to determine whether a preset file name exists in the target file name; the second determination unit is used to determine that the first comparison result is that the target file block required for the computing task does not exist in the file block list if it is determined that the preset file name does not exist in the target file name; the third determination unit is used to determine the first comparison result according to the shard information contained in the preset file name if it is determined that the preset file name exists in the target file name, wherein the shard information is used to indicate information about the shard where the file block corresponding to the preset file name is located.

[0104] In the above embodiment of the present application, the third determining unit includes: an acquiring subunit, a first constructing subunit, a second constructing subunit, and a determining subunit.

[0105] Among them, the acquisition subunit is used to obtain the fragment name and fragment serial number in the fragment information; the first construction subunit is used to construct character information of the preset file name based on the fragment name, wherein the character information is used to represent information of characters contained in the preset file name; the second construction subunit is used to construct a preset sub-bitmap of the character information based on the fragment serial number, wherein the preset sub-bitmap is used to represent the character information through binary data; the determination subunit is used to determine the first comparison result based on the preset sub-bitmap and the target sub-bitmap corresponding to the target file name.

[0106] Among them, the determination sub-unit is further used to compare the preset sub-bitmap with the target sub-bitmap corresponding to the target file name to obtain a second comparison result, wherein the second comparison result is used to indicate whether there is a sub-bitmap identical to the preset sub-bitmap in the target sub-bitmap; if the second comparison result is that there is a sub-bitmap identical to the preset sub-bitmap in the target sub-bitmap, it is determined that the first comparison result is that the target file block exists in the file block list, and the file block corresponding to the sub-bitmap in the file block list is determined to be the target file block; if the second comparison result is that there is no sub-bitmap identical to the preset sub-bitmap in the target sub-bitmap, it is determined that the first comparison result is that the target file block does not exist in the file block list.

[0107] Among them, the first construction sub-unit is also used to construct an array based on the target value if the target value contained in the shard name is greater than the preset value, and determine that the array is character information; if the target value is less than or equal to the preset value, construct a string based on the target value, and determine that the string is character information.

[0108] Among them, the first construction sub-unit is also used to obtain at least one computing node corresponding to the computing task, and build a host topology map based on the host address information; use the host topology map to call the target data in the target file block, and push the target data to at least one computing node to execute the computing task and obtain the execution result.

[0109] Example 3

[0110] An embodiment of the present application further provides an electronic device, comprising: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods of various embodiments of the present invention when running.

[0111] The above-mentioned memory may refer to a device inside a computer for storing data and programs, and may include memory, hard disk, etc., wherein the memory may be used to temporarily store running programs and data, the hard disk may be used to store programs and data for a long time, and the memory may be used to enable the computer to read and write data, as well as execute programs; the above-mentioned processor may be responsible for executing instructions in computer programs and performing data processing, and may be responsible for controlling and executing various operations, including arithmetic operations, logical operations, data transmission, etc.

[0112] Example 4

[0113] An embodiment of the present application further provides a computer-readable storage medium, which includes a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the methods in various embodiments of the present invention.

[0114] The above-mentioned computer storage medium may refer to a medium in a computer memory used to store certain discontinuous physical quantities. Computer storage media mainly include semiconductors, magnetic cores, magnetic drums, magnetic tapes, laser disks, etc. The stored program included in the computer-readable storage medium may be a set of instructions that can be recognized and executed by a computer, running on an electronic computer, and serving as an information tool to meet certain needs of people.

[0115] Example 5

[0116] An embodiment of the present application further provides a computer program product, including a computer program, which implements the methods in various embodiments of the present invention when executed by a processor.

[0117] The above-mentioned computer program product may refer to a software program that has been written, tested and released, which can be run on a computer or other device. The computer program product may include an application, an operating system, tool software, etc., which is used to implement specific functions or solve specific problems.

[0118] Example 6

[0119] An embodiment of the present application further provides a computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the method in each embodiment of the present invention is implemented.

[0120] The above-mentioned non-volatile computer-readable storage medium may refer to a medium for storing data. The non-volatile computer-readable storage medium can keep the data from being lost when the power is off, and can be used to store long-term data, such as operating systems, applications and user files. The non-volatile storage medium may include hard disk drives, solid-state drives, optical disks and flash memory storage devices, etc.

[0121] Example 7

[0122] The embodiments of the present application further provide a computer program, which implements the methods in the above-mentioned embodiments of the present invention when executed by a processor.

[0123] The above-mentioned computer program may refer to a collection of instructions used to tell a computer to perform a specific task or operation. A computer program may be written by a programmer using a specific programming language and may include algorithms, data structures, logic, and control flows. Computer programs may be used for a variety of purposes, including application software, operating systems, and the like.

[0124] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0125] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of the units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0126] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0127] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0128] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk, etc. Various media that can store program codes.

[0129] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A method for executing a computing task, characterized in that: include: In response to receiving an execution instruction of a computing task, obtaining a preset file name of a file required by the computing task, wherein the preset file name is used to represent a name or identifier of a file required to be processed by the computing task; Determining a file block list corresponding to the preset file name, comparing a target file name corresponding to at least one file block in the file block list with the preset file name to obtain a first comparison result, wherein the first comparison result is used to indicate whether the target file block required by the computing task exists in the file block list; executing the computing task based on the first comparison result to obtain an execution result, wherein the execution result is used to indicate whether the computing task is successfully executed; Among them, comparing the target file name corresponding to at least one file block in the file block list with the preset file name to obtain a first comparison result includes: using a Bloom filter to determine whether the preset file name exists in the target file name; if it is determined that the preset file name exists in the target file name, constructing a preset sub-bitmap based on the preset file name; and determining the first comparison result based on the preset sub-bitmap and the target sub-bitmap corresponding to the target file name.

2. The method according to claim 1, characterized in that Executing the computing task based on the first comparison result to obtain an execution result includes: If the first comparison result indicates that the target file block exists in the file block list, determining host address information of the target file block, and calling target data in the target file block based on the host address information of the target file block to execute the computing task, thereby obtaining the execution result; If the first comparison result is that the target file block does not exist in the file block list, it is determined that the execution result is that the computing task is not successfully executed.

3. The method according to claim 1, characterized in that The method further comprises: If it is determined that the preset file name does not exist in the target file name, it is determined that the first comparison result is that the target file block required by the computing task does not exist in the file block list.

4. The method according to claim 3, characterized in that Constructing a preset sub-bitmap based on the preset file name, including: Obtaining a fragment name and fragment sequence number from the fragment information contained in the preset file name, wherein the fragment information is used to indicate information about the fragment where the file block corresponding to the preset file name is located; Constructing character information of the preset file name based on the slice name, wherein the character information is used to represent information of characters contained in the preset file name; A preset sub-bitmap of the character information is constructed based on the fragment sequence number, wherein the preset sub-bitmap is used to represent the character information through binary data.

5. The method according to claim 1, characterized in that Determining the first comparison result based on the preset sub-bitmap and the target sub-bitmap corresponding to the target file name includes: Comparing the preset sub-bitmap with the target sub-bitmap corresponding to the target file name to obtain a second comparison result, wherein the second comparison result is used to indicate whether the target sub-bitmap contains a sub-bitmap identical to the preset sub-bitmap; If the second comparison result is that the target sub-bitmap contains a sub-bitmap identical to the preset sub-bitmap, determining that the first comparison result is that the target file block exists in the file block list, and determining that the file block corresponding to the sub-bitmap in the file block list is the target file block; If the second comparison result is that the target sub-bitmap does not contain a sub-bitmap identical to the preset sub-bitmap, it is determined that the first comparison result is that the target file block does not exist in the file block list.

6. The method according to claim 4, characterized in that Constructing character information of the preset file name based on the slice name includes: If the target value included in the shard name is greater than a preset value, constructing an array based on the target value, and determining the array as the character information; If the target value is less than or equal to the preset value, a character string is constructed based on the target value, and the character string is determined to be the character information.

7. The method according to claim 6, characterized in that Calling target data in the target file block based on the host address information of the target file block to execute the computing task and obtain the execution result includes: Obtain at least one computing node corresponding to the computing task, and construct a host topology map based on the host address information; The target data in the target file block is called using the host topology map, and the target data is pushed to the at least one computing node to execute the computing task, thereby obtaining the execution result.

8. A computing task execution device, characterized in that: include: an acquisition module, configured to, in response to receiving an execution instruction of a computing task, acquire a preset file name of a file required by the computing task, wherein the preset file name is used to represent a name or identifier of a file required to be processed by the computing task; a determination module, configured to determine a file block list corresponding to the preset file name, compare a target file name corresponding to at least one file block in the file block list with the preset file name, and obtain a first comparison result, wherein the first comparison result is used to indicate whether the target file block required by the computing task exists in the file block list; an execution module, configured to execute the computing task based on the first comparison result to obtain an execution result, wherein the execution result is used to indicate whether the computing task is successfully executed; The determination module is also used to compare the target file name corresponding to at least one file block in the file block list with the preset file name through the following steps to obtain a first comparison result: using a Bloom filter to determine whether the preset file name exists in the target file name; if it is determined that the preset file name exists in the target file name, constructing a preset sub-bitmap based on the preset file name; and determining the first comparison result based on the preset sub-bitmap and the target sub-bitmap corresponding to the target file name.

9. An electronic device, characterized in that: include: a memory storing an executable program; A processor, configured to run the program, wherein the program, when running, executes the method for executing a computing task as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored executable program, wherein when the executable program is run, the device where the storage medium is located is controlled to execute the method for executing the computing task according to any one of claims 1 to 7.

11. A computer program product, characterized in that The computer program comprises a computer program which, when executed by a processor, implements the method for executing a computing task according to any one of claims 1 to 7.

Images