Data extraction method and device, electronic equipment, storage medium and acceleration card

Abstract

The application discloses a data extraction method and device, a storage medium and an acceleration card. The method comprises the following steps: obtaining to-be-extracted data, extraction rule data and data processing parameters from a host computing device and storing the data in a storage area of the acceleration card; reading the data processing parameters from the storage area by a core of the acceleration card, reading the to-be-extracted data and the extraction rule data from the storage area according to the data processing parameters, performing data extraction on the to-be-extracted data based on the extraction rule data, and storing the data extraction result of the to-be-extracted data in the storage area; and returning the data extraction result in the storage area to the host computing device. Therefore, the data extraction task is transferred from the host computing device to the acceleration card for execution, so that the computing power occupation of the host computing device can be reduced, and the data extraction processing efficiency can be improved.

Description

Technical Field

[0001] This application relates to the field of data processing, and more particularly to a data extraction method, apparatus, storage medium, and accelerator card. Background Technology

[0002] Data extraction from a database is a crucial function that helps users quickly obtain useful information. For example, extracting useful information A and C from data A, B, and C.

[0003] However, existing data extraction processing for variable-length string data in databases can only be implemented using the CPU of the main computing device, which consumes a lot of CPU computing resources. Furthermore, due to the inability of the CPU instruction set architecture to efficiently and quickly extract data from large amounts of data, the current data extraction efficiency is also low. Summary of the Invention

[0004] In view of this, embodiments of this application provide a data extraction scheme that reduces the computing power consumption of the main computing device and improves the data extraction and processing efficiency by transferring the data extraction task from the main computing device to the accelerator card.

[0005] According to a first aspect of the embodiments of this application, a data extraction method is provided, applied to an accelerator card coupled to a main computing device. The method includes: obtaining data to be extracted, extraction rule data, and data processing parameters from the main computing device and storing them in the storage area of ​​the accelerator card; reading the data processing parameters from the storage area through the core of the accelerator card; reading the data to be extracted and the extraction rule data from the storage area according to the data processing parameters; performing data extraction on the data to be extracted based on the extraction rule data; storing the data extraction result of the data to be extracted in the storage area; and returning the data extraction result in the storage area to the main computing device.

[0006] According to a second aspect of the present application, a data extraction apparatus is provided, applied to an accelerator card coupled to a main computing device. The apparatus includes: a reading module, configured to acquire data to be extracted, extraction rule data, and data processing parameters from the main computing device, and store them in the storage area of ​​the accelerator card; an extraction module, configured to read the data processing parameters from the storage area through the core of the accelerator card, read the data to be extracted and the extraction rule data from the storage area according to the data processing parameters, perform data extraction on the data to be extracted based on the extraction rule data, and store the data extraction result of the data to be extracted in the storage area; and a return module, configured to return the target data in the storage area to the main computing device.

[0007] According to a third aspect of the embodiments of this application, an accelerator card is provided, comprising: a storage area and a core, wherein the storage area is used to store data to be extracted, extraction rule data, data processing parameters, and data extraction results of the data to be extracted obtained from the main computing device; the core reads the data processing parameters from the storage area through the core of the accelerator card, reads the data to be extracted and the extraction rule data from the storage area according to the data processing parameters, performs data extraction on the data to be extracted based on the extraction rule data, and obtains the data extraction results of the data to be extracted.

[0008] According to a fourth aspect of the embodiments of this application, a computer storage medium is provided, the computer storage medium storing computer program code, which, when executed by a processor, causes the processor to perform the data extraction method as described in the first aspect.

[0009] In summary, the data extraction scheme provided in this application obtains the data to be extracted, extraction rule data, and data processing parameters from the main computing device. Then, the accelerator card reads the data to be extracted and the extraction rule data according to the data processing parameters, uses the extraction rule data to extract the data to be extracted, and returns the data extraction results to the main computing device. This shifts the data extraction process from the main computing device to the accelerator card, thereby reducing the computing power consumption of the main computing device and improving the data extraction processing efficiency. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0011] Figure 1 A schematic diagram of the architecture of the data extraction method or apparatus for implementing the various embodiments of this application.

[0012] Figure 2 This is a flowchart illustrating the data extraction method as an exemplary embodiment of this application.

[0013] Figure 3 This is a schematic diagram illustrating the composition structure of the string to be extracted, which is an exemplary embodiment of this application.

[0014] Figure 4 This is a schematic diagram illustrating data interaction between the core and the storage area, which is an exemplary embodiment of this application.

[0015] Figure 5 This is a schematic diagram of an implementation scheme for data read / write between the core and storage area, which is an exemplary embodiment of this application.

[0016] Figure 6 This is a flowchart illustrating a data extraction method as an exemplary embodiment of this application.

[0017] Figure 7 This is a schematic diagram of the structure of the register to be processed, which is an exemplary embodiment of this application.

[0018] Figures 8A to 8H This is a schematic diagram of an embodiment of the register to be processed, which is an exemplary embodiment of this application.

[0019] Figure 9 This is a schematic diagram of an embodiment of the rule register of this application.

[0020] Figures 10A to 10D This is a schematic diagram of an embodiment of a processed register that is an exemplary embodiment of this application.

[0021] Figure 11 This is a structural block diagram of a data extraction apparatus that is an exemplary embodiment of this application.

[0022] Figure 12 This is a structural block diagram of an electronic device that is an exemplary embodiment of this application. Detailed Implementation

[0023] To enable those skilled in the art to better understand the technical solutions in the embodiments of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art should fall within the protection scope of the embodiments of this application.

[0024] This application relates to an effective data content extraction function for databases, which can extract data of variable-length string types. Existing technical solutions mainly rely on the main computing device (also known as the CPU) to execute database functions to implement the data extraction process, which has the following main drawbacks:

[0025] First, currently, the function of extracting data of variable-length string type in database can only be implemented by CPU. However, the existing CPU architecture can no longer provide enough computing power to support the ever-growing scale of data business.

[0026] Secondly, the current CPU instruction set architecture is unable to efficiently and quickly extract large amounts of data, resulting in low data extraction efficiency based on the CPU.

[0027] Third, due to the large volume of data processing, the CPU's computing power is limited, resulting in a high CPU utilization rate during the data extraction process. This prevents the server from performing other business processes while the data extraction is in progress.

[0028] Therefore, with the continuous improvement of disk read / write and network bandwidth performance, the performance bottleneck in the database field has shifted to computing power. In view of this, the embodiments of this application provide a data extraction scheme based on a heterogeneous architecture, capable of supporting large-scale data extraction of variable-length string types from databases.

[0029] The specific implementation of each embodiment of this application will be described in detail below with reference to the accompanying drawings.

[0030] Figure 1 This is a schematic diagram illustrating the architecture of the data extraction methods or apparatus for implementing the embodiments of this application. (See attached diagram.) Figure 1 As shown, the system architecture 100 includes a main computing device 110 and an accelerator card 120.

[0031] The main computing device 110 is the primary device that performs data extraction tasks, and the accelerator card 120 is an auxiliary device plugged into the main computing device 110 to improve the processing speed of the main computing device 110 in a certain aspect. In this embodiment of the present disclosure, the accelerator card 120 is used to improve the data extraction and processing speed of the main computing device 110.

[0032] The main computing device 110 can be a client or a server. In some embodiments, the main computing device 110 can be a desktop computer, laptop computer, mobile phone, PDA, dedicated terminal, or a cluster of terminals. In other embodiments, the main computing device 110 can also be a portion of a terminal, such as a virtual machine. The accelerator card 120 can be an auxiliary device (such as a card inserted into a card slot) or another independent device connected to the main computing device 110 via an interface.

[0033] In some embodiments, the main computing device 110 and the accelerator card 120 can communicate with each other via a PCIe interface. The accelerator card 120 can be inserted into the main computing device 110 through a PCIe slot so that the main computing device 110 and the accelerator card 120 can transmit data via PCIe.

[0034] In the various embodiments of this disclosure, the main computing device 110 executes a task. When a part of the task requires data extraction, the data information related to the data extraction can be transmitted to the accelerator card 120, which performs the data extraction processing and returns the obtained data extraction results to the main computing device 110, thereby reducing the computing power required by the main computing device 110 to perform the data extraction processing. Simultaneously, since the accelerator card 120 has a structure where the core 122 and the storage area 124 are separate, the execution efficiency of data extraction can be improved through the cooperation of the core 122 and the storage area 124.

[0035] like Figure 1 As shown, the main computing device 110 includes a processor (CPU) 112 and memory 114. The processor 112 is the core component for executing the tasks of the main computing device 110. The memory 114 is a component that stores the data required for the tasks of the main computing device 110, as well as the intermediate and final results generated by the main computing device 110 in executing the tasks. Raw data information related to the data extraction task is also stored in the memory 114. When executing the data extraction task, the processor 112 can retrieve the raw data related to the data extraction task from the memory 114 and send it to the accelerator card 120.

[0036] Accelerator card 120 includes a core 122 and a storage area 124. Core 122 is the processing unit in accelerator card 120 used to execute data extraction tasks. Because core 122 is designed specifically for executing data extraction tasks, it improves the processing efficiency of data extraction. Storage area 124, such as DDR memory, is a dedicated area in accelerator card 120 used to store data related to data extraction tasks. When executing data extraction tasks, accelerator card 120 does not need to call the processor 112 and memory 114 in the main computing device 110, thus avoiding repeated occupation of the transmission channel between the main computing device 110 and accelerator card 120 and reducing transmission overhead. Furthermore, due to the separate design mechanism between core 122 and storage area 124 in accelerator card 120, the accelerator card 120 achieves higher data extraction efficiency through the cooperation of core 122 and storage area 124 when performing data extraction tasks.

[0037] Figure 2 A flowchart illustrating a data extraction method according to an exemplary embodiment of the present disclosure is shown. As shown, the method 200 of this embodiment mainly includes the following steps:

[0038] Step 202: Obtain the data to be extracted, the extraction rule data, and the data processing parameters from the main computing device, and store them in the storage area of ​​the accelerator card.

[0039] In some embodiments, the accelerator card 120 may, in response to a data extraction execution instruction from the main computing device 110, obtain the data to be extracted A, the extraction rule data B, and the data processing parameters C from the memory 114 of the main computing device 110, and store them in the storage area 124 of the accelerator card 120.

[0040] In some embodiments, the data A to be extracted may include multiple strings to be extracted. The strings to be extracted may be fixed-length strings or variable-length strings.

[0041] When the string to be extracted is a variable-length string, each string may include an identifier field of fixed length and a content field of variable length. The identifier field of each string stores content length information, and the actual length of the content field of each string can be determined based on the content length information in the identifier field.

[0042] refer to Figure 3 The identifier field for each string to be extracted (a1, a2) can be set to a fixed 4 bytes (see reference). Figure 3 As shown in the example, by identifying the length information of the content in the identifier segment of the string to be extracted a1, the actual length of the content segment of the string to be extracted a1 can be parsed to be 5 bytes (i.e., "a", "b", "c", "d", "e"), and the total length of the string to be extracted a1 is 9 bytes; by identifying the length information of the content in the identifier segment of the string to be extracted a2, the actual length of the content segment of the string to be extracted a2 can be parsed to be 1 byte (i.e., "s"), and the total length of the string to be extracted a2 is 5 bytes.

[0043] For example, the maximum length of the content character field can be set to 60 bytes, that is, the length of each string to be extracted does not exceed 64 bytes.

[0044] In some embodiments, the extraction rule data B includes multiple character extraction rules, each character extraction rule corresponding to a string to be extracted in the data to be extracted A.

[0045] In this embodiment, the character extraction rules may include a first extraction rule or a second extraction rule, and different mask values ​​may be used to identify the first and second extraction rules. For example, the mask value corresponding to the first extraction rule may be identified as 1, and the mask value corresponding to the second extraction rule may be identified as 0.

[0046] When using different mask assignments to identify different character extraction rules, the length of each character extraction rule can be set to 1 bit.

[0047] Step 204: Read data processing parameters from the storage area through the core of the accelerator card. Based on the data processing parameters, read the data to be extracted and the extraction rule data from the storage area. Perform data extraction on the data to be extracted based on the extraction rule data, and store the data extraction results of the data to be extracted in the storage area.

[0048] refer to Figure 4 When performing a data extraction task through the accelerator card 120, the core 122 can read the data processing parameters C in the storage area 124. Based on the data processing parameters C, the data to be extracted A and the extraction rule data B are read from the storage area 124. The extraction rule data B is used to perform the data extraction operation on the data to be extracted A to obtain the data extraction result S of the data to be extracted A, and the data extraction result S is returned to the storage area 124.

[0049] In some embodiments, core 1122 includes a pending register 1222, a processed register 1224, and a rule register 1226.

[0050] refer to Figure 5 It can read the data to be extracted A and the extraction rule data B from the storage area, cache the data to be extracted A in the processing register 1222, cache the extraction rule data B in the rule register 1224, use the character extraction rules in the extraction rule data B to perform data extraction on each string to be extracted in the data to be extracted A, and obtain the string extraction results corresponding to each string to be extracted. The string extraction results can be concatenated and cached in the processed register 1226 to obtain the data extraction result S, and return the data extraction result S in the processed register 1226 to the storage area 124.

[0051] In some embodiments, the data processing parameter C includes the total amount of data A to be read and the total amount of data processed.

[0052] Data processing parameters can be read from storage area 124, and the total amount of reading and processing in data processing parameter C can be identified. The data to be extracted A and the extraction rule data B in storage area 124 can be read in a loop until the actual amount of reading of data A matches the total amount of reading. The extraction rule data can be used in a loop to extract data until the actual amount of processing of data to be extracted matches the total amount of processing.

[0053] In some embodiments, the data to be extracted A and the extraction rule data B in the storage area 124 can be read in a loop based on an independent thread, and the extraction rule data can be reused to perform data extraction on the data to be extracted, so as to improve the efficiency of data extraction and processing.

[0054] For example, the total number of reads can be the total number of bytes of data A to be extracted, which is used to identify the total number of bytes of data A to be extracted that need to be read from storage area 120.

[0055] For example, the total processing quantity can be the number of input strings to be extracted, used to identify the total number of strings to be extracted that need to be extracted. Alternatively, if each row corresponds to one string to be extracted from data A, the total processing quantity can also be the total number of rows in data A.

[0056] In some embodiments, for any target string among the strings to be extracted, the character extraction rules corresponding to the target string can be read, and the target string can be selectively extracted based on the recognition of the character extraction rules.

[0057] In some embodiments, if the character extraction rules include a first extraction rule, the string to be extracted is extracted; if the character extraction rules include a second extraction rule, the string to be extracted is not extracted. For example, if the mask value corresponding to the string to be extracted is 1, the string to be extracted is extracted and output; if the mask value corresponding to the string to be extracted is 0, it means that no data extraction and output is required.

[0058] For example, based on the strings to be extracted shown in Table 1 below and the character extraction rules corresponding to each string to be extracted, the string extraction results corresponding to each string to be extracted shown in Table 2 below can be obtained:

[0059]

[0060]

[0061] (Table 1)

[0062]

[0063] (Table 2)

[0064] Step 206: Return the data extraction results from the storage area to the main computing device.

[0065] In some embodiments, the data extraction result S in the storage area 124 can be returned to the memory 114 via the PCIe channel between the accelerator card 120 and the main computing device 110.

[0066] In summary, the data extraction method provided in this embodiment transmits data information related to data extraction to an accelerator card, which then performs the data extraction process and returns the obtained data extraction results to the main computing device. This achieves the goal of offloading the data extraction function from the traditional CPU to a heterogeneous system composed of accelerator cards, which can significantly improve the processing efficiency of data extraction.

[0067] Furthermore, after the data extraction task is offloaded to the accelerator card for execution, the main computing device's processor (CPU) only needs to send some simple data extraction task instructions and does not need to be responsible for heavy computing tasks. This effectively frees up CPU utilization and provides system computing power for the CPU to handle other tasks.

[0068] Figure 6 This disclosure includes a flowchart of a data extraction method according to another exemplary embodiment. (See attached flowchart.) Figure 6 As shown, the data extraction method 600 of this embodiment includes the following steps:

[0069] Step 602: Determine the data buffer length of the data to be extracted in the register to be processed based on the current position of the pointer in the register to be processed.

[0070] In some embodiments, the register to be processed 1222 may include a plurality of consecutive storage blocks, each storage block being used to store a variable-length string to be extracted.

[0071] For example, in Figure 7 In the example shown, the register to be processed 1222 includes three storage blocks, storage blocks, and storage blocks, which can store the first line of string, the second line of string to be extracted, and the third line of string to be extracted, respectively.

[0072] In some embodiments, each storage block includes two storage units, each storage unit being used to store an identifier character field or a content character field for each string to be extracted (see reference). Figure 7 ).

[0073] In this embodiment, the initial position of the pointer is located at the first byte position of the first storage block among the multiple storage blocks of the register to be processed 1222 (reference). Figure 8A The pointer's current position corresponds to the first byte position of a memory cell (see reference). The data to be extracted, A, is read from register 1222 in units of memory, with each memory cell being read as a unit. Therefore, each current position of the pointer should correspond to the first byte position of a memory cell (see reference). Figures 8A to 8H ).

[0074] Step 604: Determine whether the data cache length is greater than the preset length threshold. If not, proceed to step 606; if yes, proceed to step 608.

[0075] In some embodiments, a preset length threshold can be determined based on the bit width of the register to be processed. Generally, the preset length threshold can be set to half the total length of the register to be processed. For example, if the bit width of the register to be processed 1222 is 128 bytes, the preset length threshold can be set to 64 bytes (see reference). Figure 7 (The dashed dividing line in the middle).

[0076] refer to Figure 7 In this embodiment, when the current position of the pointer does not exceed the dashed dividing line in the register to be processed 1222, it means that the number of remaining untrunculated bytes in the register to be processed 1222 is greater than the preset length threshold; conversely, when the current position of the pointer exceeds the dashed dividing line in the register to be processed 1222, it means that the number of remaining untrunculated bytes in the register to be processed 1222 is less than or equal to the preset length threshold.

[0077] Step 606: Read the padding data that meets the preset length threshold from the data to be extracted in the storage area, cache it in the register to be processed, and update the current position of the pointer.

[0078] In this embodiment, the preset length threshold is the maximum length of the string to be extracted. Therefore, when it is determined that the data cache length is less than or equal to the preset length threshold (for example, the data cache length is ≤64 bytes), it means that the remaining undextended data in the processing register 1222 may not be able to satisfy the truncation operation of the next string to be extracted, and a data supplement reading operation is performed.

[0079] refer to Figure 8G It can read 64 bytes of padding data from the data to be extracted in storage area 124, and reassemble the padding data with the currently cached data to be extracted in the processing register 1222 to form a new 128-byte wide processing register 1222 (see reference). Figure 8H ).

[0080] In some embodiments, when the data cache length is not greater than a preset length threshold (i.e., after data padding operation is performed), the current position of the pointer is updated in the following manner:

[0081] Based on the current position of the pointer and the preset length threshold of the padding data, perform the difference calculation, and update the current position of the pointer according to the difference calculation result.

[0082] For example, if the preset length threshold is 64 bytes and the current pointer position points to the 70th byte in the register 1222 to be processed, subtracting the preset length threshold from the current pointer position will cause the pointer position to move back 64 bytes and point to the 6th byte in the register 1222 to be processed (see reference). Figure 8G and Figure 8H ).

[0083] Step 608: Extract a string from the data to be extracted in the processing register to determine it as the target string, and selectively extract the target string using the character extraction rules in the rule register, and update the current position of the pointer.

[0084] In this embodiment, the preset length threshold is the maximum length of the string to be extracted. Therefore, when it is determined that the length of the data cache is greater than the preset length threshold (for example, the length of the data cache is greater than 64 bytes), it means that the remaining untrimmed data in the processing register 1222 can satisfy the truncation operation of the next string to be extracted, and then the data truncation operation is performed.

[0085] refer to Figure 8C When it is necessary to extract the first line of the string to be extracted from the register 1222, the actual length of the content segment of the first line of the string to be extracted can be determined to be 5 bytes based on the identifier segment of the first line of the string to be extracted. Based on the fixed length (4 bytes) of the identifier segment of the first line of the string to be extracted and the actual length (5 bytes) of the content segment, the total length of the first line of the string to be extracted can be determined to be 9 bytes, and the first line of the string to be extracted can be extracted from the register 1222 accordingly (see reference). Figure 8D ).

[0086] The first line of the string to be extracted can be used as the target string. The character extraction rules corresponding to the target character segment can be obtained from the rule register 1224, and the corresponding data extraction operation can be performed on the target string accordingly.

[0087] Specifically, when the character extraction rule includes the first extraction rule (e.g., mask value is 1), the target string is extracted; when the character extraction rule includes the second extraction rule (e.g., mask value is 0), the target string is not extracted.

[0088] In some embodiments, the rule register 1224, which can be used to cache and extract rule data B, can be a 64-bit shift register.

[0089] refer to Figure 9In the embodiment that uses mask values ​​to identify character extraction rules, each line of the string to be extracted from the processing register 1222 corresponds to a mask value, and each mask value has a width of 1 bit. A 64-bit extraction rule data B (containing the mask values ​​of 64 lines of strings to be extracted) can be read from the storage area 124 (DDR) and cached in the rule register 1224. The least significant bit of the rule register 1224 (shift register) is used as the mask value of the target string, and data extraction processing is selectively performed on the target string (i.e., when the mask value is 1, the target string is extracted and output). At the same time, the rule register 1224 is shifted right by one bit to update the mask value of the least significant bit in the rule register 1224. After the rule register 1224 is shifted right 63 times, all mask values ​​currently stored in the rule register 1224 have been used. Then, the next 64-bit extraction rule data B is read from the storage area 124 (DDR), and the above operation is repeated until all the strings to be extracted in the data to be extracted have been processed (that is, the actual amount of data to be extracted matches the total amount of data processed).

[0090] In some embodiments, when the data cache length is greater than a preset length threshold (i.e., after a data truncation operation has been performed), the current position of the pointer can be updated in the following way:

[0091] Based on the current position of the pointer, determine the current storage block pointed to by the pointer among multiple storage blocks, identify the identifier character field in the current storage block, determine the actual length of the content character field in the current storage block, and perform a summation calculation based on the current position of the pointer, the fixed length of the identifier character field, and the actual length of the content character field to determine the target position of the pointer. Move the pointer based on the target position and update the target position to the current position of the pointer.

[0092] For example, according to Figure 8A Based on the current position of the pointer (e.g., pointing to the first byte in the register 1222 to be processed), it can be determined that the pointer is pointing to the first byte of the identifier segment of the second line of the string to be extracted. Therefore, the storage block in the register 1222 to be processed that stores the second line of the string to be extracted is determined as the current storage block. The identifier segment of the second line of the string to be extracted is identified, and the actual length of the content segment of the second line of the string to be extracted is found to be 6 bytes. The total length of the second string is determined to be 10 bytes (i.e., 4 bytes of identifier segment and 6 bytes of content segment). Based on the sum of the current position of the pointer (the first byte) and the total length of the second string (10), it can be determined that the target position of the pointer is the 11th byte in the register 1222 to be processed. The pointer is moved according to the target position, and the target position is changed to the current position of the pointer (refer to the reference). Figure 8C ).

[0093] In some embodiments, the movement of the pointer may include:

[0094] refer to Figure 8A Based on the current position of the pointer, it is determined that the pointer points to the identifier segment of the first line of the string to be extracted in register 1224. The stored information in the identifier segment is then identified, confirming that the actual length of the content segment of the first line of the string to be extracted is 10 bytes. Based on the fixed length of the identifier segment (4 bytes), the control pointer is moved from the first byte of the identifier segment to the first byte of the content segment of the first line of the string to be extracted (see reference). Figure 8B Based on the actual length of the content character segment (10 bytes), the control pointer is moved from the first byte position of the content character segment of the first line of the string to be extracted to the first byte position of the storage block storing the second line of the string to be extracted in the processing register 1222, which is also the first byte position of the identifier character segment of the second line of the string to be extracted (see reference). Figure 8C ).

[0095] The following will combine Figures 8A to 8H The specific implementation process of this embodiment is described in detail below:

[0096] refer to Figure 8A In the initial state, the core 122 continuously reads two 64-byte data A to be extracted from the storage area 124, concatenates them into 128 bytes, and caches them in the processing register 1222.

[0097] The pointer is initially positioned to the first byte of the first memory block of register 1222, which is also the first byte of the identifier segment of the first line of the string to be extracted. The pointer's current position is then checked to see if it is greater than 64 bytes. Figure 8A In the example shown, the current position of the pointer does not exceed 64 bytes (preset length threshold), which means that the number of bytes that have not yet been truncated in the register 1222 to be processed (i.e. the data buffer length of the data to be extracted in the register to be processed) is greater than 64 bytes. Therefore, the first line of the string to be extracted is truncated.

[0098] Since the character identifier segment of each string to be extracted is fixed at 4 bytes, after reading 4 bytes from the current position of the pointer, the jump proceeds to the first byte position of the content segment of the first line of the string to be extracted (see reference). Figure 8B ), and based on the actual length of the content character segment identified in the identifier character segment, read the content character segment to obtain the complete information of the first line of the string to be extracted (see reference). Figure 8C ), and based on this, the first line of the string to be extracted is truncated to obtain, as shown below. Figure 8D The result shown is the cropping result.

[0099] After completing the substring extraction operation on the first line of the string to be extracted, check if the current pointer position is greater than 64 bytes, and then... Figure 8D As shown, the current position of the pointer does not exceed 64 bytes (preset length threshold), indicating that the number of bytes yet to be truncated in the register 1222 to be processed is greater than 64 bytes. Therefore, the truncation operation on the second line of the string to be extracted will continue.

[0100] like Figure 8D As shown, based on the current position of the pointer (i.e., the first byte position of the identifier segment of the string to be extracted in the second line), after reading 4 bytes, it jumps to the first byte position of the content segment of the string to be extracted in the second line (see reference). Figure 8E ), and based on the actual length of the content character segment identified in the identifier character segment, read the content character segment to obtain the complete information of the second line of the string to be extracted (see reference). Figure 8F ), and based on this, the second line of the string to be extracted is truncated to obtain the following result. Figure 8G The result shown is the cropping result.

[0101] After completing the substring extraction operation on the second line, check if the current pointer position is greater than 64 bytes. Figure 8G As shown, the current pointer position has exceeded 64 bytes (the preset length threshold), indicating that the number of bytes yet to be truncated in the processing register 1222 (i.e., the data buffer length of the data to be extracted in the processing register) is no more than 64 bytes. This may not be sufficient to satisfy the truncation operation for the next line of the string to be extracted. Therefore, a new 64-byte data A to be extracted is read from storage area 124 and reassembled with the data A cached in the processing register 1222 to form a new 128-byte processing register 1222, ensuring that the number of bytes yet to be truncated in the processing register 1222 is greater than 64 bytes (see reference). Figure 8H The pointer is then moved back 64 bytes to update its current position based on the data padding operation.

[0102] according to Figure 8H As can be seen, after performing the data padding operation and updating the current position of the pointer, if the current position of the pointer does not exceed 64 bytes (the preset length threshold), then the truncation operation on the third line of the string to be extracted can continue.

[0103] In some embodiments, the same data caching principle as the processing register 1222 can be used to concatenate the extraction results of each string corresponding to each string to be extracted into a data extraction result S and cache it in the processed register 1226, and then return the data extraction result in the processed register 1226 to the storage area 124.

[0104] Specifically, the extraction results of each string corresponding to each string to be extracted can be concatenated into a data extraction result S and cached in the processed register 1226. The processed register 1226 can write the data extraction result S in batches into the storage area 124 in 64-byte units.

[0105] The data buffer length of the data extraction result S in the processed register 1226 can be determined based on the current position of the pointer in the processed register 1226. If the data buffer length is not greater than the preset length threshold, the string extraction result of the string to be extracted is obtained and cached in the processed register 1226, and the current position of the pointer is updated. If the data buffer length is greater than the preset length threshold, the write data that meets the preset length threshold is extracted from the data extraction result S in the processed register 1226, the write data is stored in the storage area 124, and the current position of the pointer is updated.

[0106] refer to Figure 10A The pointer can be initialized to point to the first byte of the processed register 1226. Then, it checks if the current pointer position is greater than 64 bytes. Figure 10A In the example shown, if the current position of the pointer does not exceed 64 bytes (the preset length threshold), then the caching operation of the first line of string extraction result is performed.

[0107] Retrieve the extracted result of the first line of string and cache it in the processed register 1226. Update the current position of the pointer according to the full length of the extracted result of the first line of string so that the pointer points to the position of the last byte of the extracted result of the first line of string (see reference). Figure 10B ).

[0108] After completing the caching operation for the first line of string extraction results, check if the current pointer position is greater than 64 bytes, and then... Figure 10B As shown, if the current pointer position is still less than 64 bytes (the preset length threshold), the caching operation for the extracted string result of the second line continues, and the current pointer position is updated so that the pointer points to the last byte position of the extracted string result of the second line (see reference). Figure 10C ).

[0109] After completing the caching operation for the second line of string extraction results, check if the current pointer position is greater than 64 bytes, and then... Figure 10C As shown, if the current position of the pointer has exceeded 64 bytes (the preset length threshold), then the data extraction result S will be written.

[0110] Extract 64 bytes of write data from the data extraction result S cached in register 1226, write the write data to memory area 124, and subtract 64 bytes (i.e., the length of the written data) from the current position of the pointer, so that the current position of the updated pointer is less than 64 bytes (a preset length threshold), so that the caching operation of the string extraction result in the third line can continue (see reference). Figure 10D ).

[0111] In summary, the data extraction method provided in this embodiment, through the design mechanism of registers and pointers, divides the reading operation of the data to be extracted, the data extraction operation, and the writing operation of the data extraction result into independent processing threads, which can meet the processing needs of large batches of data, improve data extraction efficiency, and realize the data extraction function of variable length string type.

[0112] Figure 11 This is a structural block diagram of a data extraction apparatus 1100 as an exemplary embodiment of this application. As shown, the data extraction apparatus 100 of this embodiment can be applied to an accelerator card coupled to a main computing device (see reference). Figure 1 ), and includes:

[0113] The reading module 1102 is used to obtain the data to be extracted, the extraction rule data, and the data processing parameters from the main computing device and store them in the storage area of ​​the accelerator card;

[0114] The extraction module 1104 is used to read the data processing parameters from the storage area through the core of the accelerator card, read the data to be extracted and the extraction rule data from the storage area according to the data processing parameters, perform data extraction on the data to be extracted based on the extraction rule data, and store the data extraction result of the data to be extracted in the storage area.

[0115] The return module 1106 is used to return the target data in the storage area to the main computing device.

[0116] Optionally, the data processing parameters include the total amount of data to be read and the total amount of data to be processed.

[0117] Optionally, the reading module 1102 is further configured to: read the data processing parameters from the storage area, identify the total reading amount and the total processing amount in the data processing parameters; cyclically read the data to be extracted and the extraction rule data in the storage area until the actual reading amount of the data to be extracted matches the total reading amount; and cyclically use the extraction rule data to perform data extraction on the data to be extracted until the actual processing amount of the data to be extracted matches the total processing amount.

[0118] Optionally, the core includes a pending register, a processed register, and a rule register; the data to be extracted includes multiple strings to be extracted; and the extraction rule data includes character extraction rules corresponding to each string to be extracted.

[0119] Optionally, the extraction module 1104 is further configured to: read the data to be extracted and the extraction rule data from the storage area, and cache them in the processing register and the rule register respectively; perform data extraction on each string to be extracted in the data to be extracted using the character extraction rules in the extraction rule data, and obtain the string extraction results corresponding to each string to be extracted; concatenate the string extraction results and cache them in the processed register to obtain the data extraction result; and return the data extraction result in the processed register to the storage area.

[0120] Optionally, the register to be processed includes a pointer; the extraction module 1104 is further configured to: determine the data cache length of the data to be extracted in the register to be processed based on the current position of the pointer in the register to be processed; if the data cache length is not greater than a preset length threshold, read the padding data that meets the preset length threshold from the data to be extracted in the storage area, cache it in the register to be processed, and update the current position of the pointer; if the data cache length is greater than the preset length threshold, extract a string to be extracted from the data to be extracted in the register to be processed to determine it as the target string, and selectively extract the target string using the character extraction rules in the rule register, and update the current position of the pointer.

[0121] Optionally, the extraction module 1104 is further configured to: when the data cache length is not greater than the preset length threshold, perform a difference calculation based on the current position of the pointer and the preset length threshold of the padding data, and update the current position of the pointer based on the difference calculation result.

[0122] Optionally, each string to be extracted includes an identifier character field of fixed length and a content character field of variable length, and the register to be processed includes multiple consecutive storage blocks, each storage block being used for one string to be extracted.

[0123] Optionally, the extraction module 1104 is further configured to: when the data cache length is greater than the preset length threshold, determine the current storage block pointed to by the pointer among the plurality of storage blocks according to the current position of the pointer; identify the identifier character segment in the current storage block, determine the actual length of the content character segment in the current storage block, and perform a summation calculation based on the current position of the pointer, the fixed length of the identifier character segment, and the actual length of the content character segment to determine the target position of the pointer; move the pointer based on the target position and update the target position to the current position of the pointer.

[0124] Optionally, the extraction module 1104 is further configured to: read the character extraction rule corresponding to the target string from the rule register; identify the character extraction rule; if the character extraction rule includes a first extraction rule, extract the target string; if the character extraction rule includes a second extraction rule, do not extract the target string.

[0125] This application provides an accelerator card, including: a storage area and a core. The storage area is used to store data to be extracted, extraction rule data, data processing parameters, and data extraction results of the data to be extracted, obtained from the main computing device. The core reads the data processing parameters from the storage area through the core of the accelerator card, reads the data to be extracted and the extraction rule data from the storage area according to the data processing parameters, and performs data extraction on the data to be extracted based on the extraction rule data to obtain the data extraction results of the data to be extracted.

[0126] An exemplary embodiment of this application provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores a computer program executable by the at least one processor, which, when executed by the at least one processor, causes the electronic device to perform a data extraction method according to various exemplary embodiments of this application.

[0127] Please refer to Figure 12The present invention describes a structural block diagram of an electronic device 1200 that can serve as a server or client of this application, which is an example of a hardware device that can be applied to various aspects of this application. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the application described and / or claimed herein.

[0128] like Figure 12 As shown, the electronic device 1200 includes a computing unit 1201, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1202 or a computer program loaded from a storage unit 1208 into a random access memory (RAM) 1203. The RAM 1203 may also store various programs and data required for the operation of the device 1200. The computing unit 1201, ROM 1202, and RAM 1203 are interconnected via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0129] Multiple components in electronic device 1200 are connected to I / O interface 1205, including: input unit 1206, output unit 1207, storage unit 1208, and communication unit 1209. Input unit 1206 can be any type of device capable of inputting information to electronic device 1200. Input unit 1206 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of electronic device. Output unit 1207 can be any type of device capable of presenting information and may include, but is not limited to, a display, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 1208 may include, but is not limited to, disk and optical disk. Communication unit 1209 allows electronic device 1200 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth™ devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0130] The computing unit 1201 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1201 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1201 performs the various methods and processes described above. For example, in some embodiments, the data extraction methods described above can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 1208. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 1200 via ROM 1202 and / or communication unit 1209. In some embodiments, the computing unit 1201 can be configured to perform the data extraction methods described above by any other suitable means (e.g., by means of firmware).

[0131] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0132] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0133] As used in this application, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0134] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0135] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or middleware components (e.g., an application server), or frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0136] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other.

[0137] It should be noted that, depending on the implementation needs, the various components / steps described in the embodiments of this application can be broken down into more components / steps, or two or more components / steps or parts of the operation of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of this application.

[0138] The above embodiments are only used to illustrate the embodiments of this application, and are not intended to limit the embodiments of this application. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of this application. Therefore, all equivalent technical solutions also fall within the scope of the embodiments of this application, and the patent protection scope of the embodiments of this application should be defined by the claims.

Claims

1. A data extraction method applied to an accelerator card coupled to a main computing device, characterized in that, The method includes: The system obtains the data to be extracted, the extraction rule data, and the data processing parameters from the main computing device and stores them in the storage area of ​​the accelerator card. The data processing parameters are read from the storage area through the core of the accelerator card. Based on the data processing parameters, the data to be extracted and the extraction rule data are read from the storage area. Data extraction is performed on the data to be extracted based on the extraction rule data, and the data extraction result of the data to be extracted is stored in the storage area. Return the data extraction results from the storage area to the main computing device; The kernel includes a register to be processed, a processed register, and a rule register; the data to be extracted includes multiple strings to be extracted; and the extraction rule data includes character extraction rules corresponding to each string to be extracted. The step of reading the data to be extracted and the extraction rule data from the storage area, performing data extraction on the data to be extracted based on the extraction rule data, and storing the data extraction result of the data to be extracted in the storage area includes: The data to be extracted and the extraction rule data are read from the storage area and cached in the processing register and the rule register, respectively. Using the character extraction rules in the extraction rule data, data extraction is performed on each string to be extracted in the data to be extracted, and the extraction results of each string corresponding to each string to be extracted are obtained. The results of each string extraction are concatenated and cached in the processed register to obtain the data extraction result; Return the data extraction result from the processed register to the storage area.

2. The method according to claim 1, characterized in that, The data processing parameters include the total amount of data to be read and the total amount of data to be processed; The step of reading the data processing parameters from the storage area, reading the data to be extracted and the extraction rule data from the storage area according to the data processing parameters, and performing data extraction on the data to be extracted based on the extraction rule data includes: Read the data processing parameters from the storage area and identify the total amount read and the total amount processed in the data processing parameters; The data to be extracted and the extraction rule data in the storage area are read in a loop until the actual amount of data to be extracted matches the total amount of data to be read. The extraction rule data is used repeatedly to extract data from the data to be extracted until the actual amount of data to be extracted matches the total amount of data to be processed.

3. The method according to claim 1, characterized in that, The register to be processed includes a pointer; The step of reading the data to be extracted and the extraction rule data from the storage area, and performing data extraction on the data to be extracted based on the extraction rule data, includes: Based on the current position of the pointer in the register to be processed, determine the data buffer length of the data to be extracted in the register to be processed; If the data cache length is not greater than a preset length threshold, read the padding data that meets the preset length threshold from the data to be extracted in the storage area, cache it in the register to be processed, and update the current position of the pointer; If the data cache length is greater than the preset length threshold, a string to be extracted is extracted from the data to be extracted in the processing register to be determined as the target string, and the target string is selectively extracted using the character extraction rules in the rule register, and the current position of the pointer is updated.

4. The method according to claim 3, characterized in that, If the data cache length is not greater than the preset length threshold, the current position of the pointer is updated in the following way: Based on the current position of the pointer and the preset length threshold of the padding data, a difference calculation is performed, and the current position of the pointer is updated based on the difference calculation result.

5. The method according to claim 3, characterized in that, Each string to be extracted includes an identifier character field of fixed length and a content character field of variable length. The register to be processed includes multiple consecutive storage blocks, each storage block being used for one string to be extracted. Wherein, if the data cache length is greater than the preset length threshold, the current position of the pointer is updated in the following manner; Based on the current position of the pointer, determine the current storage block pointed to by the pointer among the plurality of storage blocks; Identify the identifier character segment in the current storage block, determine the actual length of the content character segment in the current storage block, and perform a summation calculation based on the current position of the pointer, the fixed length of the identifier character segment, and the actual length of the content character segment to determine the target position of the pointer; The pointer is moved based on the target position, and the target position is updated to the current position of the pointer.

6. The method according to claim 3, characterized in that, The step of selectively extracting the target string using the character extraction rules in the rule register includes: Read the character extraction rules corresponding to the target string from the rule register; Identify the character extraction rules; if the character extraction rules include a first extraction rule, extract the target string; if the character extraction rules include a second extraction rule, do not extract the target string.

7. A data extraction device, applied to an accelerator card coupled to a main computing device, characterized in that, The device includes: The reading module is used to obtain the data to be extracted, the extraction rule data, and the data processing parameters from the main computing device, and store them in the storage area of ​​the accelerator card; The extraction module is used to read the data processing parameters from the storage area through the core of the accelerator card, read the data to be extracted and the extraction rule data from the storage area according to the data processing parameters, perform data extraction on the data to be extracted based on the extraction rule data, and store the data extraction result of the data to be extracted in the storage area. The return module is used to return the data extraction results from the storage area to the main computing device; The kernel includes a register to be processed, a processed register, and a rule register; the data to be extracted includes multiple strings to be extracted; and the extraction rule data includes character extraction rules corresponding to each string to be extracted. The extraction module is configured to: read the data to be extracted and the extraction rule data from the storage area, and cache them in the processing register and the rule register, respectively; perform data extraction on each string to be extracted in the data to be extracted using the character extraction rules in the extraction rule data, and obtain the string extraction results corresponding to each string to be extracted; concatenate the string extraction results and cache them in the processed register to obtain the data extraction result; and return the data extraction result in the processed register to the storage area.

8. An accelerator card, characterized in that, include: The storage area is used to store the data to be extracted, extraction rule data, data processing parameters, and data extraction results obtained from the main computing device; The core of the accelerator card reads the data processing parameters from the storage area, reads the data to be extracted and the extraction rule data from the storage area according to the data processing parameters, performs data extraction on the data to be extracted based on the extraction rule data, and obtains the data extraction result of the data to be extracted. The kernel includes a register to be processed, a processed register, and a rule register; the data to be extracted includes multiple strings to be extracted; and the extraction rule data includes character extraction rules corresponding to each string to be extracted. The step of reading the data to be extracted and the extraction rule data from the storage area, performing data extraction on the data to be extracted based on the extraction rule data, and obtaining the data extraction result of the data to be extracted includes: The data to be extracted and the extraction rule data are read from the storage area and cached in the processing register and the rule register, respectively. Using the character extraction rules in the extraction rule data, data extraction is performed on each string to be extracted in the data to be extracted, and the extraction results of each string corresponding to each string to be extracted are obtained. The results of each string extraction are concatenated and cached in the processed register to obtain the data extraction result; Return the data extraction result from the processed register to the storage area.

9. A computer storage medium, characterized in that, The computer storage medium stores computer program code, which, when executed by a processor, causes the processor to perform the method according to any one of claims 1-6.

Images