Data processing method, electronic equipment and storage medium
By constructing array vectors, storing arrays with the same type and similar meanings as value arrays and indexing arrays, the problem of low data query and calculation efficiency in the prior art is solved, and more efficient computing performance and data processing efficiency are achieved.
Patent Information
- Application Number
- CN202510325254.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-19
- Publication Date
- 2025-07-04
AI Technical Summary
In the prior art, data query and calculation efficiency are inefficient because multiple memory search jumps are required between multiple arrays.
By constructing an array vector, the values of arrays with the same type and similar meaning are stored as value arrays, and the positions of each array in the value array are stored as index arrays. Therefore, when executing calculation instructions, you only need to determine the array elements that need to be operated in the value array based on the index array, and reduce the number of memory jumps.
It realizes more efficient data query and computing performance, reduces the number of memory access jumps between arrays, and improves computing efficiency.
Smart Images
Figure CN120256430A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, and in particular, to a data processing method, an electronic device, and a storage medium. Background Art
[0002] Databases often need to store multiple data columns with the same type and similar meanings, such as multi-level stock quote data in the financial scenario and multi-dimensional sensor data in the Internet of Things scenario. In the prior art, different data columns are generally stored as different arrays. When performing data query or calculation, data can be obtained by querying the arrays, and calculation results can be obtained by calculating the data.
[0003] However, when performing data query and data calculation using the prior art, multiple memory access jumps are required between multiple arrays, which affects the efficiency of data query and data calculation. Summary of the Invention
[0004] The purpose of this application is to provide a data processing method, an electronic device, and a storage medium for solving the problem of low efficiency of data query and data calculation in the prior art, aiming at the deficiencies in the above prior art.
[0005] To achieve the above purpose, the technical solutions adopted in this application are as follows:
[0006] In a first aspect, this application provides a data processing method, and the method includes:
[0007] Construct an array vector according to multiple initial arrays, where the array vector includes: a value array and an index array. The value array includes array elements of each of the initial arrays arranged in sequence. Each array element in the index array corresponds to one of the initial arrays, and the value of each array element is used to indicate the position of the last array element of the corresponding initial array in the value array. The data types stored in each of the initial arrays are the same, and the array elements at the same position in each of the initial arrays correspond to the same list header field or row header field;
[0008] Receive a data processing instruction;
[0009] Determine at least one target array element from at least one value array according to the data processing instruction and the index array, and perform data processing on each of the target array elements to obtain a data processing result.
[0010] Optionally, the constructing an array vector according to multiple initial arrays includes:
[0011] Add each array element of the first initial array to the value-taking array. The arrangement order of the array elements in the value-taking array is the same as that of the array elements in the initial array, and use the current number of array elements in the value-taking array as the first array element in the index array;
[0012] Add the array elements of the next initial array to the value-taking array, and use the current number of array elements in the value-taking array as the next array element in the index array;
[0013] Repeat the step of adding the array elements of the next initial array to the value-taking array and using the current number of array elements in the value-taking array as the next array element in the index array until all the array elements of all the initial arrays are added to the value-taking array. Use the obtained value-taking array and index array as the array vector.
[0014] Optionally, the data processing instruction is: the first calculation instruction;
[0015] The step of determining at least one target array element from at least one value-taking array according to the data processing instruction and the index array, and performing data processing on each of the target array elements to obtain a data processing result includes:
[0016] Determine all the array elements in the value-taking array as target array elements;
[0017] Perform calculations on each of the target array elements according to the first calculation instruction to obtain the data processing result.
[0018] Optionally, the data processing instruction is: a row calculation instruction or a column calculation instruction;
[0019] The step of determining at least one target array element from at least one value-taking array according to the data processing instruction and the index array, and performing data processing on each of the target array elements to obtain a data processing result includes:
[0020] If the data processing instruction is a row calculation instruction, determine at least one target array element in each row in the value-taking array according to the array elements in the index array, and perform calculations on the target array elements in each row to obtain the data processing result for each row;
[0021] If the data processing instruction is a column calculation instruction, determine at least one target array element in each column in the value-taking array according to the array elements in the index array, and perform calculations on the target array elements in each column to obtain the data processing result for each column.
[0022] Optionally, determining at least one target array element for each row in the value array according to the array elements in the index array includes:
[0023] Determining a start position offset and an end position offset corresponding to each of the array elements according to the array elements in the index array;
[0024] Taking the array elements between the start position offset and the end position offset corresponding to the array element as the target array element for the current row.
[0025] Optionally, determining at least one target array element for each column in the value array according to the array elements in the index array includes:
[0026] A. Determining the number of columns according to the difference between adjacent array elements in the index array;
[0027] B. Taking 0 as the initial offset and the value obtained by subtracting 1 from the first array element in the index array as the end offset, and determining the first target array element for each column in the value array based on the initial offset and the end offset;
[0028] C. Taking the first array element in the index array as the initial offset and the next adjacent array element as the end offset, querying backward from the position indicated by the initial offset in the value array. If the offset of the current target array element is greater than or equal to the end offset, setting the value of the target array element in the current column to be empty. If the offset of the current target array element is less than the end offset, taking the current target array element as the target array element for the current column;
[0029] D. Taking the next array element in the index array as the new initial offset for each column and the next adjacent array element as the end offset, and repeating steps C - D until the current target array element is the last array element in the value array, to obtain at least one target array element for each column.
[0030] Optionally, the data processing instruction is: a correlation coefficient calculation instruction, and the at least one value array includes: a first value array and a second value array;
[0031] Determining at least one target array element from at least one value array according to the data processing instruction and the index array, and performing data processing on each of the target array elements to obtain a data processing result, includes:
[0032] Determine at least one first target array element from the first value array according to the data processing instruction and the index array, and determine the corresponding second target array elements of each of the first target array elements from the second value array;
[0033] Determine the correlation coefficients of each of the first target array elements and each of the second target array elements, and obtain the data processing results of each of the first target array elements and each of the second target array elements.
[0034] Optionally, the method further includes:
[0035] Receive a value array summation instruction;
[0036] According to the value array summation instruction, sum the array elements at the corresponding positions in the first value array and the second value array to obtain new array elements at each position, and form an array composed of each of the new array elements as a new value array.
[0037] In a second aspect, an embodiment of the present application further provides an electronic device, including: a processor, a storage medium, and a bus. The storage medium stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to perform the steps of a data processing method as described in any one of the first aspects.
[0038] In a third aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it performs the steps of a data processing method as described in any one of the first aspects.
[0039] The beneficial effects of the present application are as follows: By storing multiple rows of data in a value array in a combined manner and storing the index values of each row of data in an index array, the data is aggregated in an array in memory during calculation. When calculating two arrays, there is no need to perform memory access jumps between multiple arrays as in the prior art. Only need to determine the position of the data in the value array based on the index array and then operate on the value array, realizing more convenient calculation and improving the calculation performance and data calculation efficiency.
[0040] To make the above objects, features, and advantages of the present application more obvious and understandable, the following specifically lists preferred embodiments and, in conjunction with the accompanying drawings, makes a detailed description as follows. Description of the Drawings
[0041] To more clearly illustrate the technical solutions of the embodiments of the present application, the accompanying drawings required for the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present application and should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.
[0042] Figure 1 Shows a flowchart of a data processing method provided by an embodiment of the present application;
[0043] Figure 2 Shows a flowchart of constructing an array vector provided by an embodiment of the present application;
[0044] Figure 3 Shows a flowchart of data calculation based on a first calculation instruction provided by an embodiment of the present application;
[0045] Figure 4 Shows a flowchart of data calculation based on a row calculation instruction and a column calculation instruction provided by an embodiment of the present application;
[0046] Figure 5 Shows a flowchart of determining the target array elements of each row provided by an embodiment of the present application;
[0047] Figure 6 Shows a flowchart of determining the target array elements of each column provided by an embodiment of the present application;
[0048] Figure 7 Shows a flowchart of data calculation based on a correlation coefficient calculation instruction provided by an embodiment of the present application;
[0049] Figure 8 Shows a flowchart of summing two value arrays provided by an embodiment of the present application;
[0050] Figure 9 Shows a schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners
[0051] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the following will clearly and completely describe the technical solutions in the embodiments of this application with reference to the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part, rather than all, of the embodiments of this application. Usually, the components of the embodiments of this application described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents the selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative efforts fall within the scope of protection of this application.
[0052] It should be noted that the term "including" will be used in the embodiments of this application to indicate the existence of the subsequently stated features, but does not exclude the addition of other features.
[0053] In the prior art, when storing data with similar meanings and the same type, generally, each piece of data is stored in a different array.
[0054] Taking the storage of multiple different quotes in the financial field as an example, assuming that it is necessary to store the three-level quote data of stock A at different times in a database, and each level of quote is represented by a column respectively, then the quote data table shown in Table 1 below can be obtained.
[0055] Table 1 Example of Quote Data Representation
[0056]
[0057] Referring to Table 1, data of different levels are stored in different columns. When storing Table 1 in memory, the data at different times can be stored as an array respectively, and all arrays are represented as a vector "quote [[30.13, 30.12, 30.11], [31.23, 31.31], [31.20, 30.55, 30.70]]", where the meanings of each array are as follows:
[0058] Time 1: [30.13, 30.12, 30.11]
[0059] Time 2: [31.23, 31.31]
[0060] Time 3: [31.20, 30.55, 30.70]
[0061] It can be seen that the quotation data at each moment is stored in different arrays respectively. At this time, if we want to determine the moment when Quotation 1 is the highest, we need to first find the starting address of the arrays at each moment in the memory, then obtain the quotations of Quotation 1 at different moments through the offset, and finally calculate the maximum value, that is, max(Quotation at Moment 1[1], Quotation at Moment 2[1], Quotation at Moment 3[1]).
[0062] However, when using the existing method for array storage, when querying or calculating the data in the arrays, especially when calculating the elements in multiple arrays, multiple memory access jumps need to be performed between multiple arrays, thus affecting the efficiency of data query and data calculation.
[0063] Based on this, the present application proposes a data processing method. By storing the values of arrays with the same type and similar meanings as a value-taking array, and storing the positions of each array in the value-taking array as an index array, when executing the calculation instructions originally for multiple arrays, it is only necessary to determine the array elements to be operated on in the value-taking array based on the index array, and only operate on the value-taking array, thereby greatly reducing the number of memory access jumps between arrays and improving the efficiency of data query and data calculation.
[0064] Next, in combination with Figure 1 , the data processing method of the present application will be described. The execution subject of this method can be an electronic device with data processing capabilities, such as Figure 1 shown, this method includes:
[0065] S101. Construct an array vector according to multiple initial arrays. The array vector includes: a value-taking array and an index array. The value-taking array includes the array elements of each initial array arranged in sequence. The array elements in the index array correspond to each initial array one by one, and the values of the array elements are used to indicate the positions of the last array elements of the corresponding initial arrays in the value-taking array. The data in the same data column / row with the same storage type are stored in each initial array, and the array elements at the same positions in each initial array have the same meaning.
[0066] Optionally, the storage format of the array vector in the memory can be as shown in Table 2 below. Wherein, the length of the index array is the number of array elements in the index array, and the length of the value-taking array is the number of array elements in the value-taking array.
[0067] Table 2 Storage Format of Array Vector
[0068] Index array length Index array content Value array length Value array content
[0069] Exemplarily, for an array vector composed of three initial arrays [[30.13, 30.12, 30.11], [31.23, 31.31], [31.20, 30.55, 30.70, 31.32]], its index array has three array elements, and its value array has nine array elements. The array elements in the index array are used to represent the positions of the last array elements of each initial array in the value array. The storage content of the array vector on the disk is shown in Table 3 below.
[0070] Table 3 Example of Array Vector Storage
[0071] 3 3,5,9 9 30.13,30.12,30.11,31.23,31.31,31.20,30.55,30.70,31.32
[0072] An initial array can be an array that stores multiple data rows or data columns of the same type. The data in each data row or data column is arranged in the initial array in the order from top to bottom and from left to right. Among them, the elements at the same position in the initial array represent the same meaning.
[0073] Optionally, the data types are the same, which can mean that the row headers corresponding to the data in the initial array are the same and in the same arrangement order, or the column headers corresponding to the data are the same and in the same arrangement order.
[0074] For example, when storing multi-level stock quotes in a financial scenario, the column header fields corresponding to each column of data respectively represent the quotes of different levels of the same stock, and the row header fields corresponding to each row of data respectively represent the quotes of the same stock at different times. Each column of data is stored as an initial array, and the first array element in each initial array respectively represents the stock quotes at different times under the first level. When storing Internet of Things sensor data, the column header fields corresponding to each column of data respectively represent the sensor data of different dimensions of the same measurement target, and the row header fields corresponding to each row of data respectively represent the sensor data of the same measurement target at different times. Each row of data is stored as an initial array, and the first array element of each initial array respectively represents the sensor measurement data of different dimensions at the first time.
[0075] In the first implementation, the data with the same field type shown in Table 1 can be stored by rows as multiple initial arrays. The data in each row is stored as an array, and the array elements in each initial array respectively represent a value in the table. Moreover, the order of the array elements in the initial array is the same as the order of the header fields in the first row. At this time, the row header fields corresponding to the array elements at the same position in the initial array are the same.
[0076] In the second implementation manner, the data with the same field type shown in Table 1 can be stored column by column as multiple initial arrays, and the data in each column is stored as an array. The order of the array elements in each initial array is the same as the order of the header fields in the first column. At this time, the list header fields corresponding to the array elements at the same position in the initial arrays are the same.
[0077] After obtaining multiple initial arrays, in the present application, the initial arrays can be merged into a value-taking array. The array elements in the value-taking array include the array elements of the initial arrays, and the array elements are arranged in sequence according to the order in the initial arrays. The number of elements in the initial arrays can be different, that is, the lengths of the initial arrays that make up the value-taking array can be different.
[0078] Exemplarily, for the 3 initial arrays obtained from Table 1, their array elements can be saved into the value-taking array to obtain the value-taking array "quotation [30.13, 30.12, 30.11, 31.23, 31.31, 31.20, 30.55, 30.70]", where "30.13, 30.12, 30.11" are the array elements of the first initial array, "31.23, 31.31" are the array elements of the second initial array, and "31.20, 30.55, 30.70" are the array elements of the third initial array.
[0079] In order to determine the positions of the array elements corresponding to each initial array in the value-taking array, an index array can be constructed while constructing the value-taking array. The value of the array element in the index array can be the offset of the last array element in the initial array in the value-taking array, which is used to represent the position of the last array element in each initial array in the value-taking array.
[0080] Exemplarily, for the value-taking array "quotation [30.13, 30.12, 30.11, 31.23, 31.31, 31.20, 30.55, 30.70]" composed of 3 initial arrays in Table 1, its corresponding index array can be expressed as [3, 5, 8], where "3" represents the position of the last array element in the first initial array in the value-taking array, "5" represents the position of the last array element in the second initial array in the value-taking array, and "8" represents the position of the last array element in the third initial array in the value-taking array.
[0081] It should be noted that array vectors can be constructed for multiple groups of initial arrays with similar meanings and the same type to obtain at least one array vector. In subsequent processing, based on the index array, array elements can be queried in the value-taking array, and calculations can be directly performed on the array elements, so there is no need to perform memory access jumps between multiple initial arrays.
[0082] S102. Receive a data processing instruction.
[0083] Optionally, the data processing instruction can be a query or calculation instruction for the array elements in the value-taking array of the current array vector, or a query instruction or calculation instruction for the value-taking arrays in two or more array vectors. Exemplarily, the data processing instruction can be a query or calculation instruction for the array elements in the array vector of the stock quotes in the above financial scenario, or a query or calculation instruction for the array elements in the array vector of multi-dimensional sensor data in the above Internet of Things field.
[0084] The data processing instruction can indicate the calculation operation to be performed, and can also indicate the position of the array elements to be calculated in the initial array.
[0085] S103. Determine at least one target array element from at least one value-taking array according to the data processing instruction and the index array, and perform data processing on each target array element to obtain a data processing result.
[0086] Optionally, the target array element can be the array element to be calculated indicated by the data processing instruction in the value-taking array.
[0087] The index array can indicate the position of the array elements in the initial array in the value-taking array. Therefore, according to the data processing instruction and the index array, the target array elements to be calculated can be determined, and the target array elements can be calculated according to the calculation method indicated by the data processing instruction to obtain a data processing result.
[0088] Exemplarily, assuming that it is necessary to perform a summation operation on the third initial array in the above example, the array elements corresponding to the third initial array can be determined from the value-taking array based on the index array, and the sum of each array element can be calculated. Assuming that it is necessary to query the maximum value of Quote 1 of the initial arrays at each moment in the above example, since the position of Quote 1 in each initial array is the first array element, the three array elements "20.13, 31.23, 31.20" can be queried from the value-taking array based on the index array, and the maximum value of these three array elements can be used as the query result.
[0089] It should be noted that the above S101 step can be executed multiple times to respectively construct multiple array vectors from multiple groups of initial arrays with similar meanings and the same type. If the index arrays of the array vectors are the same, array calculations can be performed on the value-taking arrays of each array vector. At this time, the data processing instruction can be a calculation instruction for two or more value-taking arrays.
[0090] In another example, assume that the data processing instruction is a summation instruction for value-taking array 1 and value-taking array 2, and value-taking array 1 and value-taking array 2 have the same index array. Then, according to the positions of the array elements to be summed indicated by the data processing instruction, the corresponding target array elements in value-taking array 1 and value-taking array 2 can be summed to obtain the data processing result.
[0091] In the embodiments of the present application, an array vector is constructed based on multiple initial arrays. The array vector includes: a value-taking array and an index array. The value-taking array includes the array elements of each initial array arranged in sequence. Each array element in the index array corresponds to one of the initial arrays, and the value of each array element is used to indicate the position of the last array element of the corresponding initial array in the value-taking array. A data processing instruction is received, and at least one target array element is determined from at least one value-taking array according to the data processing instruction and the index array, and data processing is performed on each target array element to obtain a data processing result.
[0092] By merging multiple rows of data and storing them in a value-taking array, and storing the index values of each row of data in the index array, the data is aggregated in one array in the memory during calculation. When calculating two arrays, there is no need to perform memory access jumps between multiple arrays as in the prior art. It only needs to determine the position of the data in the value-taking array based on the index array and then operate on the value-taking array, achieving more convenient calculation and improving the calculation performance and data calculation efficiency.
[0093] The following is a further description of constructing the array vector based on the above multiple initial arrays. As Figure 2 shown, step S101 includes:
[0094] S201. Add each array element of the first initial array to the value-taking array. The arrangement order of the array elements in the value-taking array is the same as the arrangement order of the array elements in the initial array, and use the current number of array elements in the value-taking array as the first array element in the index array.
[0095] Taking the initial array in Table 1 as an example, the first initial array "30.13, 30.12, 30.11" can be added to the value-taking array. During this process, the order of each array element in the initial array remains unchanged, and use the current number of value-taking array elements "3" as the first array element in the index array. At this time, the value-taking array can be represented as [30.13, 30.12, 30.11].
[0096] S202. Add the array elements of the next initial array to the value-taking array, and use the current number of array elements in the value-taking array as the next array element in the index array.
[0097] Continuing with the example in Table 1 above, the array elements "31.23, 31.31" of the second initial array are added to the value-taking array, resulting in a new value-taking array [30.13, 30.12, 30.11, 31.23, 31.31], and the current number of array elements "5" is recorded as the second array element of the index array.
[0098] S203. Repeat the step of adding the array elements of the next initial array to the value-taking array and using the current number of array elements in the value-taking array as the next array element in the index array until all the array elements of all the initial arrays are added to the value-taking array, and use the obtained value-taking array and index array as the array vectors.
[0099] The array elements "31.20, 30.55, 30.70" of the third initial array are added to the value-taking array, resulting in a new value-taking array [30.13, 30.12, 30.11, 31.23, 31.31, 31.20, 30.55, 30.70], and the number of current array elements "8" is used as the third array element of the index array. At this time, all the initial arrays have been added to the value-taking array, so the finally obtained [30.13, 30.12, 30.11, 31.23, 31.31, 31.20, 30.55, 30.70] can be used as the value-taking array, and [3, 5, 8] can be used as the index array.
[0100] In the first implementation manner, the data processing instruction is: the first calculation instruction. The first calculation instruction is used to indicate an instruction for calculating all the elements in the value-taking array, including: an instruction for summing all the elements in the value-taking array, and an instruction for adding, subtracting, multiplying, or dividing all the elements in the value-taking array by a number.
[0101] At this time, the process of determining at least one target array element from at least one value-taking array according to the data processing instruction and the index array, and performing data processing on each target array element to obtain a data processing result, as Figure 3 shown, includes:
[0102] S301. Determine all the array elements in the value-taking array as target array elements.
[0103] S302. Calculate each target array element according to the first calculation instruction to obtain a data processing result.
[0104] If the first calculation instruction is an instruction for summing all the array elements, all the array elements can be obtained from the value-taking array, the sum of all the array elements can be calculated, and the calculated sum is used as the data processing result.
[0105] If the first calculation instruction is an instruction to calculate all elements in the value-taking array, all array elements can be obtained from the value-taking array, the calculation method and the parameters used in the calculation can be obtained from the first calculation instruction, and the parameters and each array element are calculated according to the calculation method indicated by the first calculation instruction to obtain the calculation results of each array element, and the calculation results of each array element are output as a new value-taking array in the order of each array element, and the new value-taking array is used as the data processing result.
[0106] In the second implementation manner, the data processing instruction is: a row calculation instruction or a column calculation instruction. At this time, the data processing instruction is used to indicate the calculation of some array elements in the value-taking array, such as calculating a certain row or a certain column in the initial array. The data processing instruction includes position information, which is used to indicate the row and column positions of the array elements that need to be calculated in the initial array.
[0107] The process of determining at least one target array element from at least one value-taking array according to the data processing instruction and the index array, and performing data processing on each target array element to obtain the data processing result, as Figure 4 shown, includes:
[0108] S401. If the data processing instruction is a row calculation instruction, at least one target array element in each row is determined in the value-taking array according to the array elements in the index array, and the target array elements in each row are calculated to obtain the data processing results of each row.
[0109] Optionally, the target array element may be the array element that needs to be calculated indicated by the row calculation instruction in the value-taking array.
[0110] In the first implementation manner, the row calculation instruction may indicate the summation of a certain row in Table 1, that is, a certain initial array. At this time, based on the array elements in the index array, the target array elements of the initial array indicated by the row calculation instruction in the value-taking array are determined, and the sum of the target array elements is used as the data processing result of the row calculation instruction.
[0111] In the second implementation manner, the row calculation instruction may indicate adding, subtracting, multiplying, or dividing the array elements of a certain initial array by a parameter. At this time, based on the row calculation instruction and the index array, the target array elements are determined in the value-taking array, and the target array elements are calculated according to the row calculation instruction to obtain the data processing result.
[0112] S402. If the data processing instruction is a column calculation instruction, at least one target array element in each column is determined in the value-taking array according to the array elements in the index array, and the target array elements in each column are calculated to obtain the data processing results of each column.
[0113] Optionally, the target array element may be an array element in the value array that needs to be calculated as indicated by the column calculation instruction in the value array.
[0114] In the first implementation, the column calculation instruction may indicate to sum a certain column in Table 1. At this time, based on the array elements in the index array, the target array element of the initial array indicated by the column calculation instruction in the value array can be determined in the value array, and the sum of the target array elements is used as the data processing result of the column calculation instruction.
[0115] In the second implementation, the column calculation instruction may indicate to add, subtract, multiply, or divide the data in a certain column by a parameter. At this time, based on the column calculation instruction and the index array, the target array element can be determined in the value array, and the target array element is calculated according to the column calculation instruction to obtain the data processing result.
[0116] The further description of determining at least one target array element of each row in the value array according to the array elements in the index array is as Figure 5 shown. The above step S401 includes:
[0117] S501. Determine the start position offset and end position offset corresponding to each array element according to the array elements in the index array.
[0118] Among them, each initial array stores the data of each row in the table. The end position offset of each row can be determined based on the index array, and the start position offset of each row can be calculated.
[0119] Exemplarily, if the value array is value[30.13, 30.12, 30.11, 31.23, 31.31, 31.20, 30.55, 30.70] and the index array is index[3, 5, 8], then the indexes of the data of each row in the value array are 0, 3, 5 respectively. The start position offsets of the first array element of each row are 0, 3, 5 respectively, and the end offsets are 2, 4, 7 respectively.
[0120] It should be understood that value[0] represents the first array element in the value array. The array elements in the index array are used to represent the position of the last array element in the initial array in the value array. Therefore, the value obtained by subtracting 1 from the array element in the index array can be used as the end position offset of an initial array, 0 can be used as the start position offset of the first initial array, and the values obtained by subtracting 1 from the other array elements except the last array element are used as the start position offsets of the second initial array and the remaining initial arrays in turn, so as to obtain the start position offset and end position offset of each initial array.
[0121] S502. Use the array elements between the start position offset and the end position offset corresponding to the array elements as the target array elements of the current row.
[0122] If the end position offsets of each row are 3, 5, and 8 respectively, then the array elements of the first row are value[0 - 2] in the value array, that is, "30.13, 30.12, 30.11" in the value array, the array elements of the second row are value[3 - 4] in the value array, that is, "31.23, 31.31" in the value array, and the array elements of the third row are value[5 - 7] in the value array, that is, "31.20, 30.55, 30.70" in the value array.
[0123] If each initial array stores the data of each row in the table, the further description of determining at least one target array element of each column in the value array according to the array elements in the index array at this time is as follows Figure 6 As shown, the above step S402 includes:
[0124] S601. Determine the number of columns according to the differences between adjacent array elements in the index array.
[0125] Optionally, the differences can be made between adjacent array elements in the index array, and the largest obtained difference is used as the number of columns.
[0126] S602. Use 0 as the initial offset, use the value obtained by subtracting 1 from the first array element in the index array as the end offset, and determine the first target array element of each column in the value array based on the initial offset and the end offset.
[0127] Exemplarily, assume there is an array vector [[1, 3, 5], [2, 6], [9, 8, 7]], then its corresponding value and index arrays are: value = [1, 3, 5, 2, 6, 9, 8, 7], index = [3, 5, 8].
[0128] In this application, 0 can be used as the initial offset, value[2] can be used as the end offset, and based on the initial offset and the end offset, the first target array element value[0] "1" of the first column, the first target array element value[1] "3" of the second column, and the first target array element value[2] "5" of the third column in the value array are determined.
[0129] S603. Use the first array element in the index array as the initial offset, and the next adjacent array element as the end offset. Start querying backward from the position indicated by the initial offset in the value array. If the offset of the current target array element is greater than or equal to the end offset, set the value of the target array element in the current column to null. If the offset of the current target array element is less than the end offset, use the current target array element as the target array element in the current column.
[0130] Continuing with the above example, the first array element index[0] = "3" in the index array can be used as the new initial offset, and the next adjacent array element index[1] = "5" as the end offset. Start querying from the position value[3] in the value array. Since value[3] does not exceed the end offset value[5], value[3] = "2" can be used as the second target array element in the first column.
[0131] For the next adjacent array element value[4] in the value array, since value[4] does not exceed the end offset value[5], value[4] = "6" can be used as the second target array element in the second column. The next adjacent array element value[5] is equal to the end offset, so the second target array element in the third column is set to null.
[0132] S604. Use the next array element in the index array as the new initial offset for each column, and the next adjacent array element as the end offset, and repeat the above steps S603 - S604 until the current target array element is the last array element in the value array, to obtain at least one target array element for each column.
[0133] Continuing with the above example, the next array element index[1] = "5" in the index array can be used as the new initial offset, and the next adjacent array element index[2] = "8" as the end offset. Start querying from the position value[5] in the value array. Since value[5] does not exceed the end offset value[8], value[5] = "9" can be used as the third target array element in the first column.
[0134] For the next adjacent array element value[6] in the value array, since value[6] does not exceed the end offset value[8], value[6] = "8" can be used as the third target array element in the second column. The next adjacent array element value[7] does not exceed the end offset value[8], so value[7] = "7" can be used as the third target array element in the third column. Thus, the first column is [1, 2, 9], the second column is [3, 6, 8], and the third column is [5, null, 7].
[0135] Optionally, the data processing instruction is: a correlation coefficient calculation instruction, and at least one value array includes: a first value array and a second value array.
[0136] Wherein, the index arrays of the first value array and the second value array are the same, that is, the array elements in the index array are the same, and the order of the array elements is also the same.
[0137] The process of determining at least one target array element from at least one value array according to the data processing instruction and the index array, and performing data processing on each target array element to obtain a data processing result is as Figure 7 shown, including:
[0138] S701. Determine at least one first target array element from the first value array according to the data processing instruction and the index array, and determine the corresponding second target array element of each first target array element from the second value array.
[0139] The data processing instruction can indicate the value array that needs to be calculated, and the position of the data that needs to be calculated in the initial array. Based on the data processing instruction and the index array, the first target array element and the second target array element that need to be calculated can be determined in the first value array and the second value array.
[0140] Wherein, the positions of the first target array element and the second target array element correspond one by one, and they have the same index value.
[0141] Exemplarily, when the first target array element is the data in the nth row of Table 1, the second target array element is the data in the nth row of Table 2, and the index arrays of Table 1 and Table 2 are the same.
[0142] S702. Determine the correlation coefficient of each first target array element and each second target array element to obtain the data processing result of each first target array element and each second target array element.
[0143] Calculate the correlation coefficient for the first target array element and the second target array element corresponding to the first target array element, and the data processing result of the first target array element can be obtained. After completing the calculation of the correlation coefficients for all the first target array elements and the second target array elements, all the obtained data processing results can be used as the data processing result of this data processing instruction.
[0144] Exemplarily, assume there are two vectors: a = [[1, 3, 5], [2, 6], [9, 8, 7]], b = [[1, 2, 3], [4, 5], [6, 7, 8]], then the first value array can be represented as [1, 3, 5, 2, 6, 9, 8, 7], and the second value array can be represented as [1, 2, 3, 4, 5, 6, 7, 8]. If it is necessary to calculate the correlation coefficients of each row in the first value array and the second value array, then based on the above steps S501 - S502, determine the first target array elements of each row in the first value array and the second target array elements of each row in the second value array, that is, [corr([1, 3, 5], [1, 2, 3]), corr([2, 6], [4, 6]), corr([9, 8, 7], [6, 7, 8])] = [1, 1, -1].
[0145] In another example, if it is necessary to calculate the correlation coefficients of each column in the above first value array and the second value array, based on the above steps S601 - S604, determine the first target array elements of each column in the first value array and the second target array elements of each column in the second value array, that is, [corr([1, 2, 9], [1, 4, 6]), corr([3, 6, 8], [2, 5, 7]), corr([5, null, 7], [3, null, 8])] = [0.866, 1, 1].
[0146] Optionally, as Figure 8 shown, the steps for directly summing the value arrays with the same multiple index arrays in this application include:
[0147] S801: Receive the value array summation instruction.
[0148] Optionally, the value array summation instruction is used to indicate the value arrays that need to be summed and indicate the positions of the array elements to be summed in the initial array.
[0149] S802: According to the value array summation instruction, sum the array elements at the corresponding positions in the first value array and the second value array to obtain new array elements at each position, and form an array composed of each new array element as the new value array.
[0150] In the first implementation manner, the array elements at all corresponding positions in the first value array and the second value array can be summed. For example, adding the value arrays value1[1, 3, 5, 2, 6, 9, 8, 7] and value2[2, 3, 4, 1, 2, 1, 2, 3], directly adding the values at the corresponding positions, to obtain a new value array [3, 6, 9, 3, 8, 10, 10, 10].
[0151] In the second implementation manner, partial data in the first value array and the second value array can be summed. For example, summing the corresponding data in a certain row or summing the corresponding data in a certain column, and retaining the remaining data of one of the value arrays. At this time, the steps for determining the target array elements in a certain row or a certain column to be calculated can refer to the above steps S501 - S502 and steps S601 - S604, which will not be elaborated herein in the present application.
[0152] In the embodiments of the present application, multiple columns of data are merged and stored in a value array, such that the data is concentrated in one array in the memory during calculation. When calculating two arrays, there is no need to perform memory access jumps between multiple arrays as in the prior art, and only the value array needs to be operated on, achieving more convenient calculation and improving the calculation performance.
[0153] Figure 9 The figure shows a schematic structural diagram of an electronic device provided by the embodiments of the present application, including: a processor 901, a storage medium 902, and a bus 903. The storage medium 902 stores machine-readable instructions executable by the processor 901. When the electronic device runs a data processing method as in the embodiments, the processor 901 communicates with the storage medium 902 through the bus 903, and the processor 901 executes the machine-readable instructions, the preamble part of the method item of the processor 901, to execute the steps in the above data processing method.
[0154] The embodiments of the present application further provide a computer-readable storage medium. A computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the processor executes the following steps in the above data processing method.
[0155] In the embodiments of the present application, when the computer program is run by a processor, other machine-readable instructions can also be executed to execute other methods as in the embodiments. For the specific method steps and principles executed, refer to the description of the embodiments, which will not be elaborated in detail herein.
[0156] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For another example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some communication interfaces. The indirect coupling or communication connection of devices or units can be in electrical, mechanical or other forms.
[0157] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0158] In addition, each functional unit in the embodiments provided in the present application can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit.
[0159] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in various embodiments of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.
[0160] It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. In addition, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.
[0161] Finally, it should be noted that the above embodiments are only specific implementation manners of the present application, used to illustrate the technical solutions of the present application, rather than limiting it. The protection scope of the present application is not limited thereto. Although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that any person skilled in the art within the technical scope disclosed by the present application can still modify the technical solutions recorded in the foregoing embodiments, or can easily think of changes, or perform equivalent replacements on some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application. All should be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims
1. A data processing method, characterized in that, Including: Construct an array vector based on multiple initial arrays. The array vector includes: a value array and an index array. The value array includes the array elements of each of the initial arrays arranged in sequence. Each array element in the index array corresponds to one of the initial arrays, and the value of each array element is used to indicate the position of the last array element of the corresponding initial array in the value array. The data types stored in each of the initial arrays are the same, and the array elements at the same position in each of the initial arrays correspond to the same list header field or row header field; Receive a data processing instruction; Determine at least one target array element from at least one value array according to the data processing instruction and the index array, and perform data processing on each of the target array elements to obtain a data processing result.
2. The method according to claim 1, characterized in that, The constructing an array vector based on multiple initial arrays includes: Add the array elements of the first initial array to the value array. The arrangement order of the array elements in the value array is the same as the arrangement order of the array elements in the initial array, and use the current number of array elements in the value array as the first array element in the index array; Add the array elements of the next initial array to the value array, and use the current number of array elements in the value array as the next array element in the index array; Repeat the step of adding the array elements of the next initial array to the value array and using the current number of array elements in the value array as the next array element in the index array until the array elements of all the initial arrays are added to the value array, and use the obtained value array and index array as the array vector.
3. The method according to claim 1, wherein The data processing instruction is: a first calculation instruction; The determining at least one target array element from at least one value array according to the data processing instruction and the index array, and performing data processing on each of the target array elements to obtain a data processing result includes: Determine all the array elements in the value array as target array elements; Perform calculations on each of the target array elements according to the first calculation instruction to obtain the data processing result.
4. The method according to claim 1, characterized in that, The data processing instruction is: a row calculation instruction or a column calculation instruction; The determining at least one target array element from at least one value array according to the data processing instruction and the index array, and performing data processing on each of the target array elements to obtain a data processing result includes: If the data processing instruction is a row calculation instruction, determine at least one target array element for each row in the value array according to the array elements in the index array, and perform calculations on the target array elements for each row to obtain the data processing result for each row; If the data processing instruction is a column calculation instruction, determine at least one target array element for each column in the value array according to the array elements in the index array, and perform calculations on the target array elements for each column to obtain the data processing result for each column.
5. The method according to claim 4, wherein Determining at least one target array element for each row in the value array according to the array elements in the index array includes: Determining the start position offset and the end position offset corresponding to each of the array elements according to the array elements in the index array; Taking the array elements between the start position offset and the end position offset corresponding to the array element as the target array elements for the current row.
6. The method according to claim 4, wherein Determining at least one target array element for each column in the value array according to the array elements in the index array includes: A. Determining the number of columns according to the difference between adjacent array elements in the index array; B. Taking 0 as the initial offset, taking the value obtained by subtracting 1 from the first array element in the index array as the end offset, and determining the first target array element for each column in the value array based on the initial offset and the end offset; C. Taking the first array element in the index array as the initial offset, taking the next adjacent array element as the end offset, querying backward from the position indicated by the initial offset in the value array. If the offset of the current target array element is greater than or equal to the end offset, setting the value of the target array element in the current column to null. If the offset of the current target array element is less than the end offset, taking the current target array element as the target array element for the current column; D. Taking the next array element in the index array as the new initial offset for each column, taking the next adjacent array element as the end offset, and repeating steps C - D until the current target array element is the last array element in the value array, to obtain at least one target array element for each column.
7. The method according to claim 1, characterized in that, The data processing instruction is: a correlation coefficient calculation instruction, and the at least one value array includes: a first value array and a second value array; Determining at least one target array element from at least one value array according to the data processing instruction and the index array, and performing data processing on each of the target array elements to obtain a data processing result includes: Determining at least one first target array element from the first value array according to the data processing instruction and the index array, and determining the second target array element corresponding to each of the first target array elements from the second value array; Determining the correlation coefficient between each of the first target array elements and each of the second target array elements to obtain the data processing result of each of the first target array elements and each of the second target array elements.
8. The method according to claim 7, wherein The method further includes: Receiving a value array summation instruction; According to the value array summation instruction, summing the array elements at the corresponding positions in the first value array and the second value array to obtain new array elements at each position, and taking the array composed of each of the new array elements as a new value array.
9. An electronic device, characterized in that, Including: A processor, a storage medium, and a bus, where the storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to perform the steps of a data processing method according to any one of claims 1 to 8.
10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, it performs the steps of a data processing method according to any one of claims 1 to 8.