Control methods and devices for data processing devices

Abstract

This application discloses a control method and apparatus for a data processing device, which can reduce processing time. The method includes: acquiring two target data groups, each a first data group in one of two data sets, wherein there are multiple first data groups in the first data set, and the key value of any data group in the data set is less than the key values ​​of all data groups following that data group; if the maximum key value of the first target data group is less than or equal to the maximum key value of the second target data group, then the next first data group located in the first target data group is designated as the first target data group. This reduces the computational load and improves computational efficiency.

Description

Technical Field

[0001] This application relates to the field of data processing, and more specifically, to a control method and apparatus for a data processing device. Background Technology

[0002] Graph mining algorithms are representative graph processing and data mining algorithms used to find specific subgraph patterns within a complete graph data structure and count the frequency of those patterns. Graph mining algorithms have wide applications, with common use cases including community network analysis in social media, protein analysis in bioinformatics, and drug discovery in computational chemistry.

[0003] A neighbor set of a vertex can be used to represent whether a vertex has a relationship with other vertices in a graph. The neighbor set consists of multiple numbers, each representing the index of a vertex that is related to that vertex. During graph mining, data processing equipment can be used to compare the neighbor sets of two vertices in the graph and determine the indices that are equal. These equal indices are the indices of the vertex in the graph that is related to both vertices.

[0004] The data processing device has limited processing power. Generally, it can compare two column indexes where the number of values ​​does not exceed a preset value. If the number of values ​​in one of the column indexes exceeds the preset value, the values ​​in that column index need to be grouped into multiple arrays. Then, the data processing device can compare each of these arrays with the other column index of the two column indexes.

[0005] Comparing each of these multiple arrays with the other column index of the two column indices involves a large amount of computation and requires a long processing time. Summary of the Invention

[0006] This application provides a control method and apparatus for a data processing device, which can reduce the amount of computation and shorten the data processing time.

[0007] A first aspect provides a control method for a data processing device, the method comprising: acquiring a first target data group and a second target data group, wherein the first target data group is the first data group among a plurality of first data groups in a first data set, and the second target data group is the first data group among at least one second data group in a second data set, wherein each data group in each data set of the first data set and the second data set includes at least one data, each data includes a key value, and the data groups in each data set are arranged in a first order or a second order, wherein in the case of being arranged in the first order, each key value in any data group in each data set is less than each key value in a data group following the first data group, and in the case of being arranged in the second order, each key value in any data group in each data set is greater than each key value in a data group following the second data group; performing multiple iterations, each iteration comprising: setting the first target data group as the first target data group; and performing multiple iterations, wherein each iteration includes: setting the first target data group as the first target data group; ... The data group and the second target data group are input into the data processing device, which is used to determine the key values ​​that are equal in the first target data group and the second target data group; when the data groups in each data set are arranged in the first order and the first key value is less than or equal to the second key value, the first data group in the first data set that is located after the first target data group is obtained as the first target data group, where the first key value is the largest key value in the first target data group and the second key value is the largest key value in the second target data group; when the data groups in each data set are arranged in the second order and the third key value is greater than or equal to the fourth key value, the first data group in the first data set that is located after the first target data group is obtained as the first target data group, where the third key value is the smallest key value in the first target data group and the fourth key value is the smallest key value in the second target data group.

[0008] For two data sets, each data group in the data set is arranged in order of the size of the key value. When using a data processing device to process the data, if there are multiple first data groups in the first data set, through iteration, for the first data group whose key values ​​all exceed the range of key values ​​in the second target data group, it is no longer necessary to input the data processing device, that is, no longer necessary to compare with the second target data group, thereby reducing the number of first data groups compared with the second target data group and thus reducing the amount of computation.

[0009] When the key values ​​in multiple first data sets are arranged from smallest to largest, the range of key values ​​in the second target data set can be less than or equal to the largest key value in the second target data set. In other words, if the maximum value of the key value in the first target data set is greater than or equal to the maximum value of the key value in the second target data set, subsequent first data sets do not need to be compared with the second target data set.

[0010] When the key values ​​in multiple first data sets are arranged from largest to smallest, the range of key values ​​in the second target data set can be greater than or equal to the smallest key value in the second target data set. In other words, if the smallest key value in the first target data set is less than or equal to the largest key value in the second target data set, subsequent first data sets do not need to be compared with the second target data set.

[0011] In conjunction with the first aspect, in some possible implementations, the number of at least one second data group is multiple. If the data groups in each data set are arranged in the first order and the first key value is greater than or equal to the second key value, the second data group in the second data set that is located after the second target data group is obtained as the second target data group. If the data groups in each data set are arranged in the second order and the third key value is less than or equal to the fourth key value, the second data group in the second data set that is located after the second target data group is obtained as the second target data group.

[0012] This ensures that the data set in another data set being compared to a data set in a given data set includes key values ​​whose size is greater than or equal to the minimum key value in the given data set and less than or equal to the maximum key value in the given data set. It also reduces the likelihood that the given data set will be compared to a data set in the other data set that only includes key values ​​outside of this range, thereby improving computational efficiency and reducing computational load.

[0013] In conjunction with the first aspect, in some possible implementations, the data processing apparatus includes a processing matrix comprising v×v processing units, where v is a positive integer, and the number of at least one data in each of the first and second target data groups is less than or equal to v. The i-th first data in the first target data group is input to the j-th processing unit along the second direction among the v processing units located at the first edge in the j-th input cycle of the iteration. The p-th second data in the second target data is input to the q-th processing unit along the first direction among the v processing units located at the second edge in the q-th input cycle of the iteration. The first edge and the second edge are adjacent. The processing units for different data inputs in each target data group are different. The first direction is a direction from the second edge pointing into the processing matrix and perpendicular to the second edge. The second direction is a direction from the first edge pointing into the processing matrix and perpendicular to the first edge. Each processing unit in the processing matrix is ​​used to determine whether the key value in the first data input to the processing unit in the same input cycle is equal to the key value in the second data. When v is greater than 1, each processing unit in the processing matrix is ​​also used to transfer the first data to the next processing unit along the first direction and transfer the second data to the next processing unit along the second direction in the next input cycle after receiving the first data and the second data.

[0014] Using a processing matrix to process the first target data group and the second target data group can improve processing efficiency.

[0015] In conjunction with the first aspect, in some possible implementations, the key values ​​in different data in each data set are different, and each processing unit in the processing matrix is ​​used to transmit the first data to the next processing unit along the first direction and transmit the second data to the next processing unit along the second direction when the key value in the first data is not equal to the key value in the second data.

[0016] When the key values ​​in different data sets are different, if the key value in the first data set is equal to the key value in the second data set, this equal key value will not be equal to the key values ​​in other data sets. Therefore, the processing unit can transmit the first data to the next processing unit along the first direction and the second data to the next processing unit along the second direction only if the key value in the first data set is not equal to the key value in the second data set. This reduces data transmission and computational load.

[0017] In conjunction with the first aspect, in some possible implementations, the data processing apparatus further includes a filtering matrix comprising v filtering units, each of which is located after the last processing unit in each of v rows of the processing matrix along the first direction along the second direction. Each processing unit in the processing matrix is ​​further configured to: when the key value in the first data is equal to the key value in the second data, in the next input cycle after receiving the first data and the second data, transmit the processing result of the processing unit along the second direction to the next unit, the unit being either the processing unit or the filtering unit, the processing result including the equal key values; or, in the next input cycle after receiving the processing result, transmit the processing result along the second direction to the next unit. The method further includes: when the first key value is greater than or equal to the second key value, controlling the v filtering units along the first direction to sequentially output the processing result corresponding to the second target data group according to the input cycle.

[0018] The processing results corresponding to each key value in a second data group can be output uniformly after the second data group is last input into the data processing device, thereby improving the flexibility of the processing result output.

[0019] In conjunction with the first aspect, in some possible implementations, the data processing apparatus further includes a compression triangular matrix comprising v rows of compression units along the first direction, the number of compression units increasing row by row along the first direction, each of the plurality of compression units being configured to: receive the processing result output by the filtering unit preceding the compression unit along the second direction, or receive the processing result output by the compression unit of the next row along the first direction; and in the next input cycle after receiving the processing result, transmit the processing result to the compression unit of the next row along the first direction.

[0020] By setting a compression triangular matrix in the data processing device, the processing results corresponding to a certain second data set can be output within the same input cycle, thereby improving the flexibility of the processing result output.

[0021] In conjunction with the first aspect, in some possible implementations, different key values ​​correspond to different point sets in the relationship graph. The first data is used to indicate whether there is a relationship between a first target point in the relationship graph and at least one point in the point set corresponding to the key value in the first data. The second data is used to indicate whether there is a relationship between a second target point in the relationship graph and at least one point in the point set corresponding to the key value in the second data. Each processing matrix is ​​further used to output a processing result when the key value in the first data is equal to the key value in the second data. The processing result is used to indicate the query point in the relationship graph, and the relationship between the query point and the two target points conforms to a preset condition.

[0022] It can determine the query points in the relationship graph that are connected to two target points under preset conditions, thereby identifying subgraphs in the relationship graph that have a certain subgraph model structure, and realizing graph mining.

[0023] In conjunction with the first aspect, in some possible implementations, the first data further includes a first relation value group of the first target point corresponding to the key value, and the second data further includes a second relation value group of the second target point corresponding to the key value. Each processing unit in the processing matrix is ​​further configured to perform a preset operation on each bit of the first relation value group and the second relation value group when the key value in the first data is equal to the key value in the second data. The same bit in the first relation value group and the second relation value group corresponding to the equal key value corresponds to the same point in the point set corresponding to the equal key value. The result of the preset operation on each bit is used to indicate whether the relationship between the point corresponding to the bit and the two target points conforms to the preset condition.

[0024] The key values ​​in the first data are equal to the key values ​​in the second data, meaning that the key values ​​in the first data and the key values ​​in the second data indicate the same set of points. Therefore, the query point can be determined based on the result of the bitwise preset operation on the first relation value group and the second relation value group.

[0025] By utilizing key-value pairs of points in the relationship graph, and identifying equal key values ​​in the key-value pairs of two target points, a query point is determined based on the relationship value group corresponding to the equal key values ​​of the two points, indicating a relationship that matches a preset condition. This approach reduces both the amount of data used to represent the relationship graph and the processing time required to determine query points.

[0026] In conjunction with the first aspect, in some possible implementations, when the data groups in each data set are arranged in the first order, the key values ​​in each first data group are arranged from smallest to largest; when the data groups in each data set are arranged in the second order, the key values ​​in each first data group are arranged from largest to smallest.

[0027] The key values ​​in the first dataset are arranged in ascending or descending order, allowing at least one data point to be selected as the first target data group. Multiple selections of the first target data group will all result in the same first data group being selected. This makes the division of the first data group more flexible.

[0028] Secondly, a control device for a data processing apparatus is provided, comprising an acquisition module and a processing module. The acquisition module is configured to acquire a first target data group and a second target data group, wherein the first target data group is the first data group among a plurality of first data groups in a first data set, and the second target data group is the first data group among at least one second data group in a second data set. Each data group in each of the first and second data sets includes at least one piece of data. The data groups in each data set are arranged in a first order or a second order. When arranged in the first order, each key value in any data group in each data set is less than each key value in a data group following that data group. When arranged in the second order, each key value in any data group in each data set is greater than each key value in a data group following that data group. The processing module is configured to perform multiple iterations. Each iteration includes inputting the first target data group and the second target data group into the data processing apparatus, and the data processing apparatus is configured to determine the equal key values ​​in the first target data group and the second target data group. Each iteration further includes, if the data groups in each data set are arranged in the first order and the first key value is less than or equal to the second key value, obtaining the first data group in the first data set that follows the first target data group as the first target data group, where the first key value is the largest key value in the first target data group and the second key value is the largest key value in the second target data group. Each iteration also includes, if the data groups in each data set are arranged in the second order and the third key value is greater than or equal to the fourth key value, obtaining the first data group in the first data set that follows the first target data group as the first target data group, where the third key value is the smallest key value in the first target data group and the fourth key value is the smallest key value in the second target data group.

[0029] In conjunction with the second aspect, in some possible implementations, the number of at least one second data group is multiple. If the data groups in each data set are arranged in the first order, and the first key value is greater than or equal to the second key value, then the second data group in the second data set that follows the second target data group is selected as the second target data group. If the data groups in each data set are arranged in the second order, and the third key value is less than or equal to the fourth key value, then the second data group in the second data set that follows the second target data group is selected as the second target data group.

[0030] In conjunction with the second aspect, in some possible implementations, the data processing device includes a processing matrix comprising v × v processing units, where v is a positive integer, and the number of at least one data point in each of the first and second target data groups is less than or equal to v. The i-th first data point in the first target data group is input to the j-th processing unit along a second direction among the v processing units located at the first edge in the j-th input cycle of the iteration. The p-th second data point in the second target data group is input to the q-th processing unit along a first direction among the v processing units located at the second edge in the q-th input cycle of the iteration. The first edge and the second edge are adjacent. The processing units input for different data points in each target data group are different. The first direction is a direction pointing from the second edge into the processing matrix and perpendicular to the second edge, and the second direction is a direction pointing from the first edge into the processing matrix and perpendicular to the first edge. Each processing unit in the processing matrix is ​​used to determine whether the key value in the first data input to the processing unit in the same input cycle is equal to the key value in the second data. When v is greater than 1, each processing unit in the processing matrix is ​​further configured to, in the next input cycle after receiving the first data and the second data, transmit the first data to the next processing unit along the first direction and transmit the second data to the next processing unit along the second direction.

[0031] In conjunction with the second aspect, in some possible implementations, the key values ​​in different data in each data set are different, and each processing unit in the processing matrix is ​​used to transmit the first data to the next processing unit along the first direction and transmit the second data to the next processing unit along the second direction when the key value in the first data is not equal to the key value in the second data.

[0032] In conjunction with the second aspect, in some possible implementations, the data processing device further includes a filtering matrix comprising v filtering units, each of which is located after the last processing unit in each of v rows of the processing matrix along the first direction along the second direction. Each processing unit in the processing matrix is ​​further configured to, in the next input cycle after receiving the first and second data, transmit its processing result along the second direction to the next unit (either the processing unit or the filtering unit) when the key value in the first data is equal to the key value in the second data; or, in the next input cycle after receiving the processing result, transmit the processing result along the second direction to the next unit. The processing module is further configured to, in the case where the first key value is greater than or equal to the second key value, control the v filtering units along the first direction to sequentially output the processing result corresponding to the second target data group according to the input cycle.

[0033] In conjunction with the second aspect, in some possible implementations, the data processing apparatus further includes a compression triangular matrix comprising v rows of compression units along the first direction, the number of which increases row by row along the first direction. Each of the plurality of compression units is configured to receive the processing result output by the filtering unit preceding the compression unit along the second direction, or to receive the processing result output by the compression unit of the next row along the first direction. Each of the plurality of compression units is further configured to, in the next input cycle after receiving the processing result, transmit the processing result to the compression unit of the next row along the first direction.

[0034] In conjunction with the second aspect, in some possible implementations, different key values ​​correspond to different sets of points in the relationship graph. The first data is used to indicate whether there is a relationship between a first target point in the relationship graph and at least one point in the set of points corresponding to the key value in the first data. The second data is used to indicate whether there is a relationship between a second target point in the relationship graph and at least one point in the set of points corresponding to the key value in the second data. Each processing matrix is ​​further used to output a processing result when the key value in the first data is equal to the key value in the second data. The processing result is used to indicate the query point in the relationship graph, and the relationship between the query point and the two target points conforms to a preset condition.

[0035] In conjunction with the second aspect, in some possible implementations, the first data further includes a first relational value group of the first target point corresponding to the key value, and the second data further includes a second relational value group of the second target point corresponding to the key value. Each processing unit in the processing matrix is ​​further configured to, when the key value in the first data is equal to the key value in the second data, perform a preset operation on each bit of the first relational value group and the second relational value group respectively. The same bit in the first and second relational value groups corresponding to the equal key values ​​corresponds to the same point in the point set corresponding to the equal key values. The result of the preset operation on each bit is used to indicate whether the relationship between the point corresponding to the bit and the two target points conforms to the preset condition.

[0036] In conjunction with the second aspect, in some possible implementations, when the data groups in each data set are arranged in the first order, the key values ​​in each first data group are arranged from smallest to largest; when the data groups in each data set are arranged in the second order, the key values ​​in each first data group are arranged from largest to smallest.

[0037] Thirdly, a control device for a data processing apparatus is provided, comprising a memory and at least one processor, the memory for storing a program, wherein when the program is executed in the at least one processor, the control device is used to perform the method described in the first aspect.

[0038] Fourthly, a data processing method is provided, comprising: obtaining at least one key-value pair of a first target point and at least one key-value pair of a second target point in a relation graph, wherein each key-value pair of the target points includes a key value of the target point and a relation value group of the target point corresponding to the key value, different key values ​​correspond to different point sets in the relation graph, and the relation value group of the target point corresponding to the key value of the target point is used to indicate whether the target point has a relationship with each point in the point set corresponding to the key value, and there exists a point in the point set corresponding to the key value of the target point that has a relationship with the target point in each key-value pair of the target point; determining at least one key value of the first target point and at least one key value of the second target point that are equal to each other; and determining a query point in the relation graph based on the relation value group of the first target point and the relation value group of the second target point corresponding to the equal key value, wherein the relationship between the query point and the two target points conforms to a preset condition.

[0039] In each key-value pair of the target point, there exists a point in the set of points corresponding to the key value that is related to the target point. That is, for each target point, there exists a point in the set of points corresponding to the key value in each key-value pair that is related to the target point.

[0040] By utilizing key-value pairs of points in the relationship graph, and identifying equal key values ​​in the key-value pairs of two target points, a query point is determined based on the relationship value group corresponding to the equal key values ​​of the two points, indicating a relationship that matches a preset condition. This approach reduces both the amount of data used to represent the relationship graph and the processing time required to determine query points.

[0041] In conjunction with the fourth aspect, in some possible implementations, obtaining at least one key-value pair for each target point among two target points in the relationship graph includes: obtaining the relationship graph data, which includes a row offset vector, a key value vector, and a relationship value vector; determining at least one key-value pair for each target point among the two target points based on the relationship graph data, wherein the key value vector includes at least one key value for each point among multiple points in the relationship graph, the row offset vector is used to indicate the position of at least one key value for each point in the key value vector, and the relationship value vector includes the relationship value group of the points corresponding to the key values ​​of each point among the multiple points, wherein the order of at least one key value for each point in the key value vector is the same as the order of the relationship value group of the points corresponding to the key values ​​of each point in the relationship value vector.

[0042] The key-value pairs of each point in a relationship diagram can be determined based on the relationship diagram data. Storing the relationship diagram in the format of relationship diagram data can reduce storage space.

[0043] In conjunction with the fourth aspect, in some possible implementations, the number of points in the point set corresponding to different key values ​​is equal.

[0044] The number of points in the set corresponding to different key values ​​is equal, which makes it easier to determine the key-value pairs of target points based on the relationship graph data.

[0045] In conjunction with the fourth aspect, in some possible implementations, determining the query point in the relationship graph based on the relationship value group of the first target point corresponding to the equal key value and the relationship value group of the second target point includes: performing a preset operation on each bit of the relationship value group of the first target point corresponding to the equal key value and the relationship value group of the second target point respectively; the same bit in the relationship value group of different points corresponding to each key value corresponds to the same point in the set of points corresponding to the key value; and the result of the preset operation on each bit is used to indicate whether the relationship between the point corresponding to the bit and the two target points conforms to the preset condition.

[0046] For two target points with equal key values, pre-defined operations are performed on each bit in the relation value group of each target point. These operations can be performed in parallel, which can shorten the computation time.

[0047] In conjunction with the fourth aspect, in some possible implementations, determining the equal key value between at least one key value of the first target point and at least one key value of the second target point includes: sequentially inputting at least one key-value pair of the first target point along a second direction into a plurality of processing units located at the first edge of the processing matrix according to an input cycle; sequentially inputting at least one key-value pair of the second target point along a first direction into a plurality of processing units located at the second edge of the processing matrix according to the input cycle, to determine the equal key value; the processing matrix includes v×v processing units, where v is a positive integer greater than 1; at least one key-value pair of the first target point and at least one key-value pair of the second target point are input into the processing matrix at the beginning of the same input cycle; the first edge and the second edge are adjacent; the first direction is a direction away from the second edge; the second direction is a direction away from the first edge; each processing unit is used to determine the equal key value between the key value of the first target point input into the processing unit and the key value of the second target point. Whether the key values ​​of the points are equal; determining the query point in the relationship graph based on the relationship value group of the first target point and the relationship value group of the second target point corresponding to the equal key values ​​includes: processing the relationship value group of the first target point and the relationship value group of the second target point corresponding to the equal key values ​​using the processing matrix to determine the query point; each of the plurality of processing units is used to determine the query point in the relationship graph based on the relationship value group of the first target point and the relationship value group of the second target point input to the processing unit when the key value of the first target point is equal to the key value of the second target point, and the relationship between the query point and the two target points conforms to a preset condition; each of the plurality of processing units is further used to transmit the key-value pair of the first target point to the next processing unit along the first direction and the key-value pair of the second target point to the next processing unit along the second direction according to the input cycle.

[0048] The processing matrix can be understood as a systolic array, where key-value pairs of each target point flow rhythmically between processing units in a "pipeline" manner. All processing units process the flowing data in parallel, which can improve processing speed and reduce processing time.

[0049] In conjunction with the fourth aspect, in some possible implementations, each of the plurality of processing units is specifically configured to: transmit the key-value pair of the first target point to the next processing unit along the first direction, and transmit the key-value pair of the second target point to the next processing unit along the second direction, when the key value of the first target point is not equal to the key value of the second target point.

[0050] The key values ​​in each key-value pair of a target point are different. If the key value of the first target point is not equal to the key value of the second target point, the processing unit transmits the key values ​​of both the first and second target points. However, if the key value of the first target point is equal to the key value of the second target point, the key-value pair is not transmitted. Therefore, the equal key value does not need to be compared with other key values, reducing the computational load.

[0051] In conjunction with the fourth aspect, in some possible implementations, the number of at least one key-value pair of the first target point is greater than v. The method further includes: dividing at least one key-value pair of the first target point into multiple first key-value pair groups according to the order of the key values ​​in the key-value pairs, and dividing at least one key-value pair of the second target point into at least one second key-value pair group according to the order of the key values ​​in the key-value pairs, wherein the number of key-value pairs in each of the multiple first key-value pair groups and the at least one second key-value pair group is less than or equal to v, wherein the key-value pair group with the smallest key value in the multiple first key-value pair groups is the third key-value pair group, and the key-value pair group with the smallest key value in the multiple second key-value pair groups is the fourth key-value pair group; performing multiple iterations until the maximum key value in the third key-value pair group is greater than the maximum key value in the fourth key-value pair group, wherein the iteration includes: sequentially inputting at least one key-value pair in the third key-value pair group into multiple processing units located at the first edge of the multiple processing units according to the input cycle, and inputting at least one key-value pair in the fourth key-value pair group into multiple processing units located at the first edge of the multiple processing units according to the input cycle. The input is sequentially applied to the processing units located at the second edge of the plurality of processing units according to the input cycle; if the largest key value in the third key value pair group is less than the largest key value in the fourth key value pair group, the next first key value pair group of the third key value pair group in the plurality of first key value pair groups arranged in ascending order of key value is taken as the third key value pair group; the step of sequentially applying at least one key value pair of the first target point to the processing units located at the first edge of the processing matrix along the second direction according to the input cycle, and sequentially applying at least one key value pair of the second target point to the processing units located at the second edge of the processing matrix along the first direction according to the input cycle, includes: before performing the multiple iterations and after each iteration, sequentially applying the key value pairs of the third key value pair group to the processing units located at the first edge of the processing matrix according to the input cycle, and sequentially applying the key value pairs of the fourth key value pair group to the processing units located at the second edge of the processing matrix according to the input cycle.

[0052] Following the ascending order of key values, multiple first key-value pairs are compared sequentially with second key-value pairs. If the largest key value in a first key-value pair is less than the largest key value in a second key-value pair, the next first key-value pair is input into the processing matrix and compared with the second key-value pair. If the largest key value in a first key-value pair is greater than or equal to the largest key value in a second key-value pair, comparison with that second key-value pair is unnecessary, reducing computational load.

[0053] In particular, when there are multiple first key-value pairs and multiple second key-value pairs, the method provided in this application can significantly reduce the amount of computation and the computation time compared with the method of comparing each first key-value pair with each second key-value pair.

[0054] Fifthly, a data processing apparatus is provided, comprising: an acquisition module and a processing module; the acquisition module is configured to acquire at least one key-value pair of a first target point and at least one key-value pair of a second target point in a relation graph, wherein each key-value pair of the target points includes a key value of the target point and a relation value set of the target point corresponding to the key value, different key values ​​correspond to different point sets in the relation graph, and the relation value set of the target point corresponding to the key value of the target point is used to indicate whether the target point has a relationship with each point in the point set corresponding to the key value, and in each key-value pair of the target point, there exists a point in the point set corresponding to the key value that has a relationship with the target point; the processing module is configured to determine at least one key value of the first target point and at least one key value of the second target point that are equal key values; the processing module is further configured to determine a query point in the relation graph based on the relation value set of the first target point and the relation value set of the second target point corresponding to the equal key values, wherein the relationship between the query point and the two target points conforms to a preset condition.

[0055] In conjunction with the fifth aspect, in some possible implementations, the acquisition module is specifically used to: acquire the relationship graph data, the relationship graph data including a row offset vector, a key value vector, and a relationship value vector; based on the relationship graph data, determine at least one key-value pair for each of the two target points, the key value vector including at least one key value for each of the multiple points in the relationship graph, the row offset vector indicating the position of at least one key value for each point in the key value vector, the relationship value vector including the relationship value group of the points corresponding to the key value of each point in the multiple points, and the order of at least one key value for each point in the key value vector being the same as the order of the relationship value group of the points corresponding to the key value of each point in the relationship value vector.

[0056] In conjunction with the fifth aspect, in some possible implementations, the number of points in the point set corresponding to different key values ​​is equal.

[0057] In conjunction with the fifth aspect, in some possible implementations, the processing module is specifically used to perform preset operations on each bit of the relation value group of the first target point corresponding to the equal key value and the relation value group of the second target point. The same bit in the relation value group of different points corresponding to each key value corresponds to the same point in the set of points corresponding to the key value. The result of the preset operation of each bit is used to indicate whether the relationship between the point corresponding to the bit and the two target points conforms to the preset situation.

[0058] In conjunction with the fifth aspect, in some possible implementations, the processing module includes: a controller and a processing matrix, the processing matrix including v×v processing units, where v is a positive integer greater than 1; the controller is used to sequentially input at least one key-value pair of the first target point of the two target points in the relationship graph into a plurality of processing units located at the first edge of the processing matrix along a second direction according to the input cycle, and to sequentially input at least one key-value pair of the second target point of the two target points into a plurality of processing units located at the second edge of the processing matrix along the first direction according to the input cycle, wherein at least one key-value pair of the first target point and at least one key-value pair of the second target point are input into the processing matrix at the beginning of the same input cycle, the first edge and the second edge are adjacent, the first direction is a direction away from the second edge, the second direction is a direction away from the first edge, each key-value pair of the target point includes a key value of the target point and a relation value group of the target point corresponding to the key value, different key values ​​correspond to different relation values ​​of the target point. In the relationship graph, different sets of points are defined. The relationship value set of the target point corresponding to the key value of the target point is used to indicate whether the target point has a relationship with each point in the set of points corresponding to the key value. In each key-value pair of the target point, there exists a point in the set of points corresponding to the key value that has a relationship with the target point. Each of the plurality of processing units is used to: determine whether the key value of the first target point input to the processing unit is equal to the key value of the second target point; if the key value of the first target point is equal to the key value of the second target point, determine a query point in the relationship graph according to the relationship value set of the first target point and the relationship value set of the second target point input to the processing unit, wherein the relationship between the query point and the two target points conforms to a preset condition; and according to the input cycle, transmit the key-value pair of the first target point to the next processing unit along the first direction, and transmit the key-value pair of the second target point to the next processing unit along the second direction.

[0059] In conjunction with the fifth aspect, in some possible implementations, each of the plurality of processing units is specifically configured to: transmit the key-value pair of the first target point to the next processing unit along the first direction, and transmit the key-value pair of the second target point to the next processing unit along the second direction, when the key value of the first target point is not equal to the key value of the second target point.

[0060] In conjunction with the fifth aspect, in some possible implementations, where the number of at least one key-value pair at the first target point is greater than v, the controller is further configured to:

[0061] At least one key-value pair of the first target point is divided into multiple first key-value pair groups according to the order of the key values ​​in the key-value pairs, and at least one key-value pair of the second target point is divided into at least one second key-value pair group according to the order of the key values ​​in the key-value pairs. The number of key-value pairs in each of the multiple first key-value pair groups and the at least one second key-value pair group is less than or equal to v. The key-value pair group with the smallest key value in the multiple first key-value pair groups is the third key-value pair group, and the key-value pair group with the smallest key value in the multiple second key-value pair groups is the fourth key-value pair group. Multiple iterations are performed until the largest key value in the third key-value pair group is greater than the largest key value in the fourth key-value pair group. The iteration includes: sequentially inputting at least one key-value pair in the third key-value pair group into multiple processing units located at the first edge according to the input cycle. The processing unit sequentially inputs at least one key-value pair from the fourth key-value pair group into the plurality of processing units located at the second edge of the plurality of processing units according to the input cycle; if the largest key value in the third key-value pair group is less than the largest key value in the fourth key-value pair group, the next first key-value pair group of the third key-value pair group in the plurality of first key-value pair groups arranged in ascending order of key value is taken as the third key-value pair group; the controller is specifically configured to, before performing the multiple iterations and after each iteration, sequentially input multiple key-value pairs from the third key-value pair group into the plurality of processing units located at the first edge of the processing matrix according to the input cycle, and sequentially input multiple key-value pairs from the fourth key-value pair group into the plurality of processing units located at the second edge of the processing matrix according to the input cycle.

[0062] A sixth aspect provides a data processing apparatus, comprising: a controller and a processing matrix, the processing matrix comprising v×v processing units, where v is a positive integer greater than 1; the controller is configured to sequentially input at least one key-value pair of a first target point of two target points in a relational graph into a plurality of processing units located at a first edge of the processing matrix along a second direction according to an input cycle, and to sequentially input at least one key-value pair of a second target point of the two target points into a plurality of processing units located at a second edge of the processing matrix along a first direction according to the input cycle, wherein at least one key-value pair of the first target point and at least one key-value pair of the second target point are input into the processing matrix at the beginning of the same input cycle, the first edge and the second edge are adjacent, the first direction is a direction away from the second edge, the second direction is a direction away from the first edge, each key-value pair of the target point includes a key value of the target point and a relational value group of the target point corresponding to the key value, different key values ​​corresponding to different relational values ​​in the relational graph. A set of points, wherein the key value of the target point corresponds to the set of relational values ​​of the target point, which is used to indicate whether the target point has a relationship with each point in the set of points corresponding to the key value, and in each key-value pair of the target point, there exists a point in the set of points corresponding to the key value that is related to the target point; each of the plurality of processing units is used to: determine whether the key value of the first target point input to the processing unit is equal to the key value of the second target point; if the key value of the first target point is equal to the key value of the second target point, determine a query point in the relation graph according to the set of relational values ​​of the first target point and the set of relational values ​​of the second target point input to the processing unit, wherein the relationship between the query point and the two target points conforms to a preset condition; according to the input cycle, transmit the key value of the first target point to the next processing unit along the first direction, and transmit the key value of the second target point to the next processing unit along the second direction, wherein the first direction is perpendicular to the first edge, and the second direction is perpendicular to the second edge.

[0063] In conjunction with the sixth aspect, in some possible implementations, each of the plurality of processing units is specifically configured to: transmit the key-value pair of the first target point to the next processing unit along the first direction, and transmit the key-value pair of the second target point to the next processing unit along the second direction, when the key value of the first target point is not equal to the key value of the second target point.

[0064] The key values ​​in each key-value pair of a target point are different. If the key value of the first target point is not equal to the key value of the second target point, the processing unit transmits the key values ​​of both the first and second target points. However, if the key value of the first target point is equal to the key value of the second target point, the key-value pair is not transmitted. Therefore, the equal key value does not need to be compared with other key values, reducing the computational load.

[0065] In conjunction with the sixth aspect, in some possible implementations, the controller is further configured to determine at least one key-value pair of the first target point and at least one key-value pair of the second target point based on the relationship graph data. The relationship graph data includes a row offset vector, a key value vector, and a relationship value vector. The key value vector includes at least one key value of each point among a plurality of points in the relationship graph. The row offset vector is used to indicate the position of at least one key value of each point in the key value vector. The relationship value vector includes a relationship value group of points corresponding to the key value of each point among the plurality of points. The order of at least one key value of each point in the key value vector is the same as the order of the relationship value group of points corresponding to the key value of each point in the relationship value vector.

[0066] The key-value pairs of each point in a relationship diagram can be determined based on the relationship diagram data. Storing the relationship diagram in the format of relationship diagram data can reduce storage space.

[0067] For example, the controller is further configured to determine at least one key-value pair of the first target point and at least one key-value pair of the second target point based on the stored relational graph data.

[0068] In conjunction with the sixth aspect, in some possible implementations, the number of points in the point set corresponding to different key values ​​is equal.

[0069] The number of points in the set corresponding to different key values ​​is equal, which makes it easier to determine the key-value pairs of target points based on the relationship graph data.

[0070] In conjunction with the sixth aspect, in some possible implementations, each of the plurality of processing units is configured to: when the key value of the first target point is equal to the key value of the second target point, perform a preset operation on each bit of the relation value group of the first target point and the relation value group of the second target point corresponding to the equal key value, wherein the same bit in the relation value group of different points corresponding to each key value corresponds to the same point in the point set corresponding to the key value, and the result of the preset operation of each bit is used to indicate whether the relationship between the point corresponding to the bit and the two target points conforms to the preset condition.

[0071] For two target points with equal key values, pre-defined operations are performed on each bit in the relation value group of each target point. These operations can be performed in parallel, which can shorten the computation time.

[0072] In conjunction with the sixth aspect, in some possible implementations, the number of at least one key-value pair of the first target point is greater than v. Specifically, the controller is configured to: divide at least one key-value pair of the first target point into multiple first key-value pair groups according to the order of the key values ​​in the key-value pairs, and divide at least one key-value pair of the second target point into at least one second key-value pair group. The number of key-value pairs in each of the multiple first key-value pair groups and the at least one second key-value pair group is less than or equal to v. The key-value pair group with the smallest key value among the multiple first key-value pair groups is the third key-value pair group, and the key-value pair group with the smallest key value among the multiple second key-value pair groups is the fourth key-value pair group. Iteration is performed until the maximum key value in the third key-value pair group is greater than the maximum key value in the fourth key-value pair group. The iteration includes: sequentially inputting multiple key-value pairs in the third key-value pair group according to the input cycle. The controller is configured to, before and after the multiple iterations, sequentially input multiple key-value pairs from the fourth key-value pair group into the multiple processing units located at the first edge of the processing matrix, and sequentially input multiple key-value pairs from the fourth key-value pair group into the multiple processing units located at the second edge of the processing matrix, according to the input cycle. If the largest key value in the third key-value pair group is less than the largest key value in the fourth key-value pair group, the next first key-value pair group in the multiple first key-value pair groups arranged in ascending order of key values ​​is selected as the third key-value pair group. Specifically, the controller is configured to, before and after each iteration, sequentially input multiple key-value pairs from the third key-value pair group into the multiple processing units located at the first edge of the processing matrix, and sequentially input multiple key-value pairs from the fourth key-value pair group into the multiple processing units located at the second edge of the processing matrix, according to the input cycle.

[0073] If the number of at least one key-value pair at the first target point is greater than v, the at least one key-value pair at the first target point is divided into multiple first key-value pair groups in ascending order of key value, and compared with at least one second key-value pair group at the first target point respectively.

[0074] Following the ascending order of key values, multiple first key-value pairs are compared sequentially with second key-value pairs. If the largest key value in a first key-value pair is less than the largest key value in a second key-value pair, the next first key-value pair is input into the processing matrix and compared with the second key-value pair. If the largest key value in a first key-value pair is greater than or equal to the largest key value in a second key-value pair, comparison with that second key-value pair is unnecessary, reducing computational load.

[0075] In particular, when there are multiple first key-value pairs and multiple second key-value pairs, the method provided in this application can significantly reduce the amount of computation and the computation time compared with comparing each first key-value pair with each second key-value pair.

[0076] In a seventh aspect, a data processing apparatus is provided, including a memory and at least one processor, the memory being used to store a program, wherein when the program is executed in the at least one processor, the processor is used to execute a method in any implementation of the first aspect.

[0077] Eighthly, a computer-readable medium is provided that stores program code for execution by a device, the program code including methods for performing any implementation of the first or fourth aspect.

[0078] Ninth aspect, a computer program product containing instructions is provided, which, when run on a computer, causes the computer to perform the method in any one of the implementations of the first or fourth aspect described above.

[0079] In a tenth aspect, a chip is provided, the chip including a processor and a data interface, wherein the processor reads instructions stored in a memory through the data interface and executes the method in any one of the implementations of the first aspect and the fourth aspect described above.

[0080] Eleventhly, a data processing system is provided, including a control device and a data sorting device of the data processing apparatus described in the first aspect.

[0081] Optionally, as one implementation, the chip may further include a memory storing instructions, and the processor is used to execute the instructions stored in the memory. When the instructions are executed, the processor is used to execute the method in either the first aspect or the fourth aspect.

[0082] The aforementioned chip can be a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC).

[0083] It should be understood that, in this application, the method of the first aspect can specifically refer to the first aspect and the method in any of the various implementations of the first aspect. Attached Figure Description

[0084] Figure 1 This is a schematic structural diagram of a graph mining algorithm.

[0085] Figure 2 This is a diagram illustrating a data format.

[0086] Figure 3 This is a schematic diagram of a queue-based set merging.

[0087] Figure 4 This is a schematic structural diagram of a data processing method provided in an embodiment of this application.

[0088] Figure 5 This is a schematic flowchart illustrating a method for generating relational graph data provided in an embodiment of this application.

[0089] Figure 6 This is a schematic flowchart of another data processing method provided in the embodiments of this application.

[0090] Figure 7 This is a schematic structural diagram of a data processing device provided in an embodiment of this application.

[0091] Figure 8 This is a schematic flowchart illustrating another data processing method provided in the embodiments of this application.

[0092] Figure 9 This is a schematic structural diagram of a processing unit provided in an embodiment of this application.

[0093] Figure 10 This is a schematic diagram of a key-value pair set provided in an embodiment of this application.

[0094] Figure 11 This is a schematic structural diagram of a filtering unit provided in an embodiment of this application.

[0095] Figure 12 This is a schematic structural diagram of a compression triangle provided in an embodiment of this application.

[0096] Figure 13This is a schematic structural diagram of a compression unit provided in an embodiment of this application.

[0097] Figure 14 This is a schematic diagram of the processing time of the data processing device provided in the embodiments of this application.

[0098] Figure 15 This is a schematic structural diagram of a data processing device provided in an embodiment of this application.

[0099] Figure 16 This is a schematic structural diagram of a data processing system provided in an embodiment of this application.

[0100] Figure 17 This is a schematic structural diagram of another data processing system provided in the embodiments of this application.

[0101] Figure 18 This is a schematic diagram comparing the performance of the data processing systems provided in the embodiments of this application.

[0102] Figure 19 This is a schematic flowchart of a control method for a data processing device provided in an embodiment of this application.

[0103] Figure 20 This is a schematic structural diagram of a control device for a data processing apparatus provided in an embodiment of this application.

[0104] Figure 21 This is a schematic structural diagram of a control device for a data processing apparatus provided in an embodiment of this application.

[0105] Figure 22 This is a schematic structural diagram of a data processing system provided in an embodiment of this application. Detailed Implementation

[0106] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0107] Graph mining algorithms are representative graph processing and data mining algorithms used to find specific subgraph patterns within a complete graph data structure and count the frequency of those patterns. Graph mining algorithms have wide applications, with common use cases including community network analysis in social media, protein analysis in bioinformatics, and drug discovery in computational chemistry.

[0108] Figure 1 This is a schematic structural diagram of a graph mining algorithm.

[0109] Each point on relation graph 110 (i.e., point 1 to point N) is sequentially used as point v0 in the subgraph model.

[0110] A graph, also known as a graph data structure, is used to represent whether relationships exist between nodes. The existence of a relationship between two nodes can be understood as a relationship between those two nodes.

[0111] A relationship graph consists of interconnected points and edges. Points in a relationship graph, also called nodes, can be used to represent entities. An edge between two points in a relationship graph indicates that a relationship exists between them.

[0112] An entity refers to something that is distinguishable and exists independently. Examples include a person, a city, a plant, a commodity, a piece of equipment, and an atom.

[0113] If two points are related, they are connected by an edge. If two points are not related, there is no edge between them. Two points connected by an edge are neighbors.

[0114] A relationship diagram is a network of connections that links different types of information together. Relationship diagrams provide the ability to analyze problems from the perspective of "relationships".

[0115] In the subgraph model, point v1 is a neighbor of point v0. Given that a point u0 on relation graph 110 is identified as point v0, each of u0's neighbors is sequentially designated as point v1.

[0116] In the subgraph model, point v2 is a common neighbor of points v0 and v1. Given that a point u1 on relation graph 110 is identified as point v1, the points among the common neighbors of points u0 and u1 are successively designated as points v2. The common neighbors of points u0 and u1 belong to the intersection of the neighbors of points u0 and u1 on relation graph 110.

[0117] In the subgraph model, point v3 is a common neighbor of points v0 and v1, but not a neighbor of point v2. If a point u2 is determined to be point v2 in relation graph 110, then the points that are not neighbors of u2 among the common neighbors of points u0 and u1 are successively designated as points v3.

[0118] By sequentially identifying the points that are not neighbors of u2 among the common neighbors of points u0 and u1, we can determine the valid subgraph.

[0119] The core of the depth-first search algorithm is a series of nested for loops and set operations:

[0120] for u0∈A

[0121] for u1∈N(u0)

[0122] for u2∈N(u0)∩N(u1)

[0123] for u3∈N(u0)∩N(u1)- N(u2)

[0124] Here, set A is the set of points in relation graph 110, N(ui) represents the set of neighbors of point ui, i∈{0,1,2,3}, and “∩” represents the intersection. N(u0)∩N(u1)- N(u2) represents the intersection of the set of neighbors of point u0 and the set of neighbors of point u1, minus the points in the set of neighbors of point u2.

[0125] Whenever the for loop of the graph mining algorithm reaches the innermost layer, if there is a vertex u3, then there is a valid subgraph.

[0126] Graph mining algorithms for Figure 1 Processing the relationship diagram 110 shown can yield three valid subgraphs.

[0127] Compressed sparse row (CSR) format can be used to... Figure 1 The relationship diagram 110 shown is stored.

[0128] For a relationship graph, a vector corresponding to each point can represent whether it is connected to other points. The number of vectors corresponding to each point can be equal to the number of points in the relationship graph. For example... Figure 1 The relationship diagram 110 shown includes 8 points. Each point in the diagram can have a corresponding vector of 8 bits, with each bit indicating whether the point is connected to other points. The same bits in different vectors indicate whether points corresponding to different vectors are connected to the same point. In other words, the same bits in different vectors correspond to the same point.

[0129] For example, Figure 1 In the relationship graph 110 shown, the vector corresponding to a certain point has its i-th bit indicating whether there is a connection between that point and point i, where i ∈ [0,7] and i is an integer. There is no "edge" between points 0 and 1, i.e., there is no connection, so the 0th bit of the vector corresponding to point 0 is 0; there is an "edge" between points 0 and 1, i.e., there is a connection, so the 1st bit of the vector corresponding to point 0 is "1"; there is no connection between points 0 and 2, so the 2nd bit of the vector corresponding to point 0 is "0". Therefore, it can be determined that the vector N(0) corresponding to point 0 is "01010011" and the vector N(1) corresponding to point 1 is "10101110".

[0130] An adjacency matrix can include vectors corresponding to each point in the graph. Each vector corresponding to a point in the graph can be a row in the adjacency matrix. The order of the points in each row of the adjacency matrix can be the same as the order of the points corresponding to each bit in each vector.

[0131] The adjacency matrix can be stored using the CSR format, meaning that the relationship graph can be represented using CSR format data.

[0132] Figure 2 This is a diagram of CSR format data.

[0133] CSR format data is obtained by encoding the data as a whole. CSR format data consists of three parts: row offset, column index, and graph data value. The i-th number in the row offset indicates the starting position of the first non-zero element in the i-th row of the matrix. The column index indicates the column coordinate of the column containing the non-zero element, and the graph data value indicates the specific value of the non-zero element.

[0134] exist Figure 1 In the adjacency matrix of the relation graph 110 shown, row 0 has N(0) = 01010011 and row 1 has N(1) = 10101110. The column coordinates of the columns containing the non-zero elements in row 0 are 1, 3, 6, and 7, a total of 4 positions; the column coordinates of the columns containing the non-zero elements in row 1 are 0, 2, 4, 5, and 6, a total of 5 positions. Therefore, the adjacency matrix can be represented in CSR format as: row offset "0, 4...", column index "1, 3, 6, 7, 0, 2, 4, 5, 6...", graph data value "1, 1, 1, 1, 1, 1, 1, 1, 1..."

[0135] In a relational graph represented using CSR format, the column index portion corresponding to each point can be understood as the neighbor set of that point. By determining the intersection of the neighbor sets of two points, we can identify the query point that has a relationship with points corresponding to both sets.

[0136] When performing a set merge operation on the neighbor sets of each point in an adjacency matrix stored in CSR format, the column index of the corresponding part needs to be extracted first based on the point's index and the corresponding row offset. Then, the two sets of column indexes are compared, and the points with the same index are selected. The point indicated by the same index is the query point.

[0137] For example, based on row offsets "0, 4, 9...", the positions of the non-zero values ​​in the vector corresponding to point 0 (i.e., the neighbor set corresponding to point 0) are determined by the 0th to 3rd numbers in the column index, and the positions of the non-zero values ​​in the vector corresponding to point 1 (i.e., the neighbor set corresponding to point 1) are determined by the 4th to 8th numbers in the column index. Then, as... Figure 3 As shown, the 0th to 3rd numbers in the column index are compared with the 4th to 8th numbers in the column index, that is, the neighbor set of point 0 is compared with the neighbor set of point 1 to determine the intersection of the neighbor set of point 0 and the neighbor set of point 1.

[0138] The data processing device can be used to compare the column index portions of two points. However, the processing capacity of the data processing device is limited. The data processing device can compare two neighbor sets, each containing no more than a preset number of ordinal numbers.

[0139] If the number of data in one of the two column indexes exceeds a preset value, the data in that column index needs to be grouped into multiple arrays. Then, a data processing device can be used to compare each of these arrays with the other column index, but this requires a large amount of computation and a long processing time.

[0140] To address the aforementioned problems, this application provides a data processing method.

[0141] Figure 19 This is a schematic flowchart of a data processing method based on a data processing device provided in an embodiment of this application.

[0142] The data processing device includes a processing matrix, which comprises v×v processing units, where v is a positive integer.

[0143] Method 2200 includes S2210 to S2220.

[0144] In S2210, a first target data group and a second target data group are acquired. The first target data group is the first data group among a plurality of first data groups in a first data set, and the second target data group is the first data group among at least one second data group in a second data set. Each data group in each data set of the first data set and the second data set includes at least one data item, and each data item includes a key value. The data groups in each data set are arranged in a first order or a second order. When arranged in the first order, each key value in any data group in each data set is less than each key value in the data group following the first data group. When arranged in the second order, each key value in any data group in each data set is greater than each key value in the data group following the first data group.

[0145] The first data set and the second data set can be stored in memory. The first data set includes multiple first data groups, and the second data set includes multiple second data groups.

[0146] Obtaining the first target data group and the second target data group can be achieved by reading the first target data group and the second target data group from memory.

[0147] In S2220, multiple iterations are performed.

[0148] Each iteration includes inputting the first target data set and the second target data set into the data processing device, the data processing device being used to determine the key values ​​that are equal in the first target data set and the second target data set.

[0149] Each iteration further includes, in the case that the data groups in each data set are arranged in the first order and the first key value is less than or equal to the second key value, or in the case that the data groups in each data set are arranged in the second order and the third key value is greater than or equal to the fourth key value, obtaining the first data group in the first data set that is located after the first target data group as the first target data group, wherein the first key value is the largest key value in the first target data group, the second key value is the largest key value in the second target data group, the third key value is the smallest key value in the first target data group, and the fourth key value is the smallest key value in the second target data group.

[0150] For two data sets, each data group in the data set is arranged in order of the size of the key value. When using a data processing device to process the data, if there are multiple first data groups in the first data set, through iteration, for the first data group whose key values ​​all exceed the range of key values ​​in the second target data group, it is no longer necessary to input the data processing device, that is, no longer necessary to compare with the second target data group, thereby reducing the number of first data groups compared with the second target data group and thus reducing the amount of computation.

[0151] Figure 10 Let's take the example of the first and second data sets being the same. The left column represents the first data set, and the right column represents the second data set. The first data set includes 5 data points with key values ​​of 1, 4, 7, 8, and 10. The second data set includes 9 data points with key values ​​of 1, 2, 3, 4, 5, 6, 7, 8, and 9. When v=3, the first data set can include two first data groups: the first first data group includes 3 data points with key values ​​of 1, 4, and 7, and the second first data group includes 2 data points with key values ​​of 8 and 10. The second data set can include three second data groups: the first second data group includes 3 data points with key values ​​of 1, 2, and 3; the second second data group includes 3 data points with key values ​​of 4, 5, and 6; and the third second data group includes 3 data points with key values ​​of 7, 8, and 9.

[0152] Through S2210, a first data group including key values ​​1, 4, and 7 and a second data group including key values ​​1, 2, and 3 can be obtained, serving as the first target data group and the second target data group, respectively. During S2220, since the largest key value 7 in the first target data group is greater than the largest key value 4 in the second target data group, the next first data group is no longer compared with the second target data group including key values ​​1, 2, and 3. This reduces the number of first data groups compared with the second data group, thus reducing the computational load.

[0153] In addition, during the execution of method 2200, it is not necessary to obtain all the first data group and the second data group in advance, which reduces the processing and storage capacity requirements of the device executing method 2200 and improves the flexibility and adaptability of the method.

[0154] If there are multiple second data groups in the second data set, then if the data groups in each data set are arranged in the first order and the first key value is greater than or equal to the second key value, the second data group in the second data set that is after the second target data group is obtained as the second target data group.

[0155] If there are multiple second data groups in the second data set, then if the data groups in each data set are arranged in the second order and the third key value is less than or equal to the fourth key value, the second data group in the second data set that is after the second target data group is obtained as the second target data group.

[0156] Still with Figure 10 Let's take an example. During the S2220 process, the largest key value 7 in the first target data group is greater than the largest key value 3 in the second target data group. Therefore, the next second data group after the first second data group can be obtained and used as the second target data group. The second second data group includes three data points with key values ​​of 4, 5, and 6 respectively.

[0157] During the second iteration, the largest key value 7 in the first target data group is greater than the largest key value 6 in the second target data group. Therefore, the next second data group of the second second data group can be obtained and used as the second target data group.

[0158] During the third iteration, the largest key value 7 in the first target data group is smaller than the largest key value 9 in the second target data group. Therefore, the next first data group of the current first target data group (i.e. the first first data group) can be obtained and used as the new first target data group.

[0159] When the first and second orders are the same, and both the number of the first and second data groups are multiple, the next target data group in the data set to which the target data group with the smaller key value belongs is taken as a target data group in the next iteration, while the target data group with the larger key value is taken as another target data group in the next iteration. This ensures that the data group in the other data set that is compared with a data group in a certain data set includes key values ​​within the range of the minimum key value and the maximum key value in that data group. It also reduces the likelihood of comparing that data group with a data group in the other data set that only includes key values ​​outside this range, thereby improving computational efficiency and reducing computational load.

[0160] The data processing device may include a processing matrix, which includes v×v processing units, where v is a positive integer, and the number of at least one data in each of the first and second target data groups is less than or equal to v.

[0161] During each iteration, the first target data group and the second target data group can be input into the processing matrix according to the input rules. The input rules are such that the p-th second data in the second target data group is input into the q-th processing unit along the first direction among the v processing units located on the second edge in the q-th input cycle of the iteration, and the i-th first data in the first target data group is input into the j-th processing unit along the second direction among the v processing units located on the first edge in the j-th input cycle of the iteration. The first target data group and the second data group are input at the beginning of the same input cycle, the first edge and the second edge are adjacent, the first direction is the direction from the second edge to the interior of the processing matrix and perpendicular to the second edge, and the second direction is the direction from the first edge to the interior of the processing matrix and perpendicular to the first edge.

[0162] Each processing unit in the processing matrix is ​​used to determine whether the key value in the first data input to the processing unit in the same input cycle is equal to the key value in the second data, wherein the first data is the data belonging to the first target data group and the second data is the data belonging to the second target data group.

[0163] When v is greater than 1, each processing unit in the processing matrix is ​​further configured to, in the next input cycle after receiving the first data and the second data, transmit the first data to the next processing unit along the first direction and transmit the second data to the next processing unit along the second direction.

[0164] Using a processing matrix to process the first target data group and the second target data group can improve processing efficiency.

[0165] The processing matrix can be understood as a logical matrix. Whether the actual physical locations of the processing units in the processing matrix are arranged in rows and columns is not limited in this embodiment. The first direction and the second direction can be understood as logical directions in the processing matrix.

[0166] In cases where the key values ​​in different data sets are different, each processing unit in the processing matrix is ​​configured to, when the key value in the first data is not equal to the key value in the second data, transmit the first data to the next processing unit along the first direction and transmit the second data to the next processing unit along the second direction.

[0167] When the key values ​​in different data sets are different, if the key value in the first data set is equal to the key value in the second data set, this equal key value will not be equal to the key values ​​in other data sets. Therefore, the processing unit can transmit the first data to the next processing unit along the first direction and the second data to the next processing unit along the second direction only if the key value in the first data set is not equal to the key value in the second data set. This reduces data transmission and computational load.

[0168] The data processing apparatus may further include a filtering matrix comprising v filtering units, each of which is located after the last processing unit in each of the v rows of the processing matrix along the first direction along the second direction.

[0169] Each processing unit in the processing matrix is ​​further configured to, in the next input cycle after receiving the first data and the second data, transmit the processing result of the processing unit along the second direction to the next unit, wherein the unit is the processing unit or the filtering unit, and the processing result includes the equal key values, when the key value in the first data is equal to the key value in the second data.

[0170] Each processing unit in the processing matrix is ​​further configured to, in the next input cycle after receiving the processing result, transmit the processing result along the second direction to the next unit.

[0171] Method 2200 further includes, when the first key value is greater than or equal to the second key value, controlling the v filtering units along the first direction to sequentially output the processing result corresponding to the second target data group according to the input cycle.

[0172] Therefore, the processing results corresponding to each key value in a certain second data group can be uniformly output after the second data group is last input into the data processing device, thereby improving the flexibility of the processing result output.

[0173] The data processing apparatus may further include a compression triangular matrix. The compression triangular matrix includes v rows of compression units along the first direction, with the number of compression units increasing row by row along the first direction.

[0174] The compression units in the v-row can each be located after a filter unit along the second direction.

[0175] Each of the plurality of compression units is configured to receive the processing result output by the filtering unit preceding the compression unit along the second direction, or to receive the processing result output by the compression unit in a row along the first direction.

[0176] Each of the plurality of compression units is configured to, in the next input cycle after receiving the processing result, transmit the processing result to the compression unit in the next row along the first direction.

[0177] By setting a compression triangular matrix in the data processing device, the processing results corresponding to a certain second data group can be output in the same input cycle, improving the flexibility of the processing result output.

[0178] Method 2200 can be applied to graph mining.

[0179] Different key values ​​correspond to different point sets in the relationship graph. The first data is used to indicate whether there is a relationship between the first target point in the relationship graph and at least one point in the point set corresponding to the key value in the first data. The second data is used to indicate whether there is a relationship between the second target point in the relationship graph and at least one point in the point set corresponding to the key value in the second data.

[0180] Each processing matrix is ​​also used to output a processing result when the key value in the first data is equal to the key value in the second data. The processing result is used to indicate the query point in the relationship graph, and the relationship between the query point and the two target points conforms to a preset condition.

[0181] The preset scenario can be one of four cases: related to both the first and second target points; related to the first target point but not to the second target point; not related to the first target point but related to the second target point; or not related to either the first or second target point. The preset scenario can be determined based on the subgraph model in graph mining.

[0182] Using method 2200, the query point of the connection between two target points in the relation graph can be determined, thereby identifying the subgraph in the relation graph that has a certain subgraph model structure.

[0183] For two target points, there is at most one identical key value in each data set. Therefore, if it has been determined that a key value in a certain data set of the first target point is equal to that in a certain data set of the second target point, the processing unit does not need to transmit that data set between the first and second target points, thereby reducing the computational load.

[0184] In other words, during graph mining, each processing unit can be used to transfer the first data to the next processing unit along the first direction and the second data to the next processing unit along the second direction if the key value in the first data is not equal to the key value in the second data.

[0185] It should be understood that the number of points in the point set corresponding to different key values ​​in each data point of the target point can be the same or different. The point set corresponding to each key value can include one or more points.

[0186] In some embodiments, the number of points in the point set corresponding to different key values ​​can all be 1, so each data point of the target point can include only the key value. Different key values ​​of the target point can be used to indicate different points in the relationship graph that are related to the target point, or different key values ​​of the target point can be used to indicate different points in the relationship graph that are not related to the target point. Whether the point indicated by the key value of the target point is related to the target point can be determined, for example, based on the relationship between the query point and the two target points, and this application embodiment does not limit this.

[0187] In other embodiments, the number of points in the point set corresponding to different key values ​​can be multiple. Each data point of the target point may include a key value and a set of relational values ​​for the target point corresponding to that key value. The set of relational values ​​for the target point corresponding to the key value indicates whether the target point has a relationship with each point in the point set corresponding to the key value.

[0188] In other words, the first data also includes a first relation value group of the first target point corresponding to the key value, and the second data also includes a second relation value group of the second target point corresponding to the key value.

[0189] Each processing unit in the processing matrix is ​​further configured to perform a preset operation on each bit of the first relation value group and the second relation value group when the key value in the first data is equal to the key value in the second data. The same bit in the first relation value group and the second relation value group corresponding to the equal key value corresponds to the same point in the point set corresponding to the equal key value. The result of the preset operation on each bit is used to indicate whether the relationship between the point corresponding to the bit and the two target points conforms to the preset condition.

[0190] The key values ​​in the first data are equal to the key values ​​in the second data, meaning that the key values ​​in the first data and the key values ​​in the second data indicate the same set of points. Therefore, the query point can be determined based on the result of the bitwise preset operation on the first relation value group and the second relation value group.

[0191] It should be understood that the processing result of the processing unit may also include the result of pre-defined operations on each bit of the first relation value group and the second relation value group.

[0192] For example, each filtering unit in the filtering matrix is ​​used to determine whether the query point exists based on the received processing result, and output the processing result indicating that the query point exists.

[0193] In other words, the filtering matrix can be used to filter the processing results output by the processing matrix, and to determine the processing results that contain query points among the various processing results output by the processing matrix.

[0194] For details on the filtering matrix, please refer to [link / reference]. Figure 10 and Figure 11 The explanation is as follows. For details on compressed triangular matrices, please refer to [link / reference]. Figure 12 and Figure 13 Explanation.

[0195] When the data groups in each data set are arranged in the first order, the key values ​​in each first data group are arranged from smallest to largest; when the data groups in each data set are arranged in the second order, the key values ​​in each first data group are arranged from largest to smallest.

[0196] The data in each dataset can be arranged in ascending or descending order of key values. Thus, the data groups in the dataset can be stored according to the partitioned format. Alternatively, the apparatus executing method 2000 can partition the dataset.

[0197] For example, when the data groups in each data set are arranged in the first order, in S2210, at least one first data and at least one second data can be acquired. The at least one first data is the first target data group, and the at least one second data is the second target data group. During S2220, each time at least one first data is acquired, it can be considered as a first target data group, and each time at least one second data is acquired, it can be considered as a second target data group. After the iteration ends, at least one first data that was not acquired can be considered as one or more first data groups, and at least one second data that was not acquired can be considered as one or more second data groups.

[0198] When data processing is performed using a data processing device that includes a processing matrix, v first data points and / or v second data points can be acquired each time, thereby improving computational processing efficiency.

[0199] The data in each dataset are arranged in order of the magnitude of the key value, making the division of data groups more flexible.

[0200] For example, during the iteration process, at least one data set with a quantity not exceeding v can be obtained from the first data set as the first target data set.

[0201] Similarly, the key values ​​in each second data group are arranged in ascending order along the second sequence.

[0202] Figure 4 This is a schematic structural diagram of a data processing method provided in an embodiment of this application. The data processing method 500 includes steps S510 to S530.

[0203] In S510, at least one key-value pair of the first target point and at least one key-value pair of the second target point are obtained from two target points in the relationship graph. Each key-value pair of the target points includes the key value of the target point and the relationship value group of the target point corresponding to the key value. Different key values ​​correspond to different point sets in the relationship graph. The relationship value group of the target point corresponding to the key value of the target point is used to indicate whether the target point has a relationship with each point in the point set corresponding to the key value. In each key-value pair of the target point, there are points in the point set corresponding to the key value that have a relationship with the target point.

[0204] In S520, at least one of the key values ​​of the first target point is determined to be equal to at least one of the key values ​​of the second target point.

[0205] In S530, based on the relationship value group of the first target point corresponding to the equal key value and the relationship value group of the second target point, a query point in the relationship graph is determined, and the relationship between the query point and the two target points conforms to a preset condition.

[0206] Generally, in a relational graph, each point has relationships with only a small number of points in the graph. That is, if we use each bit in the relation vector of each point to represent whether the point has a relationship with other points in the graph, where "1" indicates a relationship and "0" indicates no relationship, then the relation vector of each point can be considered as sparse data.

[0207] The points in the relationship graph are divided into multiple point sets, each point set corresponding to a key value. For each point, the relation value set corresponding to that point can be used to represent whether there is a relationship between that point and the points in the corresponding point set. If there is a point in a point set that corresponds to that point, then the key value of that point set and the relation value set corresponding to that key value can be used as a key-value pair for that point.

[0208] Data compression can be achieved by using key-value pairs of points in a relational graph to represent the graph.

[0209] Method 500 utilizes key-value pairs of points in a relationship graph. By identifying equal key values ​​in the key-value pairs of two target points in the graph, and based on the relationship value group corresponding to the equal key values ​​of the two points, it determines query points whose relationship with the two target points conforms to a preset condition. This reduces the amount of data used to represent the relationship graph while also reducing the processing time required to determine query points.

[0210] exist Figure 3 In the method shown, based on the merging of queue sets, the column indices of two points are determined by using the same column indices in the queues to identify points that are related to both points. Figure 3 Compared to the method shown, in method 500, the key value in the key value pair represents the set of points that are related to the target point. The number of key value pairs is generally less than the number of column indices. Therefore, comparing the size of the key values ​​can reduce the amount of computation and reduce the computation time.

[0211] The set of points corresponding to different key values ​​may or may not contain the same points. Removing the duplicate points from the set of points corresponding to different key values ​​can further improve data compression and reduce the storage space required to store the key-value pairs used to represent each point in the graph.

[0212] In S510, relationship graph data can be acquired, and based on the relationship graph data, at least one key-value pair for each of the two target points can be determined.

[0213] The key-value pairs of each point in a relationship graph can be represented using relationship graph data. Relationship graph data includes row offset vectors, key value vectors, and relationship value vectors.

[0214] The key value vector includes at least one key value for each point in the relationship graph.

[0215] The row offset vector is used to indicate the position of at least one key value of each point in the key value vector.

[0216] For example, the row offset vector may include multiple offset information, each offset information representing the starting position of at least one key value of a point in the key value vector. The order of the offset information of each point in the row offset vector in the relationship graph may be the same as the order of the key values ​​of that point in the key value vector.

[0217] Offset information can be an index. The key values ​​of each point are continuous in the key value vector. The offset information of a point is used to indicate the starting index of at least one key value of that point in the key value vector.

[0218] The relation value vector includes the relation value group of the points corresponding to the key values ​​of each point in the relation graph. The order of at least one key value of each point in the key value vector is the same as the order of the relation value group of the points corresponding to the key values ​​of each point in the relation value vector.

[0219] Using relational graphs to represent relational data can further improve the degree of data compression.

[0220] Different key values ​​may correspond to the same or different numbers of points in the point set. For example, cardinal and even key values ​​may correspond to different numbers of points in the point set.

[0221] The starting position of the relation value group corresponding to each key value of the target point in the relation value vector can be determined based on the number of points in the point set corresponding to each key value preceding at least one key value of the target point in the key value vector. Then, starting from this starting position, the relation value group corresponding to each key value of the target point is determined based on the number of points in the point set corresponding to each key value of the target point.

[0222] When the number of points in the point set corresponding to different key values ​​is equal, the relation value group corresponding to each key value of the target point can be determined based on the position of at least one key value of the target point in the key value vector and the number of points with the same value in the point set. This makes it easier to determine the key-value pairs of the target point based on the relation graph data.

[0223] In S530, preset operations can be performed on each bit of the relation value group of the first target point and the relation value group of the second target point corresponding to the equal key values.

[0224] In a set of relational values ​​for different points corresponding to the same key value, the same bit corresponds to the same point in the set of points corresponding to that key value.

[0225] In a relational value group corresponding to the same key value, different bits can correspond to different points in the point set corresponding to that key value, thereby reducing the amount of computation.

[0226] In the relational value group corresponding to the same key value, the result of the preset operation on each bit is used to indicate whether the relationship between the point corresponding to that bit and the two target points meets the preset condition.

[0227] Pre-defined operations performed on each bit in the relation value group of two target points corresponding to the same key value can be performed in parallel. This improves computational efficiency.

[0228] Furthermore, at least one key-value pair of the first target point and at least one key-value pair of the second target point can be processed using a processing matrix.

[0229] In S520, at least one key-value pair of the first target point can be sequentially input into a plurality of processing units located on the first edge of the processing matrix, and at least one key-value pair of the second target point can be sequentially input into a plurality of processing units located on the second edge of the processing matrix to determine the equal key value. The processing matrix includes v×v processing units, where v is a positive integer greater than 1. The i-th key-value pair in the first target data group is input into the j-th processing unit along the second direction of at least one processing unit located on the first edge in the j-th input cycle. The p-th key-value pair in the second target data is input into the q-th processing unit along the first direction of at least one processing unit located on the second edge in the q-th input cycle. The first edge is adjacent to the second edge. The first direction is a direction from the second edge pointing into the interior of the processing matrix and perpendicular to the second edge. The second direction is a direction from the first edge pointing into the interior of the processing matrix and perpendicular to the first edge. Each processing unit is used to determine whether the key value of the first target point input into the processing unit is equal to the key value of the second target point.

[0230] In S530, the processing matrix can be used to process the relationship value group of the first target point and the relationship value group of the second target point corresponding to the equal key values ​​to determine the query point. Each of the plurality of processing units is used to determine the query point in the relationship graph based on the relationship value group of the first target point and the relationship value group of the second target point input to the processing unit when the key value of the first target point is equal to the key value of the second target point. The relationship between the query point and the two target points conforms to a preset condition.

[0231] Each of the plurality of processing units is further configured to, according to the input cycle, transmit the key-value pair of the first target point to the next processing unit along the first direction, and transmit the key-value pair of the second target point to the next processing unit along the second direction.

[0232] The processing matrix can be understood as a systolic array, where key-value pairs of each target point flow rhythmically between processing units in a "pipeline" manner. All processing units process the flowing data in parallel, which can improve processing speed and reduce processing time.

[0233] For the specific structure of the processing matrix, please refer to Figures 7 to 15 Explanation.

[0234] Each processing unit may be specifically configured to: transmit the key-value pair of the first target point to the next processing unit along the first direction, and transmit the key-value pair of the second target point to the next processing unit along the second direction, when the key value of the first target point is not equal to the key value of the second target point.

[0235] The key values ​​in each key-value pair of a target point are different. If the key value of the first target point is not equal to the key value of the second target point, the processing unit transmits the key values ​​of both the first and second target points. However, if the key value of the first target point is equal to the key value of the second target point, the key-value pair is not transmitted. Therefore, the equal key value does not need to be compared with other key values, reducing the computational load.

[0236] Even if the number of key value pairs at the first target point is greater than v, the processing matrix can still be used to process the key value pairs at both the first and second target points. For details, please refer to [link to documentation]. Figure 10 and Figure 15 Explanation.

[0237] Figure 5This is a schematic flowchart illustrating a method for generating relational graph data according to an embodiment of this application. The method 600 for generating relational graph data includes steps S601 to S603.

[0238] A relationship graph consists of multiple interconnected nodes and edges connecting them. A relationship graph can be represented as a neighbor vector for each node. Each neighbor vector contains the same number of bits. The i-th bit in each node's neighbor vector indicates whether there is a relationship between that node and the i-th node in the relationship graph. In other words, the i-th bit in the neighbor vector corresponds to the i-th node in the relationship graph. A "0" indicates no relationship, and a "1" indicates a relationship.

[0239] For relation graph 110, the neighbor vector corresponding to point 0 is N(0)=01010011, and the neighbor vector corresponding to point 1 is N(1)=10101110.

[0240] In S601, neighbor vectors can be divided into multiple groups according to a preset method.

[0241] Each group can contain the same number of bits.

[0242] For example, the neighbor vector can be divided into multiple groups based on the remainder when each bit in the neighbor vector is divided by a certain divisor. Different groups correspond to different remainders. This divisor can be a preset value.

[0243] Alternatively, neighbor vectors can be divided into groups of a fixed number of bits, from left to right. If the number of bits in a neighbor vector is not divisible by the number of bits in a group, zeros can be added to the end of the neighbor vector to ensure that all groups have the same number of bits.

[0244] The following example illustrates how to partition neighbor vectors using a bit depth of 2.

[0245] In each neighbor vector, the bits corresponding to points 0 and 1 are grouped together, the bits corresponding to points 2 and 3 are grouped together, and so on, to complete the partitioning of the neighbor vectors.

[0246] The partitioned neighbor vector N(0) consists of 4 groups, namely 01, 01, 00, and 11. The partitioned neighbor vector N(1) consists of 4 groups, namely 10, 10, 11, and 11.

[0247] In S602, a key value is assigned to each group.

[0248] The key value of a group can also be understood as the group's identifier. Groups formed by partitioning different neighbor vectors, including groups with the same position in the neighbor vectors, have the same key value. For example, the key value of each group can be its index within the multiple groups formed by partitioning the neighbor vectors.

[0249] The key values ​​corresponding to the four groups 01, 01, 00, and 11 obtained by partitioning N(0) are 0, 1, 2, and 3, respectively. The key values ​​corresponding to the four groups 10, 10, 11, and 11 obtained by partitioning N(1) are 0, 1, 2, and 3, respectively.

[0250] Each point's key value array can include multiple key values ​​for that group, and not all bits of the group corresponding to a key value in the key value array are all 0. That is, when forming the key value array, the key values ​​corresponding to the group where all bits are 0 are discarded.

[0251] The key value array for point 0 can include 0, 1, and 3, while the key value array for point 1 can include 0, 1, 2, and 3.

[0252] In a key value array, the key values ​​can be arranged in a preset order. For example, the key values ​​can be arranged in ascending or descending order. The following example illustrates this using an ascending order arrangement of key values.

[0253] A key value in the key value array corresponding to each point and the group corresponding to that key value can be understood as a key-value pair.<key, value> (Key-value pair, KVP). That is, each point can have one or more key-value pairs.

[0254] In S603, generate row relationship diagram data.

[0255] The relationship graph data includes row offsets, key value vectors, and relationship value vectors.

[0256] The relation value vector consists of key value arrays for each point in the relation graph. The j-th relation value array in the relation value vector is the relation value array for the j-th point in the relation graph.

[0257] Row offsets indicate the starting position of the key value array for each point in the relational graph within the relational value vector. Row offsets can also be referred to as row data. The j-th row offset indicates the starting position of the key value array for the j-th point in the relational graph within the relational value vector.

[0258] A relation value vector consists of groups corresponding to each key value array. The order of the groups in the relation value vector is the same as the order of the key value arrays. A relation value vector can also be called value data.

[0259] Key value vectors can also be called key data.

[0260] The data format of relational graph data can be called bitmap with compressed sparse row (BCSR) format.

[0261] The neighbor matrix consists of multiple rows, each representing a neighbor vector of a point. Method 600 can be used to compress the neighbor matrix, particularly when the proportion of "0"s in the neighbor matrix is ​​high, resulting in good compression.

[0262] In graph mining, it is necessary to calculate the intersection of the neighbor sets of two points. For relational graph data generated by method 600, this can be achieved using... Figure 6 The data is processed using the methods shown.

[0263] Figure 6 This is a schematic flowchart of a data processing method provided in an embodiment of this application. Method 700 includes steps S701 to S702.

[0264] Prior to S701, relationship graph data could be obtained.

[0265] Relationship graph data is used to represent whether relationships exist between points in a relationship graph. Relationship graph data includes row offsets, key value vectors, and relationship value vectors.

[0266] In S701, based on the relationship diagram data, the key values ​​for two points to be equal are determined.

[0267] These two points can be determined based on the subgraph model.

[0268] Based on the relationship graph data, the key-value pairs of each point in the relationship graph can be determined, that is, the key value array of each point, and the group corresponding to each key value in the key value group of each point.

[0269] For the relationship graph data in Graph 110, the key value array for point 0 is determined as "0, 1, 3", and the key value array for point 1 is "0, 1, 2, 3". For point 0, the group corresponding to key value 0 is 01, the group corresponding to key value 1 is 01, and the group corresponding to key value 3 is 11. For point 1, the group corresponding to key value 0 is 10, the group corresponding to key value 1 is 10, the group corresponding to key value 2 is 11, and the group corresponding to key value 3 is 10.

[0270] Therefore, for points 0 and 1, the key values ​​that are equal include 0, 1, and 3.

[0271] In S702, determine the bit with a value of 1 in the calculation result obtained by the preset calculation method in each group corresponding to the two points with equal key values.

[0272] Different preset calculation methods correspond to different relationships between the two points. For example, the preset calculation method for a common neighbor of two points, i.e., a relationship with both points, is as follows: Where a and b represent the values ​​of the two points at a certain position; for the case where the point is related to point A but not to point B, the default calculation method is as follows. Where 'a' represents the value of point A at a certain position. This indicates that the value of a certain bit at point B is inverted. Therefore, a bit that results in a 1 can be understood as a bit whose relationship with the two points satisfies a preset condition. The following explanation uses the determination of common neighbors of two points as an example.

[0273] For the key value 0, the group corresponding to point 0 is 01, and the group corresponding to point 1 is 10. Perform a bitwise comparison between "01" and "10" to determine the bit that results in a 1 in the bitwise AND operation.

[0274] For example, a bitwise AND operation can be performed on the groups corresponding to each equal key value of the two points, and the bit that results in 1 is the bit corresponding to the two points that are related.

[0275] Based on equal key values, the set of points corresponding to those equal key values ​​can be determined. Then, based on the position of the bit with a calculation result of 1 within its group, the point in the relation graph corresponding to that bit can be determined.

[0276] For example, the grouping method of the neighbor matrix can also be obtained. Based on the grouping method of the neighbor matrix, the key value corresponding to the group including the bit with a calculated result of 1, and the position of the bit with a calculated result of 1 in its group, the point in the relation graph corresponding to the bit is determined.

[0277] For example, comparing points 0 and 1 in the groups corresponding to key values ​​of 0, 1, and 3, we can determine that the groups corresponding to key values ​​of 0 and 1 do not have any bits with a calculated result of 1, while in the group with key value of 3, the bit with a calculated result of 1 is the 0th bit of the group. The group corresponding to key value 3 in the neighbor matrix includes points 6 and 7, where the 0th bit corresponds to point 6. Therefore, we can determine that point 6 is the common neighbor of points 0 and 1.

[0278] By using method 700, common neighbors of two points can be determined by identifying equal key values ​​in the key value arrays corresponding to the two points and determining the equal positions of the two points in each group corresponding to the equal key values.

[0279] To determine the equal key values ​​in the key value array corresponding to two points, we can use... Figure 3 The example shown is a queue-based set merging implementation. Determining the equal bits of two points in the groups corresponding to their equal key values ​​can be achieved using bitwise comparisons.

[0280] In bitwise comparison, the comparison of multiple bit values ​​can be performed in parallel, resulting in high computational efficiency.

[0281] Compared to Figure 2In CSR format data, when comparing the different column index portions corresponding to two points, method 700 determines the equal key values ​​in the key value array corresponding to the two points, which can effectively reduce the number of numbers that need to be compared and improve processing efficiency.

[0282] The above text combined Figures 1 to 6 The method embodiments of this application are described below, in conjunction with the following. Figures 7 to 10 This section describes the apparatus embodiments of the present application. It should be understood that the descriptions of the method embodiments correspond to the descriptions of the apparatus embodiments; therefore, any parts not described in detail can be found in the foregoing method embodiments.

[0283] Figure 7 This is a schematic structural diagram of a data processing device provided in an embodiment of this application.

[0284] The data processing device 800 includes a processing matrix (PM) 810, a filter array (FA) 820, and a compact triangle (CT) 830.

[0285] The processing matrix 810 includes v×v processing elements (PEs) for processing key-value pairs between two points.

[0286] Each key-value pair for a point includes an array of key values ​​for that point, and the corresponding groups for each key value. The leftmost v processing units in processing matrix 810 can receive multiple key-value pairs for point A in the graph; different processing units are used to receive different key-value pairs from A's key-value pairs. The topmost v processing units in processing matrix 810 can receive multiple key-value pairs for point B in the graph; different processing units are used to receive different key-value pairs from B's key-value pairs. Point A and point B are different points.

[0287] The leftmost v processing units, in a top-to-bottom order, receive key-value pairs one clock cycle after the previous processing unit receives them. The topmost v processing units, in a left-to-right order, receive key-value pairs one clock cycle after the previous processing unit receives them.

[0288] The processing unit compares the key values ​​in the key-value pairs of the two received points. Alternatively, it can be understood as matching the key values ​​in the key-value pairs of the two received points. If the key values ​​in the key-value pairs of the two received points are equal, then the key values ​​of the two points are considered to have matched successfully.

[0289] If the key values ​​are equal, the processing unit performs a preset calculation on the key-value pair of the two points to determine the bit in the group that matches the relationship corresponding to the preset calculation method, and outputs the key value and the calculation result.

[0290] If the key values ​​are not equal, the processing unit will no longer perform the preset calculation comparison on the group of the key value pair of the two points, and in the next clock cycle, it will transmit the key value pair received from the processing unit on the left to the processing unit on the right, and transmit the key value received from the processing unit on the upper side and the group corresponding to the key value to the processing unit on the lower side.

[0291] like Figure 8 As shown in (a), at time point t0, the key-value pair A0 of point A is input from the processing unit PE0 in the first row and first column from the left, and the key-value pair B0 of point B is input from the top PE0.

[0292] PE 0 compares the key values ​​in key-value pair A0 and key-value pair B0. If the key values ​​in key-value pair A0 and key-value pair B0 are equal, PE 0 performs a preset calculation on each bit of the groups in key-value pair A0 and key-value pair B0. PE 0 then does not transmit key-value pair A0 and key-value pair B0 further. Processing matrix 810 can output the equal key values ​​and the calculation results.

[0293] If the key values ​​in key-value pair A0 and key-value pair B0 are not equal, PE 0 may not perform the preset calculation method on the groups in key-value pair A0 and key-value pair B0.

[0294] like Figure 8 As shown in (B), PE0 can transfer key-value pair A0 to PE1 in the right row and second column at time t1, and transfer key-value pair B0 to PE2 in the lower row and first column. Furthermore, at time t1, key-value pair A1 of point A is input from processing unit PE2 in the second row and first column from the left, and key-value pair B1 of point B is input from PE1 from above.

[0295] Therefore, PE 1 can compare the key values ​​in key-value pair A0 and key-value pair B1. If the key values ​​in key-value pair A0 and key-value pair B1 are equal, PE 1 performs a preset calculation on each bit in the group of key-value pair A0 and key-value pair B1. PE 1 then does not transmit key-value pair A0 and key-value pair B1 further. Processing matrix 810 can output the key value and the calculation result.

[0296] If the key values ​​in key-value pair A0 and key-value pair B1 are not equal, PE 1 may not determine the equal bits in the groups of key-value pair A0 and key-value pair B1, and may transmit key-value pair A0 to the PE to its right and key-value pair B1 to the PE below it at time t2. If there are no other PEs to the right of PE 1, PE 1 may delete the key-value pair received from the left, i.e., delete key-value pair A0.

[0297] PE 2 can compare the key values ​​in key-value pair A1 and key-value pair B0. If the key values ​​in key-value pair A1 and key-value pair B0 are equal, PE 2 performs a preset calculation on each bit of key-value pair A1 and key-value pair B0. PE 2 then does not transmit key-value pair A1 and key-value pair B0 further. Processing matrix 810 can output the equal key values ​​and the calculation results for each bit.

[0298] If the key values ​​in key-value pair A1 and key-value pair B0 are not equal, PE 2 may stop performing the preset calculation method on the pair A1 and B0, and may transmit key-value pair A1 to PE 3 on the right and key-value pair B1 to the PE below at time t2. If there are no other PEs below PE 2, PE 2 may delete the key-value pair received from above, i.e., delete key-value pair B0.

[0299] If the key values ​​in key-value pair A0 and key-value pair B1, and key-value pair A1 and key-value pair B0 are not equal, then at time point t2, key-value pair A1 and key-value pair B1 are transmitted to PE 3, so that PE 3 can compare the key values ​​in key-value pair A1 and key-value pair B1.

[0300] Figure 9 This is a schematic structural diagram of a processing unit provided in an embodiment of this application.

[0301] The processing unit 1000 includes a comparison unit 1010 and a calculation unit 1020.

[0302] The comparison unit 1010 is used to compare the key value kt input from above with the key value kl input from the left. The comparison unit 1010 can be understood as a key value comparator.

[0303] The calculation unit 1020 is used to perform a preset calculation method on each bit of the group vt input from above and the group vl input from the left when the key value kt is equal to the key value kl.

[0304] When the key value kt is equal to the key value kl, the output data of the processing unit 1000 may include the key value kt and the results of calculations performed on each bit of the two groups input to the processing unit 1000 from above and from the left. The output data of the processing unit 1000 may be transmitted to the processing unit located to the right of the processing unit 1000.

[0305] The comparison unit 1010 can also receive a valid indication 1 input from the left. The valid indication 1 is used to indicate whether the key value of point A input to the processing unit located to the left of the processing unit 1000 in the row of the processing unit 1000 has been successfully matched with the key value of point B.

[0306] If the valid indication 1 received by the processing unit 1000 indicates a successful match, the comparison unit 1010 and the calculation unit 1020 in the processing unit 1000 no longer need to perform calculations. As a result, the amount of computation can be reduced.

[0307] Processing unit 1000 can also output a valid indication 2. If valid indication 1 indicates a successful match, or if the key value kt is equal to kl, valid indication 2 output by processing unit 1000 indicates that the row containing processing unit 1000 has been successfully matched. Valid indication 2 output by processing unit 1000 can be transmitted to the processing unit to the right of processing unit 1000. For example, valid indication 1 and valid indication 2 being "1" indicates a successful match; conversely, valid indication 1 and valid indication 2 being "0" indicates a failed match.

[0308] In other words, if the valid indication 1 fails to match, the processing unit 1000 can receive the key value kt and group vt input from above, and the key value kl and group vl input from the left. The comparison unit 1010 is used to compare the key value kt with the key value kl.

[0309] If the key value kt and the key value kl are not equal, the valid indication 2 output by the processing unit 1000 indicates that the indication was not matched successfully.

[0310] When the key value kt and the key value kl are equal, the calculation unit 1020 performs a preset calculation on each bit of group vt and group vl. When the key value kt and the key value kl are equal, the processing unit 1000 outputs a valid indication 2 indicating a successful match, and the processing unit 1000 outputs the key value of the successful match and the calculation result of the group.

[0311] If valid indication 1 indicates a successful match, processing unit 1000 can receive the key value and the calculation result of the group output by the left processing unit, and output indication information 2 to indicate a successful match, and output the key value and the calculation result of the group.

[0312] The number of key-value pairs at point A and / or point B may exceed v. If the number of key-value pairs at one or more points in points A and B may exceed v, the number of key-value pairs exceeding v can be divided into multiple key-value pair sets, with each set containing no more than v key-value pairs.

[0313] Therefore, some key-value pair sets require multi-side input processing matrix 810, and there are situations where a certain key-value pair set needs to be compared with the key values ​​in multiple other key-value pair sets, such as... Figure 10 As shown.

[0314] The processing matrix 810 can include 3×3 processing units. The key values ​​of point A include 1, 4, 7, 8, 10, etc., and the key values ​​of point B include 1, 2, 3, 4, 5, 6, 7, 8, 9, etc.

[0315] The key-value pairs for point A and point B are arranged in ascending order of key values. Then, each key-value pair set is divided into sets containing three key-value pairs. The key values ​​in the two key-value sets for point A are 1, 4, 7 and 8, 10, respectively. The key values ​​in the three key-value sets for point B are 1-3, 4-6, and 7-9, respectively.

[0316] First, using processing matrix 810, the set of key values ​​with the smallest key value among multiple key value sets of point A, and the set of key value pairs with the smallest key value among multiple key value sets of point B are processed. That is, the set of key value pairs for point A including key values ​​1, 4, and 7, and the set of key value pairs for point B including key values ​​1-3 are input into processing matrix 810 from the left and top sides respectively. Figure 10 As shown.

[0317] Then, the largest key value in the set of two key-value pairs of the input processing matrix 810 can be compared.

[0318] If the two largest key values ​​belonging to two different key-value pair sets are not equal, the key-value pair set to which the larger of the two largest key values ​​belongs is input into the processing matrix 810 again, and the key-value pair set of the next smallest key value of the other point is input into the processing matrix 810.

[0319] like Figure 10 As shown, the key value 7 in the key value pair set of point A is greater than the key value 3 in the key value pair set of point B. The key value pair set of point A, which includes key values ​​1, 4, and 7, and the key value pair set of point B, which includes key values ​​4-6, are input into the processing matrix 810 from the left and top sides, respectively.

[0320] The set of key-value pairs for point A, including key values ​​1, 4, and 7, and the set of key-value pairs for point B, including key values ​​1-3, are input into processing matrix 810 at the beginning of the same clock cycle. The set of key-value pairs for point A, including key values ​​1, 4, and 7, and the set of key-value pairs for point B, including key values ​​4-6, are also input into processing matrix 810 at the beginning of the same clock cycle.

[0321] It should be understood that the time difference between the start of input processing matrix 810 for the key-value pair set of points B including key values ​​1-3 and the start of input processing matrix 810 for the key-value pair set of points B including key values ​​4-6 can be one or more clock cycles.

[0322] The filter array 820 can be used to merge the results of comparing the key values ​​of a set of key-value pairs with those of multiple other sets of key-value pairs, thereby outputting an intermediate result corresponding to that key-value pair.

[0323] The filter array 820 includes v filter units (FUs). For example, each filter unit may correspond to a row in the processing matrix 810. When a set of key-value pairs is repeatedly input to the left side of the processing matrix 810, the filter array 820 is used to merge the results of key-value comparisons between that set of key-value pairs and several other sets of key-value pairs.

[0324] In addition, the filtering unit can also be used to determine whether there are points that conform to a preset relationship based on the calculation results of each bit of group vt and group vl. Points that conform to the preset relationship are those corresponding to bits with a calculation result of 1.

[0325] Figure 11 This is a schematic structural diagram of a filtering unit provided in an embodiment of this application.

[0326] The filtering unit 1200 includes a logic processing unit 1210, a valid indication update unit 1220, a valid indication register 1230, and a result register 1240.

[0327] The logic processing unit 1210 is used to determine whether the calculation result of the pair output by the processing unit 1000 in the row of the processing matrix 810 corresponding to the filtering unit 1200 is all "0".

[0328] If the calculation result of a group is all "0", the data in that row can be considered invalid. If the calculation result of a group is all "0", the logic processing unit 1210 can output "0"; otherwise, if the calculation result of a group is not all "0", the logic processing unit 1210 can output "1".

[0329] The valid indication update unit 1220 can receive the output of the logic processing unit 1210 and the valid indication 2 output by the last processing unit 1000 in the row of the processing matrix 810 corresponding to the filtering unit 1200.

[0330] If the last processing unit 1000 in the row of the processing matrix 810 corresponding to the logic processing unit 1210 outputs a valid indication 2 of "0" (i.e., indicating that the row of the processing unit 1000 corresponding to the filtering unit 1200 did not match the key value successfully), or if the logic processing unit 1210 outputs "0", the result of the valid indication update unit 1220 is "0", that is, the indication output is invalid.

[0331] The valid indicator register 1230 is used to store the results output by the valid indicator update unit 1220.

[0332] The result register 1240 is used to store the calculation results of the group when the output of the logic processing unit 1210 is "1".

[0333] The filter unit 1200 can also acquire signals F and L. Signals F and L can be stored in registers. The initial value of signal F is 1, and the initial value of signal L is 0.

[0334] After inputting the key-value pair set of point A into the processing matrix 810 from the left and the key-value pair set of point B into the processing matrix 810 from the top, the relationship between the maximum key value in the key-value pair set of point A and the maximum key value in the key-value pair set of point B can be determined.

[0335] For example, the input period of a key-value pair set can be equal to v times the period of the input processing matrix of each key-value pair in each key-value pair set. If the maximum key value corresponding to point A is greater than the maximum key value corresponding to point B, signal F is set to "0" in the next key-value pair set input period, and signal L is immediately set to "0"; conversely, if the maximum key value corresponding to point A is greater than the maximum key value corresponding to point B, signal F is set to "1" in the next key-value pair set input period, and signal L is immediately set to "1".

[0336] Alternatively, if the maximum key value corresponding to point A is greater than the maximum key value corresponding to point B, signal F is set to "0" for the next key-value pair set input period, and signal L is immediately set to "0"; if the maximum key value corresponding to point A is equal to the maximum key value corresponding to point B, signal F is set to "1" for the next key-value pair set input period, and signal L is immediately set to "0"; if the maximum key value corresponding to point A is less than the maximum key value corresponding to point B, signal F is set to "1" for the next key-value pair set input period, and signal L is immediately set to "1".

[0337] The set of key-value pairs for point A in the input processing matrix 810, which has a value of "0", is repeatedly input. The valid indication update unit 1220 can update the valid indication 3 based on the output of the logic processing unit 1210, the valid indication 2 output by the last processing unit 1000 in the row of the processing matrix 810 corresponding to the filtering unit 1200, and the result stored in the valid indication register 1230.

[0338] Based on the output of logic processing unit 1210 and the valid indication 2 output by the last processing unit 1000 in the row of processing matrix 810 corresponding to filtering unit 1200, a new valid indication result can be determined. If both the output of logic processing unit 1210 and the valid indication 2 output by the last processing unit 1000 in the row of processing matrix 810 corresponding to filtering unit 1200 are "1", the new valid indication result can be valid, i.e., valid indication 3 indicates valid. Conversely, if any one of the output of logic processing unit 1210 or the valid indication 2 output by the last processing unit 1000 in the row of processing matrix 810 corresponding to filtering unit 1200 is "0", the new valid indication result can be invalid, i.e., valid indication 3 indicates invalid.

[0339] The results stored in the valid indicator register 1230 can be understood as historical valid indicator results.

[0340] If either the new valid indication result or the historical valid indication result is valid, the output of the valid indication update unit 1220 can be "1", that is, the indication is valid. In other words, a bitwise AND operation can be performed between the new valid indication result and the historical valid indication result.

[0341] The valid indicator register 1230 is used to store the output of the valid indicator update unit 1220, and to update the output of the valid indicator register 1230 (i.e., valid indicator 3).

[0342] When the output of the logic processing unit 1210 is "1", the filtering unit 1200 writes the calculation result of the new group into the result register 1240.

[0343] When signal L is set to "1", the set of key-value pairs for point A in the input processing matrix 810 is the last input processing matrix 810. Therefore, when signal L is set to "1", the result register 1240 can output the stored data after the filtering unit 1200 has completed updating the calculation results of the group stored in the result register 1240.

[0344] For example, when signal L is set to "1", if the valid indication result stored in valid indication register 1230 is "1", then result register 1240 outputs the stored data. Conversely, if the valid indication result stored in valid indication register 1230 is "0", then result register 1240 may not output data.

[0345] Afterwards, the result register 1240 and the valid indicator register 1230 can be cleared of the stored data.

[0346] In the next key-value pair input cycle, if signal F is "1", then the key-value pair set of point A in the input processing matrix 810 in that cycle is a non-repeating input. The result register 1240 can be initialized when signal F is "1".

[0347] In some embodiments, initialization may involve clearing the data stored in the result register 1240.

[0348] In other embodiments, initialization may involve writing the group of key-value pairs at point A in the row input of the processing matrix corresponding to the filtering unit 1200 into the result register 1240. Thus, when the logic processing unit 1210 outputs "1", the calculation result of the new group can be ANDed with the data in the result register 1240 to update the calculation result of the group.

[0349] When signal L is set to "1", the set of key-value pairs of point A in the input processing matrix 810 needs to be input into the processing matrix 810 again. The output of the filtering unit 1200 is invalid.

[0350] The data processing device 800 may also include a controller (not shown). The controller is used to set signals F and L.

[0351] The compression triangle 830 includes v×(v+1) / 2 compression units (CUs) forming a right triangle, used to compress the data output by the filter array 820.

[0352] In the compression triangle 830, there is one compression unit in the first row, and the number of compression units in each row from top to bottom gradually decreases, with the number of compression units between adjacent rows differing by one.

[0353] like Figure 12 As shown, each row of the compression triangle 830 corresponds to a filter unit in the filter matrix 820.

[0354] For each row of the compression triangle 830, if the corresponding filter unit does not output valid data, each compression unit in that row of the compression triangle 830 receives data from the compression unit above it; if the corresponding filter unit outputs valid data, the leftmost compression unit in that row of the compression triangle 830 receives the key value output by the filter unit that points A and B are equal, as well as the calculation result of the group corresponding to the key value, and the other compression units receive data from the compression unit to the left of that compression unit.

[0355] When the valid indicator 3 output by the filter unit is "1", it can be understood that the filter unit outputs valid data.

[0356] Figure 13 This is a schematic structural diagram of a compression unit provided in an embodiment of this application.

[0357] The compression unit 1400 includes interface 1401, interface 1402, interface 1403, and register 1404.

[0358] Interface 1401 is used to connect to the compression unit above compression unit 1400. Interface 1402 is used to connect to the compression unit to the upper left of compression unit 1400. Interface 1403 is used to receive the valid indication 3 output by the filter unit corresponding to the row where compression unit 1400 is located.

[0359] When valid indicator 3 is "1", interface 1401 stores the data in the compression unit above compression unit 1400 in register 1404. When valid indicator 3 is "0", interface 1402 stores the data in the compression unit above the left of compression unit 1400 in register 1404.

[0360] For example, the first, third, and fourth filter units in the filter matrix 820 output valid data. The valid data output by the first filter unit includes the calculation results of the key value 1 and the group corresponding to the key value 1. The valid data output by the third filter unit includes the calculation results of the key value 3 and the group corresponding to the key value 3. The valid data output by the fourth filter unit includes the calculation results of the key value 9 and the group corresponding to the key value 9.

[0361] Rows 1, 3, and 4 of the compression triangle 830 correspond to the 1st, 3rd, and 4th filter units, respectively. The number of compression units in rows 1, 2, 3, and 4 of the compression triangle 830 are 1, 2, 3, and 4, respectively.

[0362] The first filtering unit outputs valid data. The compression unit in the first row of the compression triangle 830 receives the valid data output by the first filtering unit.

[0363] The second filter unit did not output any valid data. Each compression unit in the second row of compression triangle 830 obtains data from the compression unit in the first row of compression triangle 830 located above the filter unit. The first compression unit in the second row of compression triangle 830 obtains data from the compression units in the first row; there are no compression units above the first compression unit in the second row, so no data is obtained.

[0364] The third filter unit outputs valid data. The first compression unit in the third row (i.e., the leftmost unit in the third row) of compression triangle 830 receives the valid data output by the third filter unit. The other compression units in the third row of compression triangle 830 obtain data from the compression unit located to the upper left of the filter unit in the second row of compression triangle 830. That is, the second and third compression units in the third row of compression triangle 830 obtain data from the two compression units in the second row, respectively.

[0365] The fourth filter unit outputs valid data. The first compression unit in the fourth row of compression triangle 830 receives the valid data output by the fourth filter unit. The other compression units in the fourth row of compression triangle 830 obtain data from the compression unit located to the upper left of the filter unit in the second row of compression triangle 830. That is, the second to fourth compression units in the fourth row of compression triangle 830 obtain data from the three compression units in the third row, respectively.

[0366] In this way, the compression unit in the bottom row of the compression triangle 830 can output the acquired data, and the output result is the compressed data.

[0367] like Figure 14 As shown in the processing matrix 810, for any row, receiving data from the processing unit in column 1 (i.e., receiving the key-value pair of points A and B) to receiving data from the processing unit in column v (the received data can be the key-value pair of points A and B, or the calculation result of the key-value pair and the corresponding group) takes v clock cycles (CCs). Similarly, receiving data from the processing unit in row 1, column 1 to the processing unit in row v, column 1 also takes v clock cycles.

[0368] In the filtering matrix 820, each filtering unit uses one clock cycle to filter the data output from the row of the processing matrix 810 corresponding to that filtering unit.

[0369] In the compression triangle 830, it takes v clock cycles to receive data from the first row and output data from the last row, and it takes 1 clock cycle to receive data from the last row and output data from the last row.

[0370] Therefore, the compression triangle 830 can output the data corresponding to each row in the same clock cycle.

[0371] The data processing device 800 can be implemented based on a dual inline memory module (DIMM), for example, it can be set in a low-load dual inline memory module (LRDIMM) to form a near-memory computing architecture.

[0372] The data processing apparatus 800 may further include a counting unit. The counting unit is used to count the subgraphs that meet the requirements.

[0373] In other words, the counting unit can record the number of key values ​​output by the compression triangle 830. Furthermore, the counting unit can accumulate counts for different input data.

[0374] A counting unit may include an adder and a register.

[0375] Figure 15 This is a schematic structural diagram of a data processing system provided in an embodiment of this application.

[0376] The data processing system 1600 includes a controller 1610 and a processing matrix 1620.

[0377] The processing matrix 1620 consists of v×v processing units, where v is a positive integer greater than 1.

[0378] The controller 1610 is configured to sequentially input at least one key-value pair of the first target point of the two target points in the relationship graph into a plurality of processing units located at the first edge of the processing matrix 1620 along a second direction according to the input cycle, and to sequentially input at least one key-value pair of the second target point of the two target points into a plurality of processing units located at the second edge of the processing matrix 1620 along a first direction according to the input cycle.

[0379] The input period can be a clock period or a positive integer multiple of a clock period.

[0380] At least one key-value pair of the first target point and at least one key-value pair of the second target point are input into the processing matrix at the beginning of the same input cycle.

[0381] The first edge and the second edge are adjacent, the first direction is the direction away from the second edge, and the second direction is the direction away from the first edge. The first edge and the second edge can be understood as two adjacent edges of the processing matrix 1620.

[0382] Each key-value pair of the target point includes a key value of the target point and a relation value group of the target point corresponding to the key value. Different key values ​​correspond to different point sets in the relation graph. The relation value group of the target point corresponding to the key value is used to indicate whether the target point has a relationship with each point in the point set corresponding to the key value. In each key-value pair of the target point, there are points in the point set corresponding to the key value that have a relationship with the target point.

[0383] Each of the plurality of processing units is configured to: determine whether the key value of the first target point input to the processing unit is equal to the key value of the second target point.

[0384] Each processing unit is further configured to: when the key value of the first target point is equal to the key value of the second target point, determine a query point in the relationship graph based on the relationship value group of the first target point and the relationship value group of the second target point input to the processing unit, wherein the relationship between the query point and the two target points conforms to a preset condition.

[0385] Each processing unit is further configured to: transmit the key-value pair of the first target point to the next processing unit along the first direction according to the input cycle, and transmit the key-value pair of the second target point to the next processing unit along the second direction, wherein the first direction is perpendicular to the first edge and the second direction is perpendicular to the second edge.

[0386] The processing matrix 1620 can be understood as a systolic array architecture. In a systolic array architecture, data flows rhythmically between the processing units of the array in a predetermined "pipeline" manner. During the data flow, all processing units process the data flowing through them simultaneously and in parallel, thus achieving a very high parallel processing speed.

[0387] Optionally, each of the plurality of processing units is specifically configured to: transmit the key-value pair of the first target point to the next processing unit along the first direction, and transmit the key-value pair of the second target point to the next processing unit along the second direction, when the key value of the first target point is not equal to the key value of the second target point.

[0388] The key values ​​in each key-value pair of a target point are different. If the key value of the first target point is not equal to the key value of the second target point, the processing unit transmits the key values ​​of both the first and second target points. However, if the key value of the first target point is equal to the key value of the second target point, the key-value pair is not transmitted. Therefore, the equal key value does not need to be compared with other key values, reducing the computational load.

[0389] Optionally, the controller 1610 is further configured to determine at least one key-value pair of the first target point and at least one key-value pair of the second target point based on the relationship graph data. The relationship graph data includes a row offset vector, a key value vector, and a relationship value vector. The key value vector includes at least one key value of each point among a plurality of points in the relationship graph. The row offset vector is used to indicate the position of at least one key value of each point in the key value vector. The relationship value vector includes a group of relationship values ​​of points corresponding to the key values ​​of each point among the plurality of points. The order of at least one key value of each point in the key value vector is the same as the order of the group of relationship values ​​of points corresponding to the key values ​​of each point in the relationship value vector.

[0390] Optionally, the number of points in the point set corresponding to different key values ​​is equal.

[0391] Optionally, each of the plurality of processing units is configured to: when the key value of the first target point is equal to the key value of the second target point, perform a preset operation on each bit of the relation value group of the first target point and the relation value group of the second target point corresponding to the equal key value, wherein the same bit in the relation value group of different points corresponding to each key value corresponds to the same point in the point set corresponding to the key value, and the result of the preset operation of each bit is used to indicate whether the relationship between the point corresponding to the bit and the two target points conforms to the preset condition.

[0392] Optionally, if the number of at least one key-value pairs of the first target point is greater than v, the controller 1610 is further configured to,

[0393] According to the order of the key values ​​in the key-value pairs, at least one key-value pair of the first target point is divided into multiple first key-value pair groups, and at least one key-value pair of the second target point is divided into at least one second key-value pair group. The number of key-value pairs in each of the multiple first key-value pair groups and the at least one second key-value pair group is less than or equal to v. The key-value pair group with the smallest key value among the multiple first key-value pair groups is the third key-value pair group, and the key-value pair group with the smallest key value among the multiple second key-value pair groups is the fourth key-value pair group.

[0394] The controller 1610 is also configured to perform multiple iterations until the maximum key value in the third key-value pair group is greater than the maximum key value in the fourth key-value pair group.

[0395] Each iteration includes: sequentially inputting multiple key-value pairs from the third key-value pair group into multiple processing units located at the first edge of the plurality of processing units according to the input cycle; sequentially inputting multiple key-value pairs from the fourth key-value pair group into multiple processing units located at the first edge of the plurality of processing units according to the input cycle; if the largest key value in the third key-value pair group is less than the largest key value in the fourth key-value pair group, the next first key-value pair group of the third key-value pair group among the plurality of first key-value pair groups arranged in ascending order of key value is taken as the third key-value pair group.

[0396] Specifically, the controller is used to input multiple key-value pairs of the third key-value pair group into multiple processing units located at the first edge of the processing matrix according to the input period before and after the multiple iterations, and to input multiple key-value pairs of the fourth key-value pair group into multiple processing units located at the second edge of the processing matrix according to the input period.

[0397] If the number of key-value pairs at a certain target point is greater than the number of key-value pairs on an edge of the processing matrix, the key-value pairs at that target point can be grouped according to the magnitude of their key values. Then, the group of key-value pairs with the smallest key values ​​can be selected and input into the processing matrix in ascending order of key values. If the maximum key value in the key-value pair group input into the processing matrix for a certain target point is less than the maximum key value in the key-value pair group input into the processing matrix for another target point, the next key-value pair group for that target point is input into the processing matrix and compared with the key-value pair groups input into the processing matrix for the other target point.

[0398] Therefore, the computational load is reduced when the number of key-value pairs at the target point is greater than the number of key-value pairs at an edge in the processing matrix.

[0399] For details on the structure of the processing matrix 1620 in the data processing system 1600, please refer to [link / reference needed]. Figures 7 to 9 The device 1600 may also include... (The sentence is incomplete and requires further context to be fully translated.) Figure 7 The filter array 820 and compression triangle 830 are shown.

[0400] Figure 16 This is a schematic structural diagram of a data processing system provided in an embodiment of this application.

[0401] The data processing system 1700 includes a device 800 and a memory 1710. The memory is used to store the relationship diagram data and the processing results obtained by the device 800 from processing the relationship diagram data.

[0402] Device 800 may be an ASIC. Memory 1710 can be understood as off-chip memory of device 800. For example, memory 1710 may be off-chip RAM of device 800.

[0403] Therefore, the data processing system 1700 can be understood as a computing system with storage and computation separation.

[0404] Figure 17 This is a schematic structural diagram of a data processing system provided in an embodiment of this application.

[0405] The data processing system 1800 includes an address indexing module 1810, two rank-level near memory computing (NMC) units 1820, and a storage module 1830.

[0406] The storage module 1830 may include multiple dynamic random access memory (DRAM) chips 1831.

[0407] When using 8-byte (x8) DRAM chips, the number of x8 DRAM chips in the storage module 1830 can be 8, with half a rank providing data (32 bytes) for one NMC. That is, the 64 bytes of data for one rank are split into two 32-byte blocks, each providing data for one NMC.

[0408] Address index module 1810 can be a registering clock driver (RCD).

[0409] The storage module 1830 may include a KVP data area, a KVP address index area, and a result data area.

[0410] The KVP data area is used to store the KVPs of each point in the relationship graph. The KVPs of each point in the relationship graph can be determined based on the relationship graph data. The relationship graph data includes row offset vectors, key value vectors, and relationship value vectors.

[0411] For example, the key-value pairs (KVPs) for each point can be stored as 32 bits, where the key value can be located in the lower bits of the 32 bits, and the corresponding relation value group can be located in the higher bits of the 32 bits. 32 bits is equivalent to 4 bytes, and an x8 DRAM chip can output key-value pairs for two nodes simultaneously.

[0412] The KVP address index area is used to store row offsets (row data). The row offsets stored in the KVP address index area can be row offset vectors in the relationship graph data. That is, each number in the row offset exists in the KVP address index area in the form of an index address. Each point in the relationship graph corresponds to an offset information in the row offset. Based on the offset information of each point, the index address of the KVP at that point in the KVP data area can be determined.

[0413] The result data area is used to store intermediate calculation results, final calculation results, etc. of the NMC 1820. For example, the result data area may include registers 1230 and result registers 1240 that effectively indicate each filtering unit 1200, registers 1404 of each compression unit 1400, etc.

[0414] In the case of a query point that needs to determine the relationship between two target points in the relationship graph, the address index module 1810 is used to determine the index address of the KVP of the target point based on the ordinal number of each target point in the row offset.

[0415] The numbers in the row offset can be stored contiguously. Each number in the row offset corresponds to a point in the relational graph. The address indexing module 1810 can determine the index address of the point based on the base address + the target point's sequence number × the offset.

[0416] The index address can include the chip select (CS) signal and the command address (C / A) signal.

[0417] The NMC 1820 can determine the location of the KVP stored in the KVP data area based on the index address. The NMC 1820 can execute methods 500 and 700. The NMC 1820 may include a data processing device 800, or the NMC 1820 may be a data processing system 1600. The NMC 1820 may be referred to as DIMMining.

[0418] The data processing system 1800 can be an LRDIMM.

[0419] The data processing system 1800 is implemented without changing the traditional storage function of LRDIMM or modifying the internal circuitry of the dynamic random access memory (DRAM) chip.

[0420] Since multiple ranks and different NMCs within the same rank operate in complete parallelism without occupying external bandwidth, the effective bandwidth utilization of the data processing system 1800 can theoretically reach twice the number of ranks in a single channel.

[0421] For example, the NMC 1820 includes a controller, a data forwarding unit, and a data processing device 800.

[0422] Storage module 1830 may also include data buffer (DB), cache, etc.

[0423] For example, the key-value pairs (KVPs) of each point in the relationship graph can be stored in a DRAM chip, and during the calculation process, the KVPs of each point in the relationship graph can be loaded into the database.

[0424] Due to the irregularity of the relationship graph, data corresponding to some points will be frequently accessed. Therefore, the key-value pairs (KVPs) of frequently accessed points can be stored in a cache. For example, when the access frequency of a point's KVP is higher than a first preset value, the KVP of that point can be stored in the cache; when the access frequency of a point's KVP is lower than a second preset value, the KVP of that point can be deleted from the cache. The second preset value can be less than or equal to the first preset value. The access frequency of a point's KVP can be determined based on the number of accesses within a certain period. The length of this period can be preset.

[0425] The data forwarding unit is used to acquire data from the DRAM chip, database, or cache and input the data into the processing array 810 of the data processing device 800.

[0426] The data processing system 1800 provided in this application embodiment is based on a DIMM-based near-memory graph mining computing architecture, which can achieve parallel computing at the memory rank level. Multiple DIMM ranks can perform readings and calculations in parallel, improving computational efficiency. Furthermore, compared to the data processing system 1700, the data processing system 1800 avoids frequent data transfers between the CPU and memory, and multiple ranks do not need to compete for memory bus access. Therefore, the near-memory computing architecture can achieve a significant performance improvement compared to the traditional CPU + memory storage-computation separation architecture.

[0427] Figure 18 This is a schematic diagram comparing the performance of the data processing systems provided in the embodiments of this application.

[0428] To identify subgraphs in a relational graph whose models are clique finding (CF), 4-node CF, 5-node CF, and motif counting (MF), respectively, key-value pairs at each node in the relational graph are processed using System 1700 and System 1800. The total time required for processing using System 1700 is defined as 1, and the total time required for processing using System 1800 is as follows: Figure 18 As shown. Relationship Figure 1 and relationships Figure 2 For different relational graph data.

[0429] It can be seen that the time required for DRAM access, cache access, and computation is almost the same for both System 1700 and System 1800 during operation. Compared to System 1700, System 1800 avoids frequent data transfers between the CPU and memory, reducing communication time and achieving a significant performance improvement.

[0430] Figure 20 This is a schematic structural diagram of a control device for a data processing apparatus provided in an embodiment of this application.

[0431] The control device 2000 of the data processing apparatus may include an acquisition module 2010 and a processing module 2020.

[0432] The acquisition module 2010 is used to acquire a first target data group and a second target data group. The first target data group is the first data group among a plurality of first data groups in a first data set, and the second target data group is the first data group among at least one second data group in a second data set. Each data group in each data set of the first data set and the second data set includes at least one data item, and each data item includes a key value. The data groups in each data set are arranged in a first order or a second order. When arranged in the first order, each key value in any data group in each data set is less than each key value in the data group following the first data group. When arranged in the second order, each key value in any data group in each data set is greater than each key value in the data group following the first data group.

[0433] The processing module 2020 is used for multiple iterations.

[0434] Each iteration includes inputting the first target data set and the second target data set into the data processing device, the data processing device being used to determine the key values ​​that are equal in the first target data set and the second target data set.

[0435] Each iteration further includes, in the case that the data groups in each data set are arranged in the first order and the first key value is less than or equal to the second key value, or in the case that the data groups in each data set are arranged in the second order and the third key value is greater than or equal to the fourth key value, obtaining the first data group in the first data set that is located after the first target data group as the first target data group, wherein the first key value is the largest key value in the first target data group, the second key value is the largest key value in the second target data group, the third key value is the smallest key value in the first target data group, and the fourth key value is the smallest key value in the second target data group.

[0436] Optionally, the number of at least one second data group may be multiple.

[0437] If the data groups in each data set are arranged in the first order and the first key value is greater than or equal to the second key value, or if the data groups in each data set are arranged in the second order and the third key value is less than or equal to the fourth key value, then the second data group located after the second target data group in the second data set is obtained as the second target data group.

[0438] Optionally, the data processing device includes a processing matrix comprising v×v processing units, where v is a positive integer, and the number of at least one data in each of the first and second target data groups is less than or equal to v.

[0439] The step of inputting the first target data group and the second target data group into the data processing device includes: inputting the first target data group and the second target data group into a processing matrix according to an input rule, wherein the input rule is such that at least one data in the second target data group is sequentially input into the processing unit located at the second edge along a first direction according to an input cycle, at least one data in the first target data group is sequentially input into the processing unit located at the first edge along a second direction according to the input cycle, the first target data group and the second data group are input at the beginning of the same input cycle, the first edge and the second edge are adjacent, the first direction is a direction away from the second edge, and the second direction is a direction away from the first edge.

[0440] Each processing unit in the processing matrix is ​​used to determine whether the key value in the first data input to the processing unit in the same input cycle is equal to the key value in the second data, wherein the first data is the data belonging to the first target data group and the second data is the data belonging to the second target data group.

[0441] When v is greater than 1, each processing unit in the processing matrix is ​​further configured to, in the next input cycle after receiving the first data and the second data, transmit the first data to the next processing unit along the first direction and transmit the second data to the next processing unit along the second direction.

[0442] Optionally, the key values ​​in different data in each dataset are different, and each processing unit in the processing matrix is ​​used to transmit the first data to the next processing unit along the first direction and transmit the second data to the next processing unit along the second direction when the key value in the first data is not equal to the key value in the second data.

[0443] Optionally, the data processing device further includes a filtering matrix comprising v filtering units, wherein the v filtering units are located after the last processing unit in each of the v rows of the processing matrix along the first direction along the second direction.

[0444] Each processing unit in the processing matrix is ​​further configured to, in the next input cycle after receiving the first data and the second data, transmit the processing result of the processing unit along the second direction to the next unit, wherein the unit is the processing unit or the filtering unit, and the processing result includes the equal key values, when the key value in the first data is equal to the key value in the second data; or, transmit the processing result along the second direction to the next unit in the next input cycle after receiving the processing result.

[0445] The processing module 2020 is further configured to, when the first key value is greater than or equal to the second key value, control the v filtering units along the first direction to sequentially output the processing result corresponding to the second target data group according to the input cycle.

[0446] Optionally, the data processing device further includes a compression triangular matrix, the compression triangular matrix including v rows of compression units along the first direction, the number of compression units increasing row by row along the first direction.

[0447] Each of the plurality of compression units is configured to receive the processing result output by the filtering unit preceding the compression unit along the second direction, or to receive the processing result output by the compression unit in a row along the first direction.

[0448] Each of the plurality of compression units is further configured to, in the next input cycle after receiving the processing result, transmit the processing result to the compression unit in the next row along the first direction.

[0449] Optionally, different key values ​​correspond to different point sets in the relationship graph. The first data is used to indicate whether there is a relationship between the first target point in the relationship graph and at least one point in the point set corresponding to the key value in the first data. The second data is used to indicate whether there is a relationship between the second target point in the relationship graph and at least one point in the point set corresponding to the key value in the second data.

[0450] Each processing matrix is ​​also used to output a processing result when the key value in the first data is equal to the key value in the second data. The processing result is used to indicate the query point in the relationship graph, and the relationship between the query point and the two target points conforms to a preset condition.

[0451] Optionally, the first data further includes a first relation value group of the first target point corresponding to the key value, and the second data further includes a second relation value group of the second target point corresponding to the key value.

[0452] Each processing unit in the processing matrix is ​​further configured to perform a preset operation on each bit of the first relation value group and the second relation value group when the key value in the first data is equal to the key value in the second data. The same bit in the first relation value group and the second relation value group corresponding to the equal key value corresponds to the same point in the point set corresponding to the equal key value. The result of the preset operation on each bit is used to indicate whether the relationship between the point corresponding to the bit and the two target points conforms to the preset condition.

[0453] Optionally, when the data groups in each data set are arranged in the first order, the key values ​​in each first data group are arranged from smallest to largest; when the data groups in each data set are arranged in the second order, the key values ​​in each first data group are arranged from largest to smallest.

[0454] In other embodiments, the device 2000 may also be a data processing device.

[0455] The acquisition module 2010 is used to acquire at least one key-value pair of a first target point and at least one key-value pair of a second target point in a relationship graph. Each key-value pair of the target points includes a key value of the target point and a relationship value group of the target point corresponding to the key value. Different key values ​​correspond to different point sets in the relationship graph. The relationship value group of the target point corresponding to the key value of the target point is used to indicate whether the target point has a relationship with each point in the point set corresponding to the key value. In each key-value pair of the target point, there are points in the point set corresponding to the key value that have a relationship with the target point.

[0456] The processing module 2020 is configured to determine at least one key value of the first target point that is equal to at least one key value of the second target point;

[0457] The processing module 2020 is further configured to determine a query point in the relationship graph based on the relationship value group of the first target point corresponding to the equal key value and the relationship value group of the second target point, wherein the relationship between the query point and the two target points conforms to a preset condition.

[0458] Optionally, the acquisition module 2010 is configured to acquire the relationship graph data, the relationship graph data including a row offset vector, a key value vector, and a relationship value vector; and, based on the relationship graph data, determine at least one key-value pair for each of the two target points, the key value vector including at least one key value for each of the multiple points in the relationship graph, the row offset vector indicating the position of at least one key value for each point in the key value vector, and the relationship value vector including the relationship value group of the points corresponding to the key values ​​of each point, wherein the order of at least one key value for each point in the key value vector is the same as the order of the relationship value group of the points corresponding to the key values ​​of each point in the relationship value vector.

[0459] Optionally, the number of points in the point set corresponding to different key values ​​is equal.

[0460] Optionally, the processing module 2020 is further configured to perform preset operations on each bit of the relation value group of the first target point corresponding to the equal key value and the relation value group of the second target point, respectively. The same bit in the relation value group of different points corresponding to each key value corresponds to the same point in the set of points corresponding to the key value. The result of the preset operation of each bit is used to indicate whether the relationship between the point corresponding to the bit and the two target points conforms to the preset condition.

[0461] Optionally, the processing module 2020 includes a controller and a processing matrix.

[0462] The controller is configured to sequentially input at least one key-value pair of the first target point into a plurality of processing units located at the first edge of the processing matrix along a second direction according to the input cycle, and to sequentially input at least one key-value pair of the second target point into a plurality of processing units located at the second edge of the processing matrix along a first direction according to the input cycle, in order to determine the equal key value.

[0463] The processing matrix includes v×v processing units, where v is a positive integer greater than 1. At least one key-value pair of the first target point and at least one key-value pair of the second target point are input into the processing matrix at the beginning of the same input cycle. The first edge and the second edge are adjacent. The first direction is the direction away from the second edge, and the second direction is the direction away from the first edge. Each processing unit is used to determine whether the key value of the first target point input into the processing unit is equal to the key value of the second target point.

[0464] Each of the plurality of processing units is configured to, when the key value of the first target point is equal to the key value of the second target point, determine a query point in the relationship graph based on the relationship value group of the first target point and the relationship value group of the second target point input to the processing unit, wherein the relationship between the query point and the two target points conforms to a preset condition.

[0465] Each of the plurality of processing units is further configured to, according to the input cycle, transmit the key-value pair of the first target point to the next processing unit along the first direction, and transmit the key-value pair of the second target point to the next processing unit along the second direction.

[0466] Optionally, each of the plurality of processing units is specifically configured to: transmit the key-value pair of the first target point to the next processing unit along the first direction, and transmit the key-value pair of the second target point to the next processing unit along the second direction, when the key value of the first target point is not equal to the key value of the second target point.

[0467] Optionally, the number of at least one key-value pair of the first target point is greater than v. The controller is further configured to divide at least one key-value pair of the first target point into a plurality of first key-value pair groups according to the order of the key values ​​in the key-value pairs, and to divide at least one key-value pair of the second target point into at least one second key-value pair group according to the order of the key values ​​in the key-value pairs, wherein the number of key-value pairs in each of the plurality of first key-value pair groups and the at least one second key-value pair group is less than or equal to v, wherein the key-value pair group with the smallest key value among the plurality of first key-value pair groups is the third key-value pair group, and the key-value pair group with the smallest key value among the plurality of second key-value pair groups is the fourth key-value pair group.

[0468] The controller is also configured to perform multiple iterations until the maximum key value in the third key-value pair group is greater than the maximum key value in the fourth key-value pair group.

[0469] The iteration includes: sequentially inputting at least one key-value pair from the third key-value pair group into the plurality of processing units located at the first edge according to the input cycle; sequentially inputting at least one key-value pair from the fourth key-value pair group into the plurality of processing units located at the second edge according to the input cycle; if the largest key value in the third key-value pair group is less than the largest key value in the fourth key-value pair group, the next first key-value pair group of the third key-value pair group among the plurality of first key-value pair groups arranged in ascending order of key value is taken as the third key-value pair group.

[0470] The controller is further configured to, before and after each iteration, sequentially input multiple key-value pairs of the third key-value pair group into multiple processing units located at the first edge of the processing matrix according to the input cycle, and sequentially input multiple key-value pairs of the fourth key-value pair group into multiple processing units located at the second edge of the processing matrix among the multiple processing units according to the input cycle.

[0471] Figure 21 This is a schematic structural diagram of a control device for a data processing apparatus provided in an embodiment of this application.

[0472] The control device 3000 includes a memory 3010 and at least one processor 3020.

[0473] The memory 3010 is used to store program instructions. The processor 3020 is used to execute the program instructions to implement the various steps, methods, operations, or functions performed by the data processing device described above.

[0474] For example, the processor 3020 is configured to acquire a first target data group and a second target data group, wherein the first target data group is the first data group in a first order among a plurality of first data groups in a first data set, and the second target data group is the first data group in at least one second data group in a second data set, wherein each data group in each data set of the first data set and the second data set includes at least one data, each data includes a key value, and the data groups in each data set are arranged in a first order or a second order, wherein in the case of being arranged in the first order, each key value in any data group in each data set is less than each key value in a data group following the first data group, and in the case of being arranged in the second order, each key value in any data group in each data set is greater than each key value in a data group following the second data group.

[0475] The 3020 processor is also used for multiple iterations.

[0476] Each iteration includes inputting the first target data set and the second target data set into the data processing device, the data processing device being used to determine the key values ​​that are equal in the first target data set and the second target data set.

[0477] Each iteration further includes, in the case that the data groups in each data set are arranged in the first order and the first key value is less than or equal to the second key value, or in the case that the data groups in each data set are arranged in the second order and the third key value is greater than or equal to the fourth key value, obtaining the first data group in the first data set that is located after the first target data group as the first target data group, wherein the first key value is the largest key value in the first target data group, the second key value is the largest key value in the second target data group, the third key value is the smallest key value in the first target data group, and the fourth key value is the smallest key value in the second target data group.

[0478] Optionally, the number of at least one second data group may be multiple.

[0479] If the data groups in each data set are arranged in the first order and the first key value is greater than or equal to the second key value, or if the data groups in each data set are arranged in the second order and the third key value is less than or equal to the fourth key value, then the second data group located after the second target data group in the second data set is obtained as the second target data group.

[0480] Optionally, the data processing device includes a processing matrix comprising v×v processing units, where v is a positive integer, and the number of at least one data in each of the first and second target data groups is less than or equal to v.

[0481] The step of inputting the first target data group and the second target data group into the data processing device includes: inputting the first target data group and the second target data group into a processing matrix according to an input rule, wherein the input rule is such that at least one data in the second target data group is sequentially input into the processing unit located at the second edge along a first direction according to an input cycle, at least one data in the first target data group is sequentially input into the processing unit located at the first edge along a second direction according to the input cycle, the first target data group and the second data group are input at the beginning of the same input cycle, the first edge and the second edge are adjacent, the first direction is a direction away from the second edge, and the second direction is a direction away from the first edge.

[0482] Each processing unit in the processing matrix is ​​used to determine whether the key value in the first data input to the processing unit in the same input cycle is equal to the key value in the second data, wherein the first data is the data belonging to the first target data group and the second data is the data belonging to the second target data group.

[0483] When v is greater than 1, each processing unit in the processing matrix is ​​further configured to, in the next input cycle after receiving the first data and the second data, transmit the first data to the next processing unit along the first direction and transmit the second data to the next processing unit along the second direction.

[0484] Optionally, the key values ​​in different data in each dataset are different, and each processing unit in the processing matrix is ​​used to transmit the first data to the next processing unit along the first direction and transmit the second data to the next processing unit along the second direction when the key value in the first data is not equal to the key value in the second data.

[0485] Optionally, the data processing device further includes a filtering matrix comprising v filtering units, wherein the v filtering units are located after the last processing unit in each of the v rows of the processing matrix along the first direction along the second direction.

[0486] Each processing unit in the processing matrix is ​​further configured to, in the next input cycle after receiving the first data and the second data, transmit the processing result of the processing unit along the second direction to the next unit, wherein the unit is the processing unit or the filtering unit, and the processing result includes the equal key values; or, in the next input cycle after receiving the processing result, transmit the processing result along the second direction to the next unit.

[0487] The processor 3020 is also configured to, when the first key value is greater than or equal to the second key value, control the v filtering units along the first direction to sequentially output the processing results corresponding to the second target data group according to the input cycle.

[0488] Optionally, the data processing device further includes a compression triangular matrix, the compression triangular matrix including v rows of compression units along the first direction, the number of compression units increasing row by row along the first direction.

[0489] Each of the plurality of compression units is configured to: receive the processing result output by the filtering unit preceding the compression unit along the second direction, or receive the processing result output by the compression unit along one row along the first direction.

[0490] Each of the plurality of compression units is configured to, in the next input cycle after receiving the processing result, transmit the processing result to the compression unit in the next row along the first direction.

[0491] Optionally, different key values ​​correspond to different point sets in the relationship graph. The first data is used to indicate whether there is a relationship between the first target point in the relationship graph and at least one point in the point set corresponding to the key value in the first data. The second data is used to indicate whether there is a relationship between the second target point in the relationship graph and at least one point in the point set corresponding to the key value in the second data.

[0492] Each processing matrix is ​​also used to output a processing result when the key value in the first data is equal to the key value in the second data. The processing result is used to indicate the query point in the relationship graph, and the relationship between the query point and the two target points conforms to a preset condition.

[0493] Optionally, the first data further includes a first relation value group of the first target point corresponding to the key value, and the second data further includes a second relation value group of the second target point corresponding to the key value.

[0494] Each processing unit in the processing matrix is ​​further configured to perform a preset operation on each bit of the first relation value group and the second relation value group when the key value in the first data is equal to the key value in the second data. The same bit in the first relation value group and the second relation value group corresponding to the equal key value corresponds to the same point in the point set corresponding to the equal key value. The result of the preset operation on each bit is used to indicate whether the relationship between the point corresponding to the bit and the two target points conforms to the preset condition.

[0495] Optionally, when the data groups in each data set are arranged in the first order, the key values ​​in each first data group are arranged from smallest to largest; when the data groups in each data set are arranged in the second order, the key values ​​in each first data group are arranged from largest to smallest.

[0496] Specifically, furthermore, the units in the above devices can be integrated in whole or in part, or they can be implemented independently. In one implementation, these units are integrated together as a system-on-a-chip (SOC). The SOC may include at least one processor for implementing any of the above methods or implementing the functions of the units in the device. The at least one processor may be of different types, such as CPU and FPGA, CPU and artificial intelligence processor, CPU and graphics processing unit (GPU), etc.

[0497] This application also provides a computer program storage medium having program instructions that, when executed, cause the method described above to be performed.

[0498] This application also provides a chip system including at least one processor, wherein when program instructions are executed in the at least one processor, the method described above is executed.

[0499] This application also provides a program product, the computer program product including program instructions, which, when executed in a computer device, cause the aforementioned data processing method to be executed.

[0500] This application also provides a data processing system, including the data processing device and the control device of the data processing device described above.

[0501] It should be understood that the processor in the embodiments of this application can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0502] It should also be understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0503] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0504] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0505] In this application, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0506] In this application, prefixes such as "first" and "second" are used merely to distinguish different descriptive objects and do not limit the position, order, priority, quantity, or content of the described objects. For example, if the described object is a "key value," the ordinal numbers before "key value" in "first key value" and "second key value" do not restrict the position, order, or priority of "interfaces." Similarly, if the described object is a "direction," the ordinal numbers before "direction" in "first direction" and "second direction" do not restrict the position, order, or priority of "directions."

[0507] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0508] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0509] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0510] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0511] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0512] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0513] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0514] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A control method for a data processing device, characterized in that, The method includes: Acquire a first target data group and a second target data group, wherein the first target data group is the first data group among a plurality of first data groups in a first data set, and the second target data group is the first data group among at least one second data group in a second data set. Each data group in each data set of the first data set and the second data set includes at least one data, and each data includes a key value, wherein the key value indicates one or more elements in the corresponding data set. The data groups in each data set are arranged in a first order or a second order. In the case of being arranged in the first order, each key value in any data group in each data set is less than each key value in the data group following the first data group. In the case of being arranged in the second order, each key value in any data group in each data set is greater than each key value in the data group following the first data group. Perform multiple iterations, each iteration including: The first target data group and the second target data group are input into the data processing device, which is used to determine the key values ​​that are equal in the first target data group and the second target data group; In each data set, if the data groups are arranged in the first order and the first key value is less than or equal to the second key value, or if the data groups are arranged in the second order and the third key value is greater than or equal to the fourth key value, the first data group in the first data set that is located after the first target data group is selected as the first target data group. The first key value is the largest key value in the first target data group, the second key value is the largest key value in the second target data group, the third key value is the smallest key value in the first target data group, and the fourth key value is the smallest key value in the second target data group.

2. The method according to claim 1, characterized in that, At least one second data group may contain multiple data groups. If the data groups in each data set are arranged in the first order and the first key value is greater than or equal to the second key value, or if the data groups in each data set are arranged in the second order and the third key value is less than or equal to the fourth key value, then the second data group located after the second target data group in the second data set is obtained as the second target data group.

3. The method according to claim 1 or 2, characterized in that, The data processing device includes a processing matrix comprising v × v processing units, where v is a positive integer, and the number of at least one data point in each target data group (first target data group and second target data group) is less than or equal to v. The i-th first data in the first target data group is input to the j-th processing unit along the second direction among the v processing units located at the first edge in the j-th input cycle of the iteration. The p-th second data in the second target data is input to the q-th processing unit along the first direction among the v processing units located at the second edge in the q-th input cycle of the iteration. The first edge and the second edge are adjacent. The processing units for different data inputs in each target data group are different. The first direction is the direction from the second edge to the interior of the processing matrix and perpendicular to the second edge. The second direction is the direction from the first edge to the interior of the processing matrix and perpendicular to the first edge. i, j, p, and q are all positive integers. Each processing unit in the processing matrix is ​​used to determine whether the key value in the first data input to the processing unit in the same input cycle is equal to the key value in the second data. When v is greater than 1, each processing unit in the processing matrix is ​​further configured to, in the next input cycle after receiving the first data and the second data, transmit the first data to the next processing unit along the first direction and transmit the second data to the next processing unit along the second direction.

4. The method according to claim 3, characterized in that, The key values ​​in different data in each dataset are different. Each processing unit in the processing matrix is ​​used to transmit the first data to the next processing unit along the first direction and transmit the second data to the next processing unit along the second direction when the key value in the first data is not equal to the key value in the second data.

5. The method according to claim 3 or 4, characterized in that, The data processing device further includes a filtering matrix comprising v filtering units, each of which is located after the last processing unit in each of v rows of the processing matrix along the first direction along the second direction. Each processing unit in the processing matrix is ​​also used for: If the key value in the first data is equal to the key value in the second data, in the next input cycle after receiving the first data and the second data, the processing result of the processing unit is transmitted along the second direction to the next unit, which is the processing unit or the filtering unit, and the processing result includes the equal key value; or, In the next input cycle after receiving the processing result, the processing result is transmitted to the next unit along the second direction; The method further includes: when the first key value is greater than or equal to the second key value, controlling the v filtering units along the first direction to sequentially output the processing results corresponding to the second target data group according to the input cycle.

6. The method according to claim 5, characterized in that, The data processing device further includes a compression triangular matrix, which comprises v rows of compression units along the first direction, with the number of compression units increasing row by row along the first direction. Each of the plurality of compression units is used for: Receive the processing result output by the filtering unit preceding the compression unit along the second direction, or receive the processing result output by the compression unit along one row along the first direction; In the next input cycle after receiving the processing result, the processing result is transmitted to the compression unit in the next row along the first direction.

7. The method according to any one of claims 3-6, characterized in that, Different key values ​​correspond to different sets of points in the relationship graph. The first data is used to indicate whether there is a relationship between a first target point in the relationship graph and at least one point in the set of points corresponding to the key values ​​in the first data. The second data is used to indicate whether there is a relationship between a second target point in the relationship graph and at least one point in the set of points corresponding to the key values ​​in the second data. Each processing matrix is ​​also used to output a processing result when the key value in the first data is equal to the key value in the second data. The processing result is used to indicate the query point in the relationship graph, and the relationship between the query point and the two target points conforms to a preset condition.

8. The method according to claim 7, characterized in that, The first data also includes a first relation value group for the first target point corresponding to the key value, and the second data also includes a second relation value group for the second target point corresponding to the key value. Each processing unit in the processing matrix is ​​further configured to perform a preset operation on each bit of the first relation value group and the second relation value group when the key value in the first data is equal to the key value in the second data. The same bit in the first relation value group and the second relation value group corresponding to the equal key value corresponds to the same point in the point set corresponding to the equal key value. The result of the preset operation on each bit is used to indicate whether the relationship between the point corresponding to the bit and the two target points conforms to the preset condition.

9. The method according to any one of claims 1-8, characterized in that, When the data groups in each data set are arranged in the first order, the key values ​​in each first data group are arranged from smallest to largest; when the data groups in each data set are arranged in the second order, the key values ​​in each first data group are arranged from largest to smallest.

10. A control device for a data processing apparatus, characterized in that, include: Acquisition module and processing module The acquisition module is used to acquire a first target data group and a second target data group. The first target data group is the first data group among a plurality of first data groups in a first data set. The second target data group is the first data group among at least one second data group in a second data set. Each data group in each data set of the first data set and the second data set includes at least one data item. Each data item includes a key value. The key value indicates one or more elements in the corresponding data set. The data groups in each data set are arranged in a first order or a second order. When arranged in the first order, each key value in any data group in each data set is less than each key value in the data group following the first data group. When arranged in the second order, each key value in any data group in each data set is greater than each key value in the data group following the first data group. The processing module is used to perform multiple iterations, each iteration including: The first target data group and the second target data group are input into the data processing device, which is used to determine the key values ​​that are equal in the first target data group and the second target data group; In each data set, if the data groups are arranged in the first order and the first key value is less than or equal to the second key value, or if the data groups are arranged in the second order and the third key value is greater than or equal to the fourth key value, the first data group in the first data set that is located after the first target data group is selected as the first target data group. The first key value is the largest key value in the first target data group, the second key value is the largest key value in the second target data group, the third key value is the smallest key value in the first target data group, and the fourth key value is the smallest key value in the second target data group.

11. The apparatus according to claim 10, characterized in that, At least one second data group may contain multiple data groups. If the data groups in each data set are arranged in the first order and the first key value is greater than or equal to the second key value, or if the data groups in each data set are arranged in the second order and the third key value is less than or equal to the fourth key value, then the second data group located after the second target data group in the second data set is obtained as the second target data group.

12. The apparatus according to claim 10 or 11, characterized in that, The data processing device includes a processing matrix comprising v × v processing units, where v is a positive integer, and the number of at least one data point in each target data group (first target data group and second target data group) is less than or equal to v. The i-th first data in the first target data group is input to the j-th processing unit along the second direction among the v processing units located at the first edge in the j-th input cycle of the iteration. The p-th second data in the second target data is input to the q-th processing unit along the first direction among the v processing units located at the second edge in the q-th input cycle of the iteration. The first edge and the second edge are adjacent. The processing units for different data inputs in each target data group are different. The first direction is the direction from the second edge to the interior of the processing matrix and perpendicular to the second edge. The second direction is the direction from the first edge to the interior of the processing matrix and perpendicular to the first edge. i, j, p, and q are all positive integers. Each processing unit in the processing matrix is ​​used to determine whether the key value in the first data input to the processing unit in the same input cycle is equal to the key value in the second data. When v is greater than 1, each processing unit in the processing matrix is ​​further configured to, in the next input cycle after receiving the first data and the second data, transmit the first data to the next processing unit along the first direction and transmit the second data to the next processing unit along the second direction.

13. The apparatus according to claim 12, characterized in that, The key values ​​in different data in each dataset are different. Each processing unit in the processing matrix is ​​used to transmit the first data to the next processing unit along the first direction and transmit the second data to the next processing unit along the second direction when the key value in the first data is not equal to the key value in the second data.

14. The apparatus according to claim 12 or 13, characterized in that, The data processing device further includes a filtering matrix comprising v filtering units, each of which is located after the last processing unit in each of v rows of the processing matrix along the first direction along the second direction. Each processing unit in the processing matrix is ​​further configured to: when the key value in the first data is equal to the key value in the second data, in the next input cycle after receiving the first data and the second data, transmit the processing result of the processing unit along the second direction to the next unit, wherein the unit is the processing unit or the filtering unit, and the processing result includes the equal key values; or, in the next input cycle after receiving the processing result, transmit the processing result along the second direction to the next unit. The processing module is further configured to, when the first key value is greater than or equal to the second key value, control the v filtering units along the first direction to sequentially output the processing results corresponding to the second target data group according to the input cycle.

15. The apparatus according to claim 14, characterized in that, The data processing device further includes a compression triangular matrix, which comprises v rows of compression units along the first direction, with the number of compression units increasing row by row along the first direction. Each of the plurality of compression units is used for: Receive the processing result output by the filtering unit preceding the compression unit along the second direction, or receive the processing result output by the compression unit along one row along the first direction; In the next input cycle after receiving the processing result, the processing result is transmitted to the compression unit in the next row along the first direction.

16. The apparatus according to any one of claims 12-15, characterized in that, Different key values ​​correspond to different sets of points in the relationship graph. The first data is used to indicate whether there is a relationship between a first target point in the relationship graph and at least one point in the set of points corresponding to the key values ​​in the first data. The second data is used to indicate whether there is a relationship between a second target point in the relationship graph and at least one point in the set of points corresponding to the key values ​​in the second data. Each processing matrix is ​​also used to output a processing result when the key value in the first data is equal to the key value in the second data. The processing result is used to indicate the query point in the relationship graph, and the relationship between the query point and the two target points conforms to a preset condition.

17. The apparatus according to claim 16, characterized in that, The first data also includes a first relation value group for the first target point corresponding to the key value, and the second data also includes a second relation value group for the second target point corresponding to the key value. Each processing unit in the processing matrix is ​​further configured to perform a preset operation on each bit of the first relation value group and the second relation value group when the key value in the first data is equal to the key value in the second data. The same bit in the first relation value group and the second relation value group corresponding to the equal key value corresponds to the same point in the point set corresponding to the equal key value. The result of the preset operation on each bit is used to indicate whether the relationship between the point corresponding to the bit and the two target points conforms to the preset condition.

18. The apparatus according to any one of claims 10-17, characterized in that, When the data groups in each data set are arranged in the first order, the key values ​​in each first data group are arranged from smallest to largest; when the data groups in each data set are arranged in the second order, the key values ​​in each first data group are arranged from largest to smallest.

19. A control device for a data processing apparatus, characterized in that, It includes a memory and at least one processor, the memory being used to store a program, which, when executed in the at least one processor, is used to perform the method as described in any one of claims 1 to 9.

20. A computer program product, characterized in that, Includes program instructions, when which, when executed, the method as described in any one of claims 1 to 9, is performed.

21. A computer-readable storage medium, characterized in that, The computer-readable medium stores program code for execution by the device, and when the program instructions are executed, the method as described in any one of claims 1 to 9 is performed.

22. A chip, characterized in that, The chip includes at least one processor, which, when program instructions are executed in the at least one processor, causes the method as described in any one of claims 1 to 9 to be performed.

23. A data processing system, characterized in that, The control device of the data processing apparatus included in any one of claims 10-18 and the data processing apparatus.

Images