Data processing method, computing device and electronic device

Abstract

The invention provides a data processing method, computing equipment and electronic equipment, and relates to the technical field of computers.According to the data processing method, target operation results of an operation threshold value are screened, operation results which do not conform to the operation threshold value are eliminated, only the target operation results conforming to the operation threshold value are written into a second storage medium, and the data processing efficiency is improved. And the memory access pressure on the second storage medium is reduced, so that the occupation of the write-in bandwidth of the second storage medium is reduced.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data processing method, a computing device, and an electronic device. Background Art

[0002] With the rise of machine learning, artificial intelligence and other fields, the demand for high-bandwidth applications is increasing. In order to meet the demand for high bandwidth, more and more scenarios are beginning to introduce computing accelerator cards to process data. Computing accelerator cards refer to chips that accelerate data calculations. Computing accelerator cards usually include data processing units suitable for matrix operations and data processing units suitable for scalar operations.

[0003] At present, when performing operations, a data processing unit suitable for matrix operations usually first performs relevant matrix operations on the data to obtain matrix operation results, and then a data processing unit suitable for scalar operations further processes the matrix operation results. Since the storage media that can be accessed by the data processing unit suitable for matrix operations and the data processing unit suitable for scalar operations are different, it is necessary to copy the matrix operation results to the storage medium that can be accessed by the data processing unit suitable for scalar operations, so that the data processing unit suitable for scalar operations can further process the matrix operation results. We know that the amount of data of the matrix operation results obtained after the relevant matrix operation is usually large, so when copying data to the storage medium that can be accessed by the data processing unit suitable for scalar operations, a large amount of write bandwidth will be occupied. Summary of the invention

[0004] The embodiments of the present application provide a data processing method, a computing device, and an electronic device, which reduce the memory access pressure on the second storage medium by writing only the operation results that meet the operation threshold to the second storage medium, thereby reducing the occupation of the write bandwidth of the second storage medium.

[0005] In order to achieve the above objectives, this application adopts the following technical solutions:

[0006] In a first aspect, a data processing method is provided, which is applied to a computing device, wherein the computing device includes a first storage medium and a second storage medium; the method includes: performing a target operation on the operation data in the first storage medium to obtain an operation result; and writing the target operation result that meets the operation threshold in the operation result into the second storage medium. Exemplarily, the first storage medium may be a shared memory or a unified buffer, and the second storage medium may be a high-bandwidth storage or a global memory.

[0007] Exemplarily, the read and write speed of the first storage medium is greater than that of the second storage medium. Before performing target operations on the operation data in the first storage medium and obtaining the operation results, the operation data and the operation threshold stored in the second storage medium are written to the first storage medium. Based on this, the operation efficiency can be improved by performing target operations on the operation data through the first storage medium.

[0008] The solution provided in the first aspect above filters the results of the target operation according to the operation threshold, excludes the operation results that do not meet the operation threshold, and only writes the target operation results that meet the operation threshold to the second storage medium. This can reduce the memory access pressure on the second storage medium while ensuring the accuracy of the results, thereby reducing the occupancy of the write bandwidth of the second storage medium, alleviating competition with other operations that require write bandwidth, and improving end-to-end performance.

[0009] As a possible implementation, the operation data includes a base vector and a query vector; the operation data in the first storage medium is subjected to a target operation to obtain an operation result, including: performing a distance operation on the base vector and the query vector to obtain a distance value between the query vector and the base vector; wherein the operation result includes a distance value. The distance value between the query vector and the base vector is filtered by an operation threshold. Based on this, by excluding the distance values ​​that do not match the operation threshold, the distance values ​​that match the operation threshold are written to the second storage medium, thereby reducing the data transfer and reducing the occupation of the write bandwidth of the second storage medium.

[0010] As a possible implementation method, distance operation is performed on a base vector and a query vector to obtain a distance value between the query vector and the base vector, including: grouping the base vector to obtain multiple vector groups; performing distance operation on the query vector and the base vectors in the multiple vector groups respectively to obtain the distance value between the query vector and the base vectors in the multiple vector groups; performing target operation on the operation data in the first storage medium to obtain the operation result, and also including: determining an index value of the distance value of the base vector in the multiple vector groups, wherein the index value of the vector group is the maximum distance value or the minimum distance value among the distance values ​​of the base vector in the vector group; wherein the operation result also includes the index value of the multiple vector groups.

[0011] Among them, the index value of the vector grouping does not meet the operation threshold, which means that the distance values ​​of the bottom library vectors in the vector grouping do not meet the operation threshold, and the index value of the vector grouping meets the operation threshold, which means that the distance values ​​of the bottom library vectors in the vector grouping meet the operation threshold. Based on this, by determining the index value of the distance value of the bottom library vector in the vector grouping and comparing the index value of the distance value of the bottom library vector in the vector grouping with the operation threshold, the target operation result that meets the operation threshold can be quickly obtained, thereby improving the processing efficiency.

[0012] As a possible implementation, the target operation result is the distance value of the bottom-base vector in the first vector group among the multiple vector groups; wherein the index value of the first vector group satisfies the operation threshold. Based on this, the distance value of the bottom-base vector in the first vector group is written into the second storage medium as the target operation result, which reduces the data transfer and the occupation of the write bandwidth of the second storage medium, can ease the competition with other operations that require write bandwidth, and improve the end-to-end performance.

[0013] As a possible implementation, the method further includes: writing the index values ​​of the plurality of vector groups into a second storage medium. Based on this, the distance values ​​of the bottom library vectors in the first vector group read are subjected to topK operations based on the index values ​​of the plurality of vector groups, thereby accelerating the operation process and improving data processing efficiency.

[0014] As a possible implementation method, the target operation result is the distance value of the base library vector in the vector grouping included in the target grouping set; wherein the target grouping set refers to a grouping set including vector groups whose index values ​​meet the operation threshold, and the grouping set refers to a set of vector groups.

[0015] Exemplarily, the grouping set is obtained by dividing the base library vectors by setting the number of sets, and the base library vectors in each grouping set are grouped according to a set data size to obtain vector groups.

[0016] Exemplarily, when the index values ​​of the vector groups included in the grouping set do not meet the operation threshold, the distance values ​​of the bottom library vectors in the vector groups included in the grouping set are eliminated. When there are vector groups whose index values ​​meet the operation threshold in the vector groups included in the grouping set, the grouping set is determined as the target grouping set, and the distance values ​​of the bottom library vectors in the vector groups included in the target grouping set are written to the second storage medium. Based on this, on the one hand, the writing of data is reduced, and the occupation of the writing bandwidth of the second storage medium is reduced. On the other hand, by writing data in the grouping set, the number of data writing is reduced, which can improve the data processing efficiency.

[0017] As a possible implementation, the method further includes: writing a tag sequence corresponding to the target group set into a second storage medium, the tag sequence being used to mark whether the index value of the vector group in the target group set meets the operation threshold; and / or, writing the index value of the vector group included in the target group set into the second storage medium. Based on this, by writing the tag sequence of the target group set and / or the index value of the vector group included in the target group set into the second storage medium, the distance values ​​can be quickly sorted based on the index value of the vector group and / or the tag sequence, thereby improving data processing efficiency.

[0018] As a possible implementation, the method further includes: performing a topK operation according to the target operation result to obtain a topK operation result; and adjusting the operation threshold according to the topK operation result. Based on this, a topK operation is performed according to the target operation result to obtain a topK operation result, and the operation threshold is adjusted according to the topK operation result, so that a highly adaptable operation threshold can be obtained, and data can be screened out without manual participation in setting the threshold. At the same time, the operation result is fuzzily filtered through the operation threshold, which reduces the complexity of the problem and reduces unnecessary computing time consumption.

[0019] As a possible implementation method, a topK operation is performed according to the target operation result, including: determining the first vector grouping that meets the operation threshold according to the marking sequence; the marking sequence is used to mark whether the index value of the vector grouping meets the operation threshold; the index value of the vector grouping is the maximum distance value or the minimum distance value among the distance values ​​of the bottom-base vectors in the vector grouping; and performing a topK operation according to the distance value of the bottom-base vector in the first vector grouping. Based on this, according to the marking sequence, the first vector grouping whose index value meets the operation threshold in the target grouping set can be obtained, and then only the distance value of the bottom-base vector in the first vector grouping is performed on the topK operation, which reduces the amount of topK operation data and improves data processing efficiency.

[0020] In a second aspect, a computing device is provided, including a first storage medium, a second storage medium, a first data processing unit, and a load / store unit;

[0021] The first data processing unit is used to perform target operation on the operation data in the first storage medium to obtain the operation result; the load / store unit is used to write the target operation result that meets the operation threshold in the operation result to the second storage medium.

[0022] Exemplarily, the computing device may be an AI chip.

[0023] In the solution provided by the second aspect above, the first data processing unit screens the results of the target operation by the operation threshold, excludes the operation results that do not meet the operation threshold, and the load / store unit only writes the target operation results that meet the operation threshold to the second storage medium, thereby reducing the memory access pressure on the second storage medium, thereby reducing the occupation of the write bandwidth of the second storage medium when the first data processing unit is running, which can ease the competition with other operations that require write bandwidth and improve end-to-end performance.

[0024] As a possible implementation, the operation data includes a base vector and a query vector; the first data processing unit is specifically used to: perform distance operation on the base vector and the query vector to obtain a distance value between the query vector and the base vector; wherein the operation result includes the distance value. The distance value between the query vector and the base vector is filtered by the operation threshold. Based on this, by excluding the distance values ​​that do not match the operation threshold, the distance values ​​that match the operation threshold are written to the second storage medium, which reduces the data transfer and reduces the occupation of the write bandwidth of the second storage medium.

[0025] As a possible implementation manner, the first data processing unit is specifically used to: group the bottom-base vectors to obtain multiple vector groups; perform distance operations on the query vector and the bottom-base vectors in the multiple vector groups respectively to obtain distance values ​​between the query vector and the bottom-base vectors in the multiple vector groups; the first data processing unit is also specifically used to: determine an index value of the distance value of the bottom-base vectors in the multiple vector groups, wherein the index value of the vector group is the maximum distance value or the minimum distance value among the distance values ​​of the bottom-base vectors in the vector group; wherein the operation result also includes the index values ​​of the multiple vector groups.

[0026] Among them, the index value of the vector grouping does not meet the operation threshold, which means that the distance values ​​of the bottom library vectors in the vector grouping do not meet the operation threshold, and the index value of the vector grouping meets the operation threshold, which means that the distance values ​​of the bottom library vectors in the vector grouping meet the operation threshold. Based on this, by determining the index value of the distance value of the bottom library vector in the vector grouping and comparing the index value of the distance value of the bottom library vector in the vector grouping with the operation threshold, the target operation result that meets the operation threshold can be quickly obtained, thereby improving the processing efficiency.

[0027] As a possible implementation, the target operation result is the distance value of the bottom-base vector in the first vector group among the multiple vector groups; wherein the index value of the first vector group satisfies the operation threshold. Based on this, the distance value of the bottom-base vector in the first vector group is written into the second storage medium as the target operation result, which reduces the data transfer and the occupation of the write bandwidth of the second storage medium, can ease the competition with other operations that require write bandwidth, and improve the end-to-end performance.

[0028] As a possible implementation, the load / store unit is further specifically used to write the index values ​​of the multiple vector groups into the second storage medium. In this way, the distance values ​​of the bottom library vectors in the read first vector group can be subjected to topK operation based on the index values ​​of the multiple vector groups, thereby accelerating the operation process and improving data processing efficiency.

[0029] As a possible implementation method, the target operation result is the distance value of the base library vector in the vector grouping included in the target grouping set; wherein the target grouping set refers to a grouping set including vector groups whose index values ​​meet the operation threshold, and the grouping set refers to a set of vector groups.

[0030] Exemplarily, the grouping set is obtained by dividing the base library vectors by setting the number of sets, and the base library vectors in each grouping set are grouped according to a set data size to obtain vector groups.

[0031] Exemplarily, when the index values ​​of the vector groups included in the grouping set do not meet the operation threshold, the distance values ​​of the bottom library vectors in the vector groups included in the grouping set are eliminated. When there are vector groups whose index values ​​meet the operation threshold in the vector groups included in the grouping set, the grouping set is determined as the target grouping set, and the distance values ​​of the bottom library vectors in the vector groups included in the target grouping set are written to the second storage medium. In this way, on the one hand, the writing of data is reduced, and the occupation of the writing bandwidth of the second storage medium is reduced. On the other hand, data is written by the method in the grouping set, and the number of data writes is reduced, which can improve data processing efficiency.

[0032] As a possible implementation, the load / store unit is further used to: write a tag sequence corresponding to the target group set into a second storage medium, the tag sequence being used to mark whether the index value of the vector group in the target group set meets the operation threshold; and / or, write the index value of the vector group included in the target group set into the second storage medium. Based on this, by writing the tag sequence of the target group set and / or the index value of the vector group included in the target group set into the second storage medium, the distance values ​​can be quickly sorted based on the index value of the vector group and / or the tag sequence, thereby improving data processing efficiency.

[0033] As a possible implementation, the computing device further includes a second data processing unit, which is used to: perform a topK operation according to a target operation result to obtain a topK operation result; and adjust an operation threshold according to the topK operation result. Based on this, a topK operation is performed according to the target operation result to obtain a topK operation result, and the operation threshold is adjusted according to the topK operation result, so that an operation threshold with high adaptability can be obtained, and data can be screened out without manual participation in setting the threshold. At the same time, the operation result is fuzzily filtered through the operation threshold, which reduces the complexity of the problem and reduces unnecessary computing time consumption.

[0034] As a possible implementation method, the second data processing unit is specifically used to: determine the first vector grouping that meets the operation threshold according to the marking sequence; the marking sequence is used to mark whether the index value of the vector grouping meets the operation threshold, and the index value of the vector grouping is the maximum distance value or the minimum distance value among the distance values ​​of the bottom library vectors in the vector grouping; perform topK operation according to the distance value of the bottom library vector in the first vector grouping. Based on this, according to the marking sequence, the first vector grouping whose index value meets the operation threshold in the target grouping set can be obtained, and then only the distance value of the bottom library vector in the first vector grouping is performed topK operation, which reduces the amount of topK operation data and improves data processing efficiency.

[0035] According to a third aspect, an electronic device is provided, comprising: a memory for storing computer program instructions; and a processor for executing the computer program instructions to support the electronic device to implement a method as in any possible implementation of the first aspect.

[0036] In a fourth aspect, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processing circuit, the method in any possible implementation manner of the first aspect is implemented.

[0037] In a fifth aspect, a chip system is provided, the chip system includes a processing circuit and a storage medium, the storage medium stores computer program instructions; when the computer program instructions are executed by the processing circuit, the method in any possible implementation of the first aspect is implemented. The chip system can be composed of a chip, or it can include a chip and other discrete devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 Schematic diagram of data processing principle of conventional technology.

[0039] Figure 2 This is one of the architectural diagrams of the AI ​​system provided in the embodiments of the present application.

[0040] Figure 3 The second schematic diagram of the architecture of the AI ​​system provided in the embodiment of the present application.

[0041] Figure 4 A schematic diagram of the structure of the AI ​​chip provided in an embodiment of the present application.

[0042] Figure 5 A flowchart of a data processing method provided in an embodiment of the present application.

[0043] Figure 6 This is one of the flowcharts of vector retrieval provided in the embodiments of the present application.

[0044] Figure 7The second flowchart of the vector search provided in the embodiment of the present application.

[0045] Figure 8 This is one of the schematic diagrams of the data processing principles provided in the embodiments of the present application.

[0046] Fig. 9 The third flowchart of the vector search provided in the embodiment of the present application.

[0047] Fig.10 The second schematic diagram of the data processing principle provided in the embodiment of the present application.

[0048] Fig.11 The third schematic diagram of the data processing principle provided in the embodiment of the present application.

[0049] Fig.12 The fourth schematic diagram of the data processing principle provided in the embodiment of the present application.

[0050] Fig.13 The fourth flowchart of the vector search provided in the embodiment of the present application.

[0051] Fig.14 The fifth schematic diagram of the data processing principle provided in the embodiment of the present application.

[0052] Fig.15 The sixth schematic diagram of the data processing principle provided for the embodiment of the present application.

[0053] Fig.16 A schematic diagram of the structure of a computing device provided in an embodiment of the present application.

[0054] Fig.17 A schematic diagram of the working principle of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0055] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0056] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this embodiment, unless otherwise specified, "plurality" means two or more.

[0057] As described in the background technology, since the storage media accessed by the data processing unit applicable to matrix operations and the data processing unit applicable to scalar operations are different, the matrix operation result needs to be copied to the storage medium accessible to the data processing unit applicable to scalar operations, so that the data processing unit applicable to scalar operations can further process the matrix operation result. The data volume of the matrix operation result is usually large, so a large amount of write bandwidth will be occupied when copying data to the storage medium accessible to the data processing unit applicable to scalar operations.

[0058] In particular, for the field of vector retrieval technology, there may be a technical problem that the write bandwidth is seriously occupied. Among them, vector retrieval technology is widely used in Internet and artificial intelligence related fields such as image recognition and natural language processing. Vector retrieval technology is usually used to retrieve vectors that match a given vector in a vector data set (base library). In the process of vector retrieval, it mainly involves matching operations between vectors and screening of matching operation results. Among them, the matching operation between vectors is mainly performed by a data processing unit suitable for matrix operations, mainly performing matching operations between base library vectors in the base library and given vectors, and the screening of matching operation results is mainly performed by a data processing unit suitable for scalar operations, mainly screening the matching operation results to obtain a vector that matches the given vector.

[0059] In vector retrieval technology, the matching operation results are first stored in shared memory, rather than in the high bandwidth memory (HBM) that can be accessed by the data processing unit suitable for scalar operations. Therefore, the matching operation results calculated by the data processing unit suitable for matrix operations need to be moved from shared memory to HBM before being passed to the data processing unit suitable for scalar operations for further calculation.

[0060] When the amount of data in the matching operation result between vectors is large, copying data from shared memory to HBM will occupy a large amount of write bandwidth.

[0061] like Figure 1 As shown, Figure 1 It is a schematic diagram of the principle of the prior art. Figure 1It can be seen that, assuming that a matrix calculation involves blocksize base vectors and batch given vectors, the amount of data to be transferred from shared memory to HBM is blocksize*batch results. However, this transfer will occupy a large amount of write bandwidth, and there will be competition with other end-to-end write operations to the HBM on the computing accelerator card, which will affect the end-to-end performance. Especially when the number of query vectors retrieved in a batch is relatively large, this conflict in write bandwidth is more obvious, and it will also have more adverse effects on end-to-end performance.

[0062] Based on the above considerations, an embodiment of the present application provides a data processing method. After performing a target operation on the operation data in the first storage medium and obtaining the operation result, only the target operation result that meets the operation threshold in the operation result is written into the second storage medium. In this way, by filtering the operation results and reducing the writing of data, the occupation of the write bandwidth of the second storage medium can be reduced while ensuring the accuracy of the results, the competition with other operations that require write bandwidth can be alleviated, and the end-to-end performance can be improved.

[0063] See also Figure 2 , Figure 2 The schematic diagram of the architecture of an artificial intelligence (AI) system provided in an embodiment of the present application. The AI ​​system includes a computing device 10, a host 20 and a client device 30. The host 20 is a common computer device. A user can input data to the host 20 through the client device 30. The client device 30 is a terminal device, including but not limited to a personal computer, a server, a mobile phone, a tablet, a computer, etc.

[0064] The host 20 includes an input / output (I / O) interface 21, a processor 22, and a memory 23. The I / O interface 21 is used to communicate with devices outside the host 20. For example, the client device 30 sends a task to the host 20 through the I / O interface 21, and after the host 20 processes the task, it sends the result of the task to the client device 30 through the I / O interface 21.

[0065] The processor 22 is the computing core and control core of the host 20. It can be a central processing unit (CPU) or other specific integrated circuits. The processor 22 can also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. In practical applications, the host 20 can also include multiple processors. The processor 22 can include one or more processor cores. An operating system and other software programs are installed in the processor 22, so that the processor 22 can access the memory 23 and various peripheral component interconnect express (PCIe) devices.

[0066] The processor 22 is connected to the memory 23 via a double data rate (DDR) bus or other types of buses. The memory 23 is the main memory of the host 20. The memory 23 is usually used to store various running software in the operating system, input data received from the client device 30, and output results to be sent to the client device 30 in the future. In order to improve the access speed of the processor 22, the memory 23 needs to have the advantage of fast access speed. In traditional computer devices, dynamic random access memory (DRAM) is usually used as the memory 23. In addition to DRAM, the memory 23 can also be other random access memories, such as static random access memory (SRAM). In addition, the memory 23 can also be a read-only memory (ROM). As for the read-only memory, for example, it can be a programmable read only memory (PROM), an erasable programmable read only memory (EPROM), etc. The embodiment of the present application does not limit the number and type of memories.

[0067] Optionally, in order to store data persistently, the AI ​​system is further provided with a data storage system 40, which may be located outside the host 20 (e.g., Figure 2 As shown), the data storage system 40 exchanges data with the host 20 through the network. Optionally, the data storage system 40 may also be located inside the host 20, such as the data storage system 40 exchanges data with the processor 22 through the PCIe bus 24. In this case, the data storage system 40 is a hard disk.

[0068] The computing device 10 is used to perform AI tasks. The processor 22 sends the received AI tasks and input data to the computing device 10. After the computing device 10 completes the AI ​​tasks according to the input data, it sends the processing results to the processor 22. Figure 2 As shown, the computing device 10 can be directly inserted into a card slot on the motherboard of the host 20, and exchange data with the processor 22 through the PCIe bus 24. It should be noted that Figure 2 The PCIe bus 24 in the embodiment may also be replaced by a bus of a compute expresslink (CXL), a universal serial bus (USB) protocol or other protocols for data transmission.

[0069] See also Figure 3 , Figure 3 A schematic diagram of another AI system architecture provided in an embodiment of the present application. Figure 2 The AI ​​system shown differs in that, in Figure 3 In the embodiment, the computing device 10 is not directly inserted into the card slot on the motherboard of the host 20, but is located in other electronic devices 50. The electronic device 50 is a device independent of the host 20. At this time, the host 20 can be connected to the computing device 10 through a wired network such as a network cable, or through a wireless network such as a wireless hotspot (WiFi) or Bluetooth.

[0070] The computing device 10 is used to process AI tasks, such as training and reasoning, and the computing device 10 can be implemented by one or more chips. For example, the chip includes any one of a CPU, a graphics processing unit (GPU), a neural-network processing unit (NPU), a tensor processing unit (TPU), an FPGA, and an ASIC. Among them, the GPU, also known as a display core, a visual processor, and a display chip, is a microprocessor that specializes in image computing on personal computers, workstations, game consoles, and some mobile devices (such as tablets, smart phones, etc.). The NPU simulates human neurons and synapses at the circuit layer, and directly processes large-scale neurons and synapses with a deep learning instruction set, and one instruction completes the processing of a group of neurons. ASIC is suitable for a single-purpose integrated circuit product. The above-mentioned AI tasks may refer to image rendering, K-means clustering, or computing tasks based on a neural network model. For example, AI tasks include matrix computing tasks in image rendering. In addition, AI tasks may also be distance operations in vector retrieval.

[0071] For example, Figure 2 The computing device 10 in the Figure 4 The AI ​​chip shown is implemented as Figure 4 As shown, Figure 4 This is a structural diagram of an AI chip provided in the present application. By way of example, the AI ​​chip includes an AI core 101, an AI CPU 102, a system buffer 103 and a memory 106.

[0072] Among them, AI CPU 102 is used to accept AI tasks and call AI cores to execute the tasks. In the case of multiple AI cores 101, AI CPU 102 is also used to undertake scheduling tasks. AI CPU 102 can be implemented by an ARM processor, which is small in size, low in power consumption, uses a 32-bit reduced instruction set, and has simple and flexible addressing. Of course, in some embodiments, AI CPU 102 can also be implemented by other processors.

[0073] The AI ​​core 101 is used to provide the neural network model involved in the AI ​​task and perform corresponding operations. The system buffer 103 mainly refers to the L2 buffer, or L2 cache, which is used to temporarily store the input data, intermediate results or final results passing through the AI ​​chip. The memory 106 is mainly used to store data involved in the calculation, which can be HBM or global memory. The I / O devices 105 included in the AI ​​chip are mainly some peripherals, such as network interface cards. In some application scenarios, data needs to be encoded or decoded, so the AI ​​chip may also include a codec 104 and an I / O device 105, and the codec 104 is used to encode or decode data.

[0074] The AI ​​core 101 includes a load / store unit 1011 (Load / Store Unit, LSU), a matrix calculation unit 1012, a scalar calculation unit 1015, a vector calculation unit 1013 and a buffer 1014. Among them, the load / store unit 1011 is used to load the data to be processed and store the processed data. It can also be used for the read and write management of the internal data in the AI ​​core 101 between different buffers, and to complete some format conversion operations. The matrix calculation unit 1012 is used to provide the core computing power of matrix multiplication. The scalar calculation unit 1015 is a single instruction single data (SISD) processor, which only processes one piece of data (usually an integer or floating point number) at the same time. The vector calculation unit 1013, also known as an array processor, is a processor that can directly operate a group of arrays or vectors for calculation. The number of buffers may be one or more. The buffer 1014 is mainly used to temporarily store some data that needs to be used repeatedly to reduce the number of repeated readings. In addition, the implementation of certain data format conversion functions also requires that the source data is located in the buffer 1014. The buffer 1014 is located in the AI ​​core, shortening the distance between the matrix computing unit 1012 and the storage, and reducing the access to the memory 106. Optionally, the buffer 1014 can be a shared memory, or a unified buffer (UB), etc.

[0075] based on Figure 2 , Figure 3 , Figure 4According to the architecture shown, the AI ​​system provided by the embodiment of the present application can execute the following process: after the host 20 receives the AI ​​task sent by the client device 30, such as vector retrieval, the host processor 22 sends the received AI task to the AI ​​CPU 102 in the computing device 10, and the AI ​​CPU 102 loads the data to be processed by the AI ​​task into the memory 106, and the load / store unit 1011 in the AI ​​core 101 loads the data to be processed into the buffer, so that the matrix calculation unit 1012, the vector calculation unit 1013 and / or the scalar calculation unit 1015 can read the data from the buffer 1014 to perform the target operation. When the operation result is obtained, the load / store unit 1011 loads the operation result into the memory 106, and then the AI ​​CPU 102 can read the operation result from the memory 106 for further processing, such as scalar calculations such as sorting and summing, and after obtaining the processing result, the processing result is sent to the host by the I / O device 105.

[0076] The specific implementation of the data processing method provided in the embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0077] like Figure 5 As shown, Figure 5 A flow chart of a data processing method provided in an embodiment of the present application, which can be applied to Figure 2 The AI ​​system shown, the data processing method is executed by a computing device. Figure 5 As shown, the data processing method provided in the embodiment of the present application includes the following steps:

[0078] S210: Perform a target operation on the operation data in the first storage medium to obtain an operation result.

[0079] In an embodiment of the present application, a computing device includes a first storage medium and a second storage medium.

[0080] The first storage medium may be a buffer in a computing device, which is mainly used to temporarily store some data that needs to be used repeatedly, and may also be used to store intermediate results.

[0081] Exemplarily, the first storage medium may be a shared memory. When the computing device is an NPU, the first storage medium may also be a unified buffer (UB).

[0082] The second storage medium is a memory commonly used for data, which can be accessed by all threads in the computing device and shared globally. Exemplarily, the second storage medium can be HBM or global memory.

[0083] In an embodiment of the present application, the read and write speed of the first storage medium is greater than that of the second storage medium, and the access delay of the first storage medium is lower than that of the second storage medium.

[0084] In an embodiment of the present application, the operation data is pre-stored in the second storage medium, so that each thread can read the operation data from the second storage medium for operation. When each thread reads the operation data from the second storage medium for operation, it is necessary to access the second storage medium for each calculation and read the operation data from the second storage medium. Considering that the first storage medium is mainly used to store repeatedly used data, in order to improve the operation efficiency, the data processing method provided in the embodiment of the present application can also write the operation data and the operation threshold stored in the second storage medium into the first storage medium before performing the target operation on the operation data in the first storage medium, and improve the operation efficiency by reading the operation data from the first storage medium.

[0085] For example, for a matrix multiplication operation C = AB, to obtain the element in the i-th row and j-th column of matrix C, it is necessary to multiply all the elements in the i-th row of matrix A and the j-th column of matrix B one by one and sum them, that is, C(i,j) = A i1 B 1j +A i2 B 2j +…+A ip B pj , where p is the number of columns of matrix A and the number of columns of matrix B. In this case, if all elements in the i-th row of matrix C are to be obtained, this operation needs to be performed p times. During these p operations, the i-th row elements of matrix A remain unchanged, and the elements of matrix B are iteratively updated. Assuming that the data of matrix A is stored in a second storage medium, the i-th row elements of matrix A must be reloaded from the second storage medium for each operation, and the data is repeatedly loaded p times. If the i-th row elements of matrix A are stored in the first storage medium, the i-th row elements of matrix A can be repeatedly used through the first storage medium. In this way, during the operation, there is only one load from the second storage medium to the first storage medium, which can avoid repeated loading of the i-th row elements of matrix A, thereby improving the operation efficiency.

[0086] Based on this, in scenarios such as matrix operations with large amounts of data, using the first storage medium can improve operating efficiency compared to the second storage medium.

[0087] In the embodiment of the present application, the operation data includes given data and base data, and the target operation of the operation data is mainly to perform target operation on the given data and the base data. The given data is data specified by the user, such as a specified picture, text, etc., and the given data can be transmitted to the second storage medium through the client device.

[0088] The base data is data pre-stored in the second storage medium, such as image data, text data, etc. The base data is mainly used as comparison data, sample data, etc. to participate in the operation. Exemplarily, the base data and the given data are subjected to a target operation, and the base data matching the given data can be obtained through operation using the base data as comparison data.

[0089] For example, in the scenario of model training, the base database data can be used as sample data to participate in the training of the model.

[0090] In an embodiment of the present application, the base database data may be pre-created according to business requirements, and the amount of the base database data is much larger than the given data.

[0091] In an embodiment of the present application, the given data will perform a target operation with each base database data and obtain the corresponding operation result. The operation threshold will be used to compare with each operation result. Therefore, the operation threshold is also reused. The operation threshold can also be written into the first storage medium to improve the operation efficiency.

[0092] In some embodiments, the calculation threshold value can be transmitted to the second storage medium of the computing device through the client device, and then written from the second storage medium to the first storage medium to participate in the calculation.

[0093] In the embodiment of the present application, the given data and the base database data may be stored in the form of a matrix, so the target operation may be an operation related to the matrix, including but not limited to one or more of the following: matrix addition, matrix multiplication, matrix subtraction, etc. In some embodiments, the target operation may also include operations such as finding the maximum value, the minimum value, and summing and averaging the elements in the matrix, which may be determined according to business needs and are not limited here.

[0094] S220: Compare the operation result with the operation threshold to obtain a target operation result that meets the operation threshold.

[0095] Exemplarily, after obtaining the operation result, the obtained operation result can be compared with the operation threshold to obtain a target operation result that meets the operation threshold.

[0096] Among them, the target operation result that meets the operation threshold can be an operation result greater than the operation threshold, or a operation result less than the operation threshold, or a operation result equal to the operation threshold, which can be set according to business needs, and the embodiments of the present application do not make specific restrictions. For example, in the scenario of image similarity calculation, if the goal is to select similar images, the result less than the operation threshold can be selected as the target operation result, and if the goal is to select images with greater differences, the result greater than or equal to the operation threshold can be selected as the target operation result.

[0097] In an optional implementation, the operation threshold is related to business requirements and can be set according to business requirements.

[0098] S230: Writing the target operation result into the second storage medium.

[0099] In the embodiment of the present application, after the operation result is obtained, the operation result is stored in the first storage medium. For the first storage medium, the content stored therein is only shared by threads in the same block, and the content seen by threads in different blocks is different. Based on this, in order to facilitate further processing of subsequent operation results, after the operation result is obtained by performing the target operation on the operation data in the first storage medium, the operation result needs to be copied to the second storage medium.

[0100] In order to reduce the bandwidth occupied by the second storage medium when writing the operation results, in the embodiment of the present application, only the target operation results that meet the operation threshold can be written into the second storage medium.

[0101] In an embodiment of the present application, the computing device may include a first data processing unit and a second data processing unit. The first data processing unit can be used to perform a target operation on the operation data in the first storage medium, obtain an operation result, compare the operation result with an operation threshold, and obtain a target operation result that meets the operation threshold. The second data processing unit can be used to obtain the target operation result from the second storage medium and further process the target operation result.

[0102] by Figure 2 , Figure 3 , Figure 4Taking the architecture shown as an example, the first data processing unit in the computing device may include a matrix computing unit, a vector computing unit and / or a scalar computing unit, and the second data processing unit in the computing device may be an AICPU. When performing a target operation, the load / store unit 1011 may copy the given data, the base data and the operation threshold from the memory 106 to the buffer 1014, and after the given data, the base data and the operation threshold are copied from the memory 106 to the buffer 1014, the matrix computing unit 1012, the vector computing unit 1013 and / or the scalar computing unit 1015 may read the given data and the base data from the buffer 1014 to perform the target operation, obtain the operation result, and then compare the operation result with the operation threshold to obtain the target operation result that meets the operation threshold, and store the target operation result in the buffer 1014, and then the load / store unit 1011 writes the target operation result from the buffer 1014 to the memory 106, so that the AI ​​CPU 102 can read the target operation result from the memory 106 for the next step of processing. The embodiment of the present application filters the results of the target operation by the operation threshold, excludes the operation results that do not meet the operation threshold, and only writes the target operation results that meet the operation threshold to the second storage medium, thereby reducing the memory access pressure on the second storage medium, thereby reducing the occupation of the write bandwidth of the second storage medium when the first data processing unit is running, which can alleviate the competition with other operations that require write bandwidth and improve end-to-end performance.

[0103] In order to better illustrate the data processing method provided in the embodiment of the present application, the embodiment of the present application is described in detail using the scenario of vector retrieval. Figure 6 The operation data in the embodiment of the present application includes a base vector and a query vector, and the vector retrieval step may include:

[0104] S310: Perform a distance operation on the base vector and the query vector to obtain a distance value between the query vector and the base vector.

[0105] The query vector may be a vector obtained by vectorizing given data, and the base vector may be a vector obtained by vectorizing base data. For example, the base vector may be a vector obtained by vectorizing base images, and the query vector may be a vector obtained by vectorizing the images to be queried. For another example, the base vector may be a vector obtained by vectorizing base text, and the query vector may be a vector obtained by vectorizing the text to be queried.

[0106] In the embodiment of the present application, the distance operation is performed on the base database vector and the query vector, and for each query vector, a distance operation is performed on the query vector and each base database vector to obtain the distance value between the query vector and each base database vector.

[0107] Optionally, when performing distance calculation on the query vector and the base database vector, the Euclidean distance, angle cosine, Hamming distance, Chebyshev distance, Mahalanobis distance, etc. between the query vector and the base database vector can be calculated. This application does not limit the distance calculation method between the query vector and the base database vector, and it can be set specifically according to business needs.

[0108] S320: Compare the distance value with the operation threshold, obtain a target operation result that meets the operation threshold, and write the target operation result into a second storage medium.

[0109] In the embodiment of the present application, the operation result includes the distance value between the query vector and the base vector. Since the number of base vectors is large, the number of distance values ​​between the query vector and the base vector is also large. Therefore, if all the distance values ​​between the query vector and the base vector are written to the second storage medium, the write bandwidth of the second storage medium will be seriously occupied.

[0110] In order to reduce the write bandwidth of the second storage medium, after obtaining the distance value between the query vector and the base library vector, the embodiment of the present application compares the distance value between the query vector and the base library vector with the operation threshold to obtain a target distance value that meets the operation threshold, and then writes the target distance value that meets the operation threshold as the target operation result into the second storage medium.

[0111] Among them, the target distance value that meets the operation threshold can be a distance value greater than the operation threshold, a distance value less than the operation threshold, or a distance value equal to the operation threshold. It can be set according to business needs, and the embodiments of the present application do not impose specific restrictions.

[0112] In the embodiment of the present application, the distance values ​​between the query vector and the base library vector are filtered by using a calculation threshold to exclude distance values ​​that do not match the calculation threshold, and the distance values ​​that match the calculation threshold are written into the second storage medium, thereby reducing data transfer and reducing the occupation of the write bandwidth of the second storage medium.

[0113] S330: Sort the target operation results to obtain the first K base vectors that best match the query vector.

[0114] After the target distance value matching the operation threshold is written into the second storage medium as the target operation result, the second data processing unit can obtain the target distance value from the second storage medium, sort the target distance value, and obtain the first K base vectors that best match the query vector. K is a positive integer and can be set according to business needs.

[0115] by Figure 2 , Figure 3 , Figure 4Taking the architecture shown as an example, after writing the target distance value into the memory 106, the AI ​​CPU 102 can read the target distance value from the memory 106, sort the target distance value, and obtain the top K base vectors that best match the query vector.

[0116] Exemplarily, the smaller the distance value, the more similar the base library vector is to the query vector. For example, in the field of face matching query, if you need to find a base library face image that matches the face to be queried, you can exclude the distance value greater than the operation threshold after obtaining the distance value between the query vector and the base library vector, and write the distance value less than or equal to the operation threshold as the target operation result to the second storage medium. Afterwards, the second data processing unit (such as the AI ​​CPU) reads the distance value less than or equal to the operation threshold from the second storage medium, sorts the distance value less than or equal to the operation threshold, and determines the top K base library vectors that are most similar to the query vector from the distance value less than or equal to the operation threshold, thereby obtaining the base library face image that matches the face to be queried.

[0117] For another example, in the scenario of sample data construction, if it is necessary to construct dissimilar training samples, then after obtaining the distance value between the query vector and the base library vector, the distance value less than the operation threshold can be excluded, and the distance value greater than or equal to the operation threshold can be written into the second storage medium as the target operation result. Then, the second data processing unit reads the distance value greater than or equal to the operation threshold from the second storage medium to select the first K base library vectors that are dissimilar to the query vector from the distance values ​​greater than or equal to the operation threshold, and then construct the training samples.

[0118] The scenarios listed above are merely examples of obtaining the top K base vectors that best match the query vector in the embodiments of the present application, and do not constitute specific limitations. The specific method can be set according to business requirements, and the embodiments of the present application do not make specific limitations.

[0119] For example, in some embodiments, in the scenario of updating the base database data, it is necessary to filter out the query vectors that are not similar to the base database vectors. Then, after obtaining the distance value between the query vector and the base database vector, a first query vector and a second query vector can be determined, wherein the first query vector is a query vector having a distance value less than an operation threshold, and the second query vector is a query vector having a distance value greater than the operation threshold. All distance values ​​of the first query vector are excluded, and the distance value of the second query vector greater than the operation threshold is written into the second storage medium as the target operation result. Afterwards, the second data processing unit reads the distance value greater than the operation threshold from the second storage medium, sorts the distance value greater than the operation threshold, and selects the first K second query vectors that are not similar to the base database vector from the distance value greater than the operation threshold, and then updates the base database data based on the selected second query vector, that is, determines the selected second query vector as the new base database data.

[0120] For another example, in some embodiments, if the distance value is the cosine value of the angle, the smaller the distance value, the less similar the base vector is to the query vector. In this case, if the first K base vectors that best match the query vector are to be screened, the distance value less than the operation threshold can be excluded, and the distance value greater than or equal to the operation threshold can be written into the second storage medium as the target operation result, so that the second data processing unit can select the first K base vectors that best match the query vector from the distance value greater than or equal to the operation threshold. If the base vectors that are dissimilar to the query vector are to be screened, the distance value greater than the operation threshold can be excluded, and the distance value less than or equal to the operation threshold can be written into the second storage medium as the target operation result, so that the second data processing unit can select the first K base vectors that are most dissimilar to the query vector from the distance value less than or equal to the operation threshold.

[0121] Considering the increase in the amount of data in the base vector, it takes a long time to calculate the query vector and the base vector in sequence. In order to improve the data processing efficiency, in some embodiments of the present application, the base vectors can be grouped and the distance values ​​of the base vectors of multiple groups can be calculated in parallel. Based on this, please refer to Figure 7 , the above distance operation is performed on the base vector and the query vector to obtain the distance value between the query vector and the base vector, which may include:

[0122] S311: Grouping the base library vectors to obtain multiple vector groups.

[0123] When grouping the base vectors, as an example, the base vectors can be grouped according to the set number of data to obtain multiple vector groups. Each vector group includes the base vectors of the set number of data. For example, if the number of data is set to burstsize, after the base vectors are grouped based on burstsize, each vector group includes burstsize base vectors.

[0124] like Figure 8 As shown, Figure 8 A schematic diagram of a data processing principle provided in an embodiment of the present application is provided. Figure 8 Each rectangle in represents a base vector, and there are M base vectors in total. After grouping the M base vectors, multiple vector groups can be obtained, among which base vectors of the same style belong to the same vector group.

[0125] S312: Perform distance operations on the query vector and the bottom-base vectors in the plurality of vector groups respectively to obtain distance values ​​between the query vector and the bottom-base vectors in the plurality of vector groups.

[0126] After obtaining multiple vector groups, distance operations may be performed on the query vector and the bottom-base vectors in the multiple vector groups to obtain distance values ​​between the query vector and the bottom-base vectors in the multiple vector groups.

[0127] For example, assuming that the bottom-base vectors are grouped to obtain vector group X1 and vector group X2, and distance operations are performed on the query vector and the bottom-base vectors in the multiple vector groups, it means that a distance operation is performed between the query vector and the bottom-base vectors in vector group X1, and a distance operation is performed between the query vector and the bottom-base vectors in vector group X2.

[0128] It can be understood that for each vector group, after performing distance calculation between the query vector and the base vectors in the vector group, the distance value obtained is the same as the number of base vectors in the vector group, that is, a distance value is obtained for each base vector in the vector group. For example, Figure 8 As shown, the number of query vectors is 1, and the Dist result is a distance value. After the distance operation, a distance value will be obtained for each base vector in the vector grouping. Therefore, the Dist result contains M distance values.

[0129] The embodiment of the present application groups the base vectors and performs distance calculation in a vector grouping manner, so that distance calculation can be performed on multiple base vectors at the same time, thereby improving data processing efficiency.

[0130] In order to quickly screen out the target operation results that match the operation threshold, in some embodiments of the present application, after obtaining the distance value between the query vector and the base vectors in the multiple vector groups, the index values ​​corresponding to the multiple vector groups can be determined, and the target operation results that match the operation threshold, i.e., the target distance value, can be quickly screened out based on the comparison result between the index value and the operation threshold. Based on this, please refer to Fig. 9 In some embodiments of the present application, vector retrieval can also be implemented in the following manner.

[0131] S410: Grouping the base library vectors to obtain multiple vector groups.

[0132] S420: Perform distance operations on the query vector and the bottom-base vectors in the plurality of vector groups respectively to obtain distance values ​​between the query vector and the bottom-base vectors in the plurality of vector groups.

[0133] The process of step S410 to step S420 may refer to step S311 to step S312, and will not be described in detail here.

[0134] S430: Determine index values ​​of distance values ​​of bottom-base vectors in a plurality of vector groups.

[0135] Each of the plurality of vector groups will obtain an index value. For example, for each vector group, after obtaining the distance value between the query vector and the bottom-base vector in the vector group, the distance value of the bottom-base vector in the vector group is calculated to obtain the index value of the vector group.

[0136] like Fig.10 As shown, Fig.10 A schematic diagram of the data processing principle provided in an embodiment of the present application. Fig.10 Dist_Z is the index value of vector grouping. The number of base vectors is M, and the set number of data is burstsize. M / burstsize vector groups can be obtained, and the number of Dist_Z is also M / burstsize.

[0137] In an embodiment of the present application, the index value of the vector grouping may be the maximum distance value or the minimum distance value among the distance values ​​of the bottom library vectors in the vector grouping, which may be specifically set according to business requirements. For example, if a distance value smaller than the operation threshold needs to be selected based on business requirements, the minimum distance value among the distance values ​​of the bottom library vectors in the vector grouping may be set as the index value of the vector grouping. If a distance value larger than the operation threshold needs to be selected, the maximum distance value among the distance values ​​of the bottom library vectors in the vector grouping may be set as the index value of the vector grouping.

[0138] S440: Compare the index values ​​of the plurality of vector groups with the operation thresholds respectively to obtain a target operation result, and write the target operation result into a second storage medium.

[0139] After obtaining the index value of the vector group, the index value of the vector group can be compared with the operation threshold. When the index value of the vector group does not meet the operation threshold, it means that the distance values ​​of the bottom-base vectors in the vector group do not meet the operation threshold, and the distance values ​​of the bottom-base vectors in the vector group are excluded. When the index value of the vector group meets the operation threshold, it means that the distance values ​​of the bottom-base vectors in the vector group meet the operation threshold, and the distance values ​​of the bottom-base vectors in the vector group are retained.

[0140] Exemplarily, when it is necessary to select a distance value smaller than the operation threshold, the index value of the vector grouping is the minimum distance value among the distance values ​​of the bottom library vectors in the vector grouping. For each vector grouping, after comparing the index value of the vector grouping with the operation threshold, if the index value of the vector grouping is greater than the operation threshold, it means that the distance values ​​of the bottom library vectors in the vector grouping are all greater than the operation threshold, that is, the distance values ​​of the bottom library vectors in the vector grouping do not meet the operation threshold, and then the distance values ​​of the bottom library vectors in the vector grouping are excluded. If the index value of the vector grouping is less than the operation threshold, it means that the vector grouping has a bottom library vector whose distance value is less than the operation threshold, and then the distance value of the bottom library vector in the vector grouping can be retained. It can be understood that when the index value of the vector grouping is equal to the operation threshold, it also means that the vector grouping does not have a bottom library vector whose distance value is less than the operation threshold, and then the distance value of the bottom library vector in the vector grouping can be excluded.

[0141] Exemplarily, when it is necessary to select a distance value greater than the operation threshold, the index value of the vector grouping is the maximum distance value among the distance values ​​of the bottom library vectors in the vector grouping. For each vector grouping, after comparing the index value of the vector grouping with the operation threshold, if the index value of the vector grouping is less than the operation threshold, it means that the distance values ​​of the bottom library vectors in the vector grouping are all less than the operation threshold, that is, the distance values ​​of the bottom library vectors in the vector grouping do not meet the operation threshold, and then the distance values ​​of the bottom library vectors in the vector grouping are excluded. If the index value of the vector grouping is greater than the operation threshold, it means that the vector grouping has a bottom library vector whose distance value is greater than the operation threshold, and then the distance value of the bottom library vector in the vector grouping can be retained. It can be understood that when the index value of the vector grouping is equal to the operation threshold, it also means that the vector grouping does not have a bottom library vector whose distance value is greater than the operation threshold, and then the distance value of the bottom library vector in the vector grouping can be excluded.

[0142] Compared with the method of comparing the distance value of each bottom-base vector with the operation threshold in turn, the embodiment of the present application determines the index value of the distance value of the bottom-base vector in the vector group, and compares the index value of the distance value of the bottom-base vector in the vector group with the operation threshold, so as to quickly obtain the result and improve the processing efficiency.

[0143] It can be understood that after comparing the index value of the vector grouping with the operation threshold, a vector grouping whose index value meets the operation threshold can be obtained. In the embodiment of the present application, the target operation result refers to the distance value of the bottom library vector in the first vector grouping among the multiple vector groupings, wherein the first vector grouping refers to the vector grouping whose index value meets the operation threshold among the multiple vector groupings, and the second vector grouping refers to the vector grouping whose index value does not meet the operation threshold among the multiple vector groupings.

[0144] like Fig.10 As shown, Fig.10 A schematic diagram of the data processing principle provided in an embodiment of the present application. Fig.10 Where Threshold is the operation threshold, Dist_Z is the index value of the vector grouping, and the number of operation thresholds is the same as the number of index values, which means that the index value of each vector grouping can be compared with the operation threshold in parallel. By comparing the index value of each vector grouping with the operation threshold, the first vector grouping whose index value meets the operation threshold can be obtained, that is, the Dist result that meets the Threshold.

[0145] After obtaining the first vector grouping and the second vector grouping, the distance values ​​of the bottom-base vectors in the second vector grouping are removed, and the distance values ​​of the bottom-base vectors in the first vector grouping are retained. Therefore, when writing data to the second storage medium, only the distance values ​​of the bottom-base vectors in the first vector grouping are written to the second storage medium. Fig.10 As shown, the distance value of the bottom library vector in the first vector group is the Dist result written into the second storage medium.

[0146] The embodiment of the present application excludes the distance value of the bottom-base vector in the second vector group, and only writes the distance value of the bottom-base vector in the first vector group as the target operation result to the second storage medium, thereby reducing data transportation and reducing the occupation of the write bandwidth of the second storage medium, which can alleviate the competition with other operations that require write bandwidth and improve end-to-end performance.

[0147] In an embodiment of the present application, after the distance values ​​of the bottom-base vectors of the first vector group are written as the target operation results to the second storage medium, the second data processing unit can read the distance values ​​of the bottom-base vectors of the first vector group from the second storage medium, and sort the distance values ​​of the bottom-base vectors of the first vector group to obtain the top N bottom-base vectors that best match the query vector.

[0148] It should be noted that, in the embodiment of the present application, each distance value of the bottom-base vector corresponds to a storage location in the second storage medium. When the distance value of the bottom-base vector in the first vector group is written to the second storage medium, the distance value of the bottom-base vector in the first vector group can be written to the corresponding storage location for storage. Since the distance value of the bottom-base vector in the second vector group is not written to the second storage medium, the storage location corresponding to the bottom-base vector belonging to the second vector group in the second storage medium does not store the distance value.

[0149] Considering that in the case of a large amount of data, if the distance values ​​of the base vectors of the first vector group are sorted one by one to obtain the first K base vectors that best match the query vector, it takes a long time and is inefficient. In order to quickly obtain the first K base vectors that best match the query vector, the embodiment of the present application may also include the following steps:

[0150] S450: Writing the index values ​​of the plurality of vector groups into a second storage medium.

[0151] like Fig.11 As shown, Fig.11 A schematic diagram of the data processing principles provided in an embodiment of the present application. Fig.11 When the distance values ​​of the base vectors in the first vector group are written into the second storage medium, the index values ​​of each vector group are also written into the second storage medium, that is, the Dist_Z result is written into the second storage medium.

[0152] S460: Perform a topK operation on the target operation result according to the index values ​​of the multiple vector groups to obtain the top K base vectors that best match the query vector.

[0153] Among them, after reading the distance value and the index value of the bottom library vector in the first vector group from the second storage medium, the second data processing unit can perform a topK operation on the distance value of the bottom library vector in the read first vector group based on the index value, and the topK operation refers to the process of determining the largest or smallest first K data from a series of data.

[0154] Exemplarily, the first K base vectors with the smallest distance value from the query vector are selected as an example for explanation, and the index value of the vector group is set to the minimum distance value among the distance values ​​of the base vectors in the vector group. When the index value of the vector group is less than the operation threshold, the vector group is the first vector group, and when the index value of the vector group is greater than the operation threshold, the vector group is the second vector group. After the second data processing unit reads the index values ​​of multiple vector groups and the distance values ​​of the base vectors of the first vector group from the second storage medium, the index values ​​of the multiple vector groups can be sorted in ascending order to obtain an index value sequence, and then the first K minimum distance values ​​are selected from the vector group corresponding to the minimum index value in the index value sequence, and the maximum distance value among the first K minimum distance values ​​is determined.

[0155] Then, the second index value is selected from the index value sequence, and the second index value is determined as the target index value. The target index value is compared with the maximum distance value. If the target index value is greater than or equal to the maximum distance value, it means that the distance values ​​of the bottom-base vectors in the vector group corresponding to the target index value are all greater than or equal to the maximum distance value, and the bottom-base vectors in the vector group corresponding to the target index value do not meet the first K bottom-base vectors with the smallest distance value, and the distance values ​​of the bottom-base vectors in the vector group corresponding to other index values ​​in the index value sequence are also greater than or equal to the maximum distance value, and the bottom-base vectors in the vector group corresponding to other index values ​​do not meet the first K bottom-base vectors with the smallest distance value, and then the bottom-base vectors corresponding to the first K minimum distance values ​​in the vector group corresponding to the minimum index value are the first K bottom-base vectors with the smallest distance value to the query vector.

[0156] If the target index value is less than the maximum distance value, it means that there is a bottom-stock vector with a distance value less than the maximum distance value in the vector group corresponding to the target index value. Therefore, the maximum distance value can be eliminated, and the target index value can be placed in the first K minimum distance values, and the maximum distance value among the first K minimum distance values ​​can be updated. Then, any distance value of the bottom-stock vector can be selected in the vector group corresponding to the target index value, and the distance value of the selected bottom-stock vector can be compared with the updated maximum distance value. If it is greater than the updated maximum distance value, the distance value of the selected bottom-stock vector can be eliminated, and a distance value of the bottom-stock vector can be selected from the vector group corresponding to the target index value again, and compared with the updated maximum distance value again. If it is less than the updated maximum distance value, the updated maximum distance value can be eliminated, and the distance value of the re-selected bottom-stock vector can be placed in the first K minimum distance values, and the maximum distance value among the first K minimum distance values ​​can be updated again, and so on, until the distance values ​​of all bottom-stock vectors in the vector group corresponding to the target index value are traversed.

[0157] After traversing the distance values ​​of all bottom-level vectors in the vector group corresponding to the target index value, the third index value can be selected from the index value sequence, and the third index value can be determined as the new target index value, and then compared with the maximum distance value of the current first K minimum distance values, and so on, all index values ​​in the index value sequence can be directly traversed.

[0158] It should be noted that if the number of distance values ​​of the bottom library vector in the vector grouping corresponding to the minimum index value is not enough K, the first K minimum distance values ​​can be selected from the distance values ​​of the bottom library vector of the vector grouping corresponding to the index values ​​of the first 2 index values ​​in the index value sequence, if the first K minimum distance values ​​are still not obtained, the first K minimum distance values ​​can be selected from the distance values ​​of the bottom library vector of the vector grouping corresponding to the index values ​​of the first 3 index values, and so on, until the first K minimum distance values ​​are obtained. It can be understood that in this case, when comparing, the comparison is started from the next index value of the selected index value. For example, the first K minimum distance values ​​are selected from the distance values ​​of the bottom library vector of the vector grouping corresponding to the index values ​​of the first 2 index values, and the maximum distance value is obtained from the first K minimum distance values, then the index value ranked in the third can be selected from the index value sequence, and the index value ranked in the third is compared with the maximum distance value. In some embodiments, the index value of the second vector group may be removed, and only the index value of the first vector group is written into the second storage medium, and the distance values ​​of the bottom-base vectors in the first vector group are sorted based on the index value of the first vector group.

[0159] In the embodiment of the present application, by writing the index value of the vector grouping into the second storage medium, the second data processing unit can quickly sort the distance values ​​based on the index value of the vector grouping, thereby improving data processing efficiency.

[0160] Considering that in some embodiments, the number of vector groups obtained by grouping the bottom library vectors is large, if the distance value of the bottom library vector of the first vector group is written to the second storage medium, the first vector group will be found in multiple vector group traversals, and then the first vector group will be written to the second storage medium. Multiple data writes will occur in the whole process, which is time-consuming and inefficient. In order to improve processing efficiency, in an embodiment of the present application, data can be written to the second storage medium in the form of group sets, where the group set refers to a set of vector groups.

[0161] In the embodiment of the present application, due to the large amount of data of the bottom library vector, in order to facilitate data processing, the bottom library vector can be divided according to the set size blocksize to obtain multiple bottom library vector blocks, each bottom library vector block includes blocksize bottom library vectors. Multiple bottom library vector blocks are divided into multiple rounds for data processing, and the bottom library vector blocks of each round are processed in turn, that is, only one bottom library vector block is processed each time. When the bottom library vector block is processed, the bottom library vector block can be divided into multiple parts (part) according to the number of cores, and each core processes the bottom library vector of a part. And each core, when processing the bottom library vector in the part, will divide the bottom library vector in the part into multiple grouping sets according to the set number of sets. The bottom library vectors in each grouping set will be grouped according to the set data size burstsize to obtain multiple vector groups, that is, each grouping set includes multiple vector groups.

[0162] Exemplarily, blocksize is set to A, the number of cores is B, the number of sets is set to C, and burstsize is set to D. Then, a base vector block includes A base vectors, and the size of the part corresponding to each core is A / B, that is, each core needs to process A / B base vectors, the number of base vectors included in each grouping set is A / B / C, and the number of vector groups included in each grouping set is A / B / C / D.

[0163] In the embodiment of the present application, on each core, the grouping sets obtained by division are processed sequentially, that is, only the base library vector of one grouping set is processed each time, while multiple cores are processed in parallel. Fig.12 As shown, Fig.12 The schematic diagram shows the principle of data processing of two cores (Core_1 and Core_2). Fig.12 The data volume of the bottom library vector is M, and the part corresponding to each core contains N bottom library vectors. Among them, the two cores perform distance operations on the assigned bottom library vectors in parallel, and each core obtains N / burstsize Dist_Z results. After each core obtains a Dist result that meets the Threshold, it writes its own Dist result that meets the Threshold to the second storage medium, and also writes its own Dist_Z result to the second storage medium, so that there are M / burstsize Dist_Z in the second storage medium.

[0164] Please refer to Fig.13 The step of writing data to the second storage medium in a grouped set manner may include:

[0165] S510: Perform distance operations on the base vectors and the query vector in the vector groups included in the grouping set to obtain distance values ​​between the base vectors and the query vector in the vector groups included in the grouping set and index values ​​of the vector groups included in the grouping set.

[0166] Among them, for each grouping set, when processing the bottom library vector of the grouping set, the bottom library vector in the vector grouping included in the grouping set can be respectively performed with the query vector to obtain the distance value between the bottom library vector in the vector grouping included in the grouping set and the query vector, and then the distance value of the bottom library vector in the vector grouping included in the grouping set can be performed with an index operation to obtain the index value of the vector grouping included in the grouping set.

[0167] S520: If there is a vector group whose index value meets the operation threshold in the vector groups included in the group set, determine the group set as the target group set, and write the distance value of the base library vector in the vector group included in the target group set to the second storage medium.

[0168] Among them, for each grouping set, after obtaining the distance value between the bottom-base vector in the vector grouping included in the grouping set and the query vector, as well as the index value of the vector grouping included in the grouping set, the index value of the vector grouping included in the grouping set is compared with the operation threshold. If the index values ​​of the vector grouping included in the grouping set do not meet the operation threshold, it means that the distance values ​​of all bottom-base vectors in the grouping set do not meet the operation threshold, and then the grouping set can be eliminated, that is, there is no need to write the distance value of the bottom-base vector in the grouping set into the second storage medium.

[0169] If there is a vector grouping whose index value meets the operation threshold in the vector groups included in the grouping set, the grouping set is determined as the target grouping set, and the distance values ​​of the bottom library vectors in the vector groups included in the target grouping set are written to the second storage medium, that is, the target operation result is the distance value of the bottom library vectors in the vector groups included in the target grouping set.

[0170] In an embodiment of the present application, when the index values ​​of the vector groups included in the grouping set do not meet the operation threshold, the distance values ​​of the bottom library vectors in the vector groups included in the grouping set are eliminated; when there are vector groups whose index values ​​meet the operation threshold in the vector groups included in the grouping set, the distance values ​​of the bottom library vectors in the vector groups included in the grouping set are written to the second storage medium. On the one hand, data writing is reduced and the occupation of the write bandwidth of the second storage medium is reduced. On the other hand, data writing is performed in the grouping set, which reduces the number of data writes and can improve data processing efficiency.

[0171] Taking into account that the target group set also includes vector groups whose index values ​​do not meet the operation threshold, in order to facilitate subsequent processing by the second data processing unit and improve processing efficiency, in an embodiment of the present application, the marker sequence corresponding to the target group set can also be written into the second storage medium.

[0172] The marking sequence is used to mark whether the index value of the vector group in the target group set meets the operation threshold. For each vector group, if the index value of the vector group meets the operation threshold, the vector group can be marked as the first mark; if the index value of the vector group does not meet the operation threshold, the vector group can be marked as the second mark. Fig.14 As shown, Fig.14 The schematic diagram of the principle of data processing provided in the embodiment of the present application, ValidFlags is a tag sequence, and for the vector grouping that meets the operation threshold in the target grouping set, the tag at the corresponding position in ValidFlags is the first tag, such as Fig.14 In the figure 1, for the vector groupings in the target grouping set that do not meet the operation threshold, the mark at the corresponding position in ValidFlags is the second mark, such as Fig.14 The number 0.

[0173] In the embodiment of the present application, the first mark and the second mark are different marks, and the first mark and the second mark can be represented by characters such as English, numbers, or a combination of characters. The embodiment of the present application does not limit the form of the first mark and the second mark, as long as they can distinguish the vector grouping.

[0174] For each grouping set, after obtaining the distance value between the base vector and the query vector in the vector grouping included in the grouping set, as well as the index value of the vector grouping included in the grouping set, the index value of the vector grouping included in the grouping set is compared with the operation threshold. For any vector grouping in the grouping set, if the index value of the vector grouping meets the operation threshold, the corresponding position of the vector grouping in the mark sequence is set to the first mark; if the index value of the vector grouping does not meet the operation threshold, the corresponding position of the vector grouping in the mark sequence is set to the second mark, thereby obtaining the mark sequence of the grouping set.

[0175] In order to improve processing efficiency and quickly obtain the target grouping set, in the embodiment of the present application, the first mark can be represented by the number 1, and the second mark can be represented by the number 0. In this way, after obtaining the mark sequence of the grouping set, the marks in the mark sequence can be accumulated. If the accumulated sum is 0, it means that the index values ​​of the vector groups included in the grouping set do not meet the operation threshold, and then the distance values ​​of the bottom library vectors in the vector groups included in the grouping set can be eliminated. If the accumulated sum is greater than or equal to 1, it means that there are vector groups whose index values ​​meet the operation threshold in the vector groups included in the grouping set, and then the grouping set is determined as the target grouping set, and the distance values ​​of the bottom library vectors in the vector groups included in the target grouping set are written to the second storage medium.

[0176] In order to facilitate the second data processing unit to calculate the distance value of the base vector in the vector group included in the target group set and improve the processing efficiency, in the embodiment of the present application, the tag sequence corresponding to the target group set is also written into the second storage medium. Fig.14 As shown, Fig.14 The ValidFlags obtained by the two cores are all written into the second storage medium, that is, the number of flags included in the ValidFlags in the second storage medium is M / burstsize.

[0177] After the second data processing unit reads from the second storage medium the distance values ​​of the bottom-base vectors in the vector groups included in the target group set and the tag sequence corresponding to the target group set, it can obtain, according to the tag sequence, the vector groups in the target group set whose index values ​​meet the operation threshold, and then perform a topK operation on the distance values ​​of the bottom-base vectors in the vector groups whose index values ​​meet the operation threshold, that is, sort the distance values ​​of the bottom-base vectors in the vector groups whose index values ​​meet the operation threshold, so as to obtain the top K bottom-base vectors that best match the query vector.

[0178] In an optional implementation, the index value of the vector group included in the target group set may also be written into the second storage medium. In this way, after the second data processing unit reads the distance value of the bottom-base vector in the vector group included in the target group set and the index value of the included vector group from the second storage medium, it may perform a topK operation on the distance value of the bottom-base vector in the vector group included in the target group set according to the index value, so as to obtain the first K bottom-base vectors that best match the query vector.

[0179] In order to further improve the processing efficiency, in the embodiment of the present application, the tag sequence corresponding to the target group set and the index value of the vector group included in the target group set can also be written into the second storage medium. In this way, after the second data processing unit reads the tag sequence corresponding to the target group set, the distance value of the bottom library vector in the vector group included in the target group set, and the index value of the included vector group from the second storage medium, it can obtain the first vector group in the target group set whose index value meets the operation threshold according to the tag sequence, and then perform a topK operation on the distance value of the bottom library vector in the first vector group according to the index value of the first vector group, so as to obtain the first K bottom library vectors that best match the query vector.

[0180] The process of performing topK operations on the distance values ​​of the bottom-store vectors in the vector grouping according to the index value can refer to the above description and will not be described in detail here.

[0181] The embodiment of the present application writes the tag sequence of the target group set and / or the index value of the vector group included in the target group set into the second storage medium, so that the second data processing unit can quickly sort the distance values ​​based on the index value and / or the tag sequence of the vector group, thereby improving data processing efficiency.

[0182] Based on the above, in an embodiment of the present application, when the target operation result is only the distance value that meets the operation threshold, the distance values ​​that meet the operation threshold can be sorted, and based on the sorting results, the topK operation results are obtained, that is, the top K most matching operation results are obtained from the sorting results.

[0183] When the target operation result is the distance value of the bottom-base vector in the first vector group among multiple vector groups, the distance values ​​of the bottom-base vector in the first vector group among the multiple vector groups can be directly sorted, and a topK operation result can be obtained based on the sorting result. Alternatively, a topK operation can be performed on the distance values ​​of the bottom-base vector in the first vector group according to the index values ​​of the multiple vector groups to obtain a topK operation result.

[0184] When the target operation result is the distance value of the bottom library vectors in the vector groupings included in the target grouping set, the distance values ​​of the bottom library vectors in the vector groupings included in the target grouping set can be directly sorted, and the topK operation results can be obtained based on the sorting results. The topK operation can also be performed on the distance values ​​of the bottom library vectors in the vector groupings included in the target grouping set according to the tag sequence corresponding to the target grouping set to obtain the topK operation results. The topK operation can also be performed on the distance values ​​of the bottom library vectors in the vector groupings included in the target grouping set according to the tag sequence corresponding to the target grouping set, the distance values ​​of the bottom library vectors in the vector groupings included in the target grouping set and the index values ​​of the included vector groupings to obtain the topK operation results.

[0185] In order to further improve data processing efficiency, after performing topK operations on the target operation results, the embodiment of the present application can also adjust the operation threshold according to the topK operation results.

[0186] Exemplarily, the Kth operation result may be obtained from the topK operation results, the operation threshold may be updated according to the Kth operation result, and the updated operation threshold may be rewritten into the first storage medium.

[0187] The kth operation result represents the worst result among the topK operation results, and the operation threshold is updated according to the Kth operation result, which can be to use the Kth operation result as a new operation threshold and rewrite it into the first storage medium.

[0188] like Fig.15 As shown, Fig.15 A schematic diagram of the data processing principles provided in an embodiment of the present application. Fig.15 The distance values ​​of the bottom library vectors in the vector grouping, the index values ​​of the vector grouping, and the tag sequence are all written into the second storage medium. The second data processing unit performs topK operations on the distance values ​​of the bottom library vectors in the vector grouping based on the index values ​​of the vector grouping and the tag sequence, and obtains the topK operation results, that is, the distance values ​​of the top K in the order, and calculates the current operation threshold based on the distance values ​​of the top K in the order, that is, the distance value of the current order in the Kth order, and rewrites the current operation threshold into the first storage medium. In this way, when performing the target operation, the operation result can be screened out based on the rewritten operation threshold, thereby reducing the data transfer and reducing the occupation of the write bandwidth.

[0189] In some embodiments, considering that the bottom library vector is divided into multiple bottom library vector blocks according to a set size blocksize, and the multiple bottom library vector blocks are divided into multiple rounds for data processing, the operation threshold can be updated based on the round, that is, after completing the distance operation on the bottom library vector of each round and obtaining the topK operation results of the distance value of the bottom library vector of this round, the operation threshold is updated based on the topK operation results of the distance value of the bottom library vector of this round.

[0190] In some embodiments, the operation threshold can also be updated based on the bottom-base vector block, that is, after completing the distance operation on the bottom-base vector of each bottom-base vector block and obtaining the topK operation results of the bottom-base vector of the bottom-base vector block, the operation threshold is updated based on the topK operation results of the bottom-base vector of the bottom-base vector block.

[0191] The above-listed implementations are only for the purpose of explaining the updating method of the operation threshold in the embodiments of the present application, and are not intended to limit the updating method. In some implementations, considering that the bottom library vector block is divided into multiple parts according to the number of cores, it is also possible to complete the distance operation on the bottom library vector in the part, obtain the topK operation result, and then update the operation threshold based on the topK operation result of the part, which can be set specifically according to business needs.

[0192] It should be noted that for the scenario of multi-core (AI Core) parallel computing, since the computing process usually involves concurrent computing of multiple cores, the computing threshold obtained by each core may not be calculated based on the real-time topK results, but may be calculated a short time ago. However, this fuzzy matching has no effect on the correctness of the vector retrieval problem, and also reduces the complexity of the problem and reduces unnecessary computing time consumption.

[0193] It should be noted that, in a multi-core parallel computing scenario, as long as the target computing result of at least one core is written into the second storage medium, the second data processing unit can perform topK computing on the target computing result.

[0194] In an embodiment of the present application, when the second data processing unit performs a topK operation, any topK operation in the middle is performed based on the result of the last topK operation. For example, taking the minimum topK distance as an example, it is set to write data to the second storage medium in the form of a grouping set. When the distance value of the bottom library vector in the vector grouping included in a grouping set and the index value of the included vector grouping are written to the second storage medium, the second data processing unit can read the distance value of the bottom library vector in the vector grouping included in the grouping set and the index value of the included vector grouping from the second storage medium, and then compare the index value of any vector grouping with the Kth distance in the topK distance obtained last time. If the index value of the vector grouping is greater than or equal to the Kth distance, the distance value of the bottom library vector in the vector grouping is eliminated, and the distance value of the bottom library vector in the vector grouping is not sorted, and then the index value of another vector grouping is selected from the grouping set to compare with the Kth distance.

[0195] If the index value of the vector grouping is less than the Kth distance, the index value of the vector grouping is placed in topK, the Kth distance value is updated, and then the distance value of a bottom-base vector is selected from the vector grouping for comparison with the updated Kth distance value. If it is greater than or equal to the updated Kth distance value, the distance value of the selected bottom-base vector is eliminated. If it is less than the updated Kth distance value, the distance value of the selected bottom-base vector is placed in topK, and the Kth distance value is updated again, and so on, until the distance values ​​of the bottom-base vectors in the vector grouping are compared, and then the index value of another vector grouping is selected from the grouping set for comparison with the Kth distance.

[0196] When all the vector groups included in the grouping set have completed the topK operation, the topK distance obtained by the grouping set will be used for the topK operation of the next grouping set.

[0197] The embodiment of the present application performs a topK operation according to the target operation result to obtain a topK operation result. The operation threshold is adjusted according to the topK operation result to obtain an operation threshold with high adaptability. The threshold does not need to be set manually, and data can be screened out. At the same time, the operation result is fuzzily filtered by the operation threshold, which reduces the complexity of the problem and reduces unnecessary computing time consumption.

[0198] The data processing method provided in the embodiment of the present application reduces the handling of the calculation results of the first data processing unit by transferring part of the work content of the second data processing unit to the first data processing unit, reduces the occupation of the write bandwidth of the second storage medium when the first data processing unit is running, alleviates the competition with other operations that require write bandwidth, and improves end-to-end performance.

[0199] It should be noted that, in addition to the vector retrieval scenario, in other scenarios, as long as the problem processing involves two different calculations occurring on different storage media, and the two calculations involve the flow of data, you can consider using the data processing method provided by the embodiment of the present application to move part of one calculation to the inside of another calculation to reduce data handling and improve end-to-end performance. For example, for the product recommendation scenario, the data processing method provided by the embodiment of the present application can be used. In the product recommendation scenario, the operation data may include product information of the product to be recommended and purchase behavior data of the candidate object. When making product recommendations, the first data processing unit can perform target operations on the operation data, that is, predict the interest according to the product information of the product to be recommended and the behavior data of the candidate object, and obtain the interest of the candidate object to the recommended product. Then, according to the operation threshold, the interest of the candidate object to the recommended product is screened to obtain the interest that meets the operation threshold, and then the interest that meets the operation threshold is written to the second storage medium, and then the second data processing unit can read the interest from the second storage medium to perform topK operations to obtain the target object and recommend the product to be recommended to the target object. Optionally, the first data processing unit is deployed with a product recommendation model, such as a collaborative filtering model, a matrix decomposition model, a logistic regression model, etc. Based on the product recommendation model, interest prediction is performed using product information of the product to be recommended and behavioral data of the candidate objects to obtain the candidate objects' interest in the recommended product.

[0200] Based on the same inventive concept, the embodiment of the present application further provides a computing device. In the embodiment of the present application, the computing device may be the above-mentioned AI chip. In some embodiments, the computing device may also be a processor, such as Figure 2 In some embodiments, the computing device may also be an electronic device configured with a processor and / or an AI chip. Fig.16 , Fig.16 A schematic diagram of a structure of a computing device provided in an embodiment of the present application. The computing device provided in an embodiment of the present application includes a first storage medium, a second storage medium, a first data processing unit, and a load / store unit.

[0201] The first data processing unit is used to perform target operations on the operation data in the first storage medium to obtain operation results.

[0202] The load / store unit is used to write the target operation result that meets the operation threshold in the operation results into the second storage medium.

[0203] Among them, the first data processing unit may include a matrix calculation unit, a scalar calculation unit and a vector calculation unit in the above-mentioned AI chip.

[0204] In an optional implementation manner, the operation data includes a base database vector and a query vector; and the first data processing unit is specifically used for:

[0205] A distance operation is performed on the base database vector and the query vector to obtain a distance value between the query vector and the base database vector; wherein the operation result includes the distance value.

[0206] In an optional implementation manner, the first data processing unit is specifically configured to:

[0207] Grouping the base library vectors to obtain multiple vector groups;

[0208] Performing distance operations on the query vector and the bottom-base vectors in the multiple vector groups respectively to obtain distance values ​​between the query vector and the bottom-base vectors in the multiple vector groups;

[0209] The first data processing unit is further specifically configured to:

[0210] Determine an index value of the distance values ​​of the bottom-base vectors in the plurality of vector groups, wherein the index value of the vector group is the maximum distance value or the minimum distance value among the distance values ​​of the bottom-base vectors in the vector group; wherein the operation result also includes the index values ​​of the plurality of vector groups.

[0211] In an optional implementation, the target operation result is a distance value of a bottom-base vector in a first vector group among the multiple vector groups; wherein the index value of the first vector group satisfies the operation threshold.

[0212] In an optional implementation manner, the load / store unit is further specifically configured to:

[0213] The index values ​​of the plurality of vector groups are written into a second storage medium.

[0214] In an optional implementation, the target operation result is a distance value of the base library vector in the vector group included in the target group set;

[0215] The target grouping set refers to a grouping set including vector groups whose index values ​​meet the operation threshold, and the grouping set refers to a set of vector groups.

[0216] In an optional embodiment, the load / store unit is further configured to:

[0217] Writing a tag sequence corresponding to the target group set into a second storage medium; the tag sequence is used to mark whether the index value of the vector group in the target group set meets the operation threshold; and / or,

[0218] The index values ​​of the vector groups included in the target group set are written into a second storage medium.

[0219] In an optional embodiment, the computing device further includes a second data processing unit, and the second data processing unit is configured to:

[0220] Perform topK operations according to the target operation results to obtain topK operation results;

[0221] Adjust the operation threshold according to the topK operation results.

[0222] Among them, the second data processing unit can be the AI ​​CPU in the above-mentioned AI chip.

[0223] In an optional implementation manner, the second data processing unit is specifically configured to:

[0224] According to the marking sequence, a first vector grouping meeting the operation threshold is determined; the marking sequence is used to mark whether the index value of the vector grouping meets the operation threshold; the index value of the vector grouping is the maximum distance value or the minimum distance value among the distance values ​​of the base vectors in the vector grouping;

[0225] A topK operation is performed according to the distance values ​​of the bottom-base vectors in the first vector grouping.

[0226] The computing device provided in the embodiment of the present application screens the results of the target operation according to the operation threshold, excludes the operation results that do not meet the operation threshold, and does not need to be written to the second storage medium for the operation results that do not meet the operation threshold, thereby reducing the memory access pressure on the second storage medium, thereby reducing the occupation of the write bandwidth of the second storage medium when the first data processing unit is running, which can alleviate the competition with other operations that require write bandwidth and improve end-to-end performance.

[0227] like Fig.17 As shown, the computing device provided in the embodiment of the present application can implement the following process when performing vector retrieval:

[0228] When performing vector retrieval, the load / store unit loads blocksize base vectors, batch query vectors and an operation threshold from the second storage medium to the first storage medium, the first data processing unit calculates distance values ​​for the blocksize base vectors and the batch query vectors to obtain blocksize*batch distance values, screens the blocksize*batch distance values ​​based on the operation threshold to obtain distance values ​​that meet the operation threshold, the load / store unit loads the distance values ​​that meet the operation threshold to the second storage medium, the second data processing unit reads the distance values ​​that meet the operation threshold from the second storage medium, performs topK operation on the distance values ​​that meet the operation threshold to obtain topK distance values, updates the operation threshold according to the topK distance value, the load / store unit reloads the updated operation threshold to the first storage medium, and the first data processing unit can screen the distance values ​​calculated in the next round according to the updated operation threshold.

[0229] In some embodiments, blocksize base library vectors are divided into multiple vector groups, and the size of each vector group is burstsize. The first data processing unit calculates the distance value between batch query vectors and each group of base library vectors respectively, and obtains the distance value (blocksize*batch) between batch query vectors and each group of base library vectors, and the index value (batch*blocksize / burstsize) of the distance value between batch query vectors and each group of base library vectors. The load / store unit loads the distance value of the base library vectors in the vector group whose index value meets the operation threshold to the second storage medium, and loads the index value of the distance value between batch query vectors and each group of base library vectors to the second storage medium, so that the second data processing unit performs topK operation on the distance value of the base library vectors in the vector group whose index value meets the operation threshold according to the index value, thereby improving processing efficiency.

[0230] A more detailed description of the above-mentioned computing device can be directly obtained by referring to the relevant description in the aforementioned method embodiment, which will not be repeated here.

[0231] Based on the same inventive concept, an embodiment of the present application provides an electronic device, which may specifically include a memory and a processor. The memory is used to store computer program instructions; the processor is used to execute computer program instructions to support the electronic device to implement the method described in the above embodiment.

[0232] The processor may be an NPU or a GPU. In some embodiments, the processor may also be the computing device described above. In an optional scenario, the electronic device may also include a communication interface, which may be used to receive tasks and task data, and output task processing results, etc.

[0233] An embodiment of the present application also provides a computer-readable storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processing circuit, the functions or steps in the above-mentioned data processing method are implemented.

[0234] In addition, an embodiment of the present application may also provide a chip system, which includes a processing circuit and a storage medium, wherein the storage medium stores computer program instructions, and when the computer program instructions are executed by the processing circuit, the functions or steps in the above-mentioned data processing method are implemented.

[0235] Through the description of the above implementation methods, technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0236] Each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units.

[0237] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as flash memory, mobile hard disk, read-only memory, random access memory, disk or optical disk.

[0238] The above is only a specific implementation of the embodiment of the present application, but the protection scope of the embodiment of the present application is not limited thereto, and any changes or replacements within the technical scope disclosed in the embodiment of the present application should be included in the protection scope of the embodiment of the present application. Therefore, the protection scope of the embodiment of the present application should be based on the protection scope of the claims.

Claims

1. A data processing method, characterized in that: Applied to a computing device, the computing device includes a first storage medium and a second storage medium; the method includes: Performing a target operation on the operation data in the first storage medium to obtain an operation result; The target operation result satisfying the operation threshold among the operation results is written into the second storage medium.

2. The data processing method according to claim 1, characterized in that: The operation data includes a base vector and a query vector; and the target operation is performed on the operation data in the first storage medium to obtain an operation result, including: Performing a distance operation on the base database vector and the query vector to obtain a distance value between the query vector and the base database vector; The calculation result includes the distance value.

3. The data processing method according to claim 2, characterized in that: The performing a distance operation on the base database vector and the query vector to obtain a distance value between the query vector and the base database vector includes: Grouping the base library vectors to obtain multiple vector groups; Performing distance operations on the query vector and the bottom-base vectors in the plurality of vector groups respectively to obtain distance values ​​between the query vector and the bottom-base vectors in the plurality of vector groups; The performing a target operation on the operation data in the first storage medium to obtain an operation result also includes: Determine an index value of the distance values ​​of the bottom-base vectors in the plurality of vector groups, wherein the index value of the vector group is a maximum distance value or a minimum distance value among the distance values ​​of the bottom-base vectors in the vector group; The operation result also includes index values ​​of the multiple vector groups.

4. The data processing method according to claim 3, characterized in that: The target operation result is a distance value of a bottom-base vector in a first vector group among the multiple vector groups; The index value of the first vector grouping meets the operation threshold.

5. The data processing method according to claim 4, characterized in that: The method further comprises: The index values ​​of the plurality of vector groups are written into the second storage medium.

6. The data processing method according to claim 3, characterized in that: The target operation result is the distance value of the base library vector in the vector group included in the target group set; The target grouping set refers to a grouping set including vector groups whose index values ​​satisfy the operation threshold, and the grouping set refers to a set of vector groups.

7. The data processing method according to claim 6, characterized in that: The method further comprises: Writing a marking sequence corresponding to the target group set into a second storage medium, the marking sequence being used to mark whether the index value of the vector group in the target group set meets the operation threshold; and / or, The index values ​​of the vector groups included in the target group set are written into a second storage medium.

8. The data processing method according to any one of claims 1 to 7, characterized in that: The method further comprises: Perform a topK operation according to the target operation result to obtain a topK operation result; The operation threshold is adjusted according to the topK operation result.

9. The data processing method according to claim 8, characterized in that: The performing topK operations according to the target operation result includes: Determine, according to a marking sequence, a first vector grouping that meets the operation threshold, wherein the marking sequence is used to mark whether an index value of a vector grouping meets the operation threshold, and the index value of the vector grouping is a maximum distance value or a minimum distance value among the distance values ​​of the base vectors in the vector grouping; A topK operation is performed according to the distance values ​​of the bottom-base vectors in the first vector group.

10. A computing device, characterized in that It includes a first storage medium, a second storage medium, a first data processing unit and a load / store unit; The first data processing unit is used to perform target operation on the operation data in the first storage medium to obtain the operation result; The load / store unit is used to write the target operation result that meets the operation threshold in the operation results into the second storage medium.

11. The computing device according to claim 10, characterized in that The operation data includes a base database vector and a query vector; the first data processing unit is specifically used for: Performing a distance operation on the base database vector and the query vector to obtain a distance value between the query vector and the base database vector; The calculation result includes the distance value.

12. The computing device according to claim 11, characterized in that The first data processing unit is specifically used for: Grouping the base library vectors to obtain multiple vector groups; Performing distance operations on the query vector and the bottom-base vectors in the plurality of vector groups respectively to obtain distance values ​​between the query vector and the bottom-base vectors in the plurality of vector groups; The first data processing unit is further specifically configured to: Determine an index value of the distance values ​​of the bottom-base vectors in the plurality of vector groups, wherein the index value of the vector group is a maximum distance value or a minimum distance value among the distance values ​​of the bottom-base vectors in the vector group; The operation result also includes index values ​​of the multiple vector groups.

13. The computing device according to claim 12, characterized in that The target operation result is a distance value of a bottom-base vector in a first vector group among the multiple vector groups; The index value of the first vector grouping meets the operation threshold.

14. The computing device according to claim 13, characterized in that The load / store unit is further specifically configured to: The index values ​​of the plurality of vector groups are written into the second storage medium.

15. The computing device according to claim 12, wherein: The target operation result is the distance value of the base library vector in the vector group included in the target group set; The target grouping set refers to a grouping set including vector groups whose index values ​​satisfy the operation threshold, and the grouping set refers to a set of vector groups.

16. The computing device according to claim 15, characterized in that The load / store unit is further configured to: Writing a tag sequence corresponding to the target group set into a second storage medium; the tag sequence is used to mark whether the index value of the vector group in the target group set meets the operation threshold; and / or, The index values ​​of the vector groups included in the target group set are written into a second storage medium.

17. The computing device according to any one of claims 10 to 16, characterized in that: The computing device further comprises a second data processing unit, wherein the second data processing unit is configured to: Perform a topK operation according to the target operation result to obtain a topK operation result; The operation threshold is adjusted according to the topK operation result.

18. The computing device according to claim 17, characterized in that The second data processing unit is specifically used for: Determine, according to a marking sequence, a first vector grouping that meets the operation threshold; the marking sequence is used to mark whether an index value of a vector grouping meets the operation threshold; the index value of the vector grouping is a maximum distance value or a minimum distance value among the distance values ​​of the base vectors in the vector grouping; A topK operation is performed according to the distance values ​​of the bottom-base vectors in the first vector group.

19. An electronic device, characterized in that: The electronic device comprises: a memory for storing computer program instructions; A processor, configured to execute the computer program instructions to support the electronic device to implement the method as described in any one of claims 1-9.

20. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processing circuit, the method according to any one of claims 1 to 9 is implemented.

Images