Data processing method and apparatus therefor

Abstract

A data processing method, comprising: acquiring a plurality of pieces of first data, different pieces of the first data being data accumulation results obtained by different processing cores, and the first data being floating-point numbers; by means of a first conversion technique, converting the plurality of pieces of first data into an integer type to obtain a plurality of pieces of second data; accumulating the plurality of pieces of second data to obtain a first calculation result; and, by means of a second conversion technique, converting the first calculation result into a floating-point type to obtain a second calculation result, the first conversion technique being the inverse conversion of the second conversion technique. In the present application, during accumulation of floating-point data obtained by a plurality of processing cores, to ensure the determinism of the calculation result, the floating-point data is first converted into integer data for accumulation, and then the accumulation result is converted back to the floating-point type, thus ensuring the determinism of the accumulation.

Description

Data processing method and apparatus thereof

[0001] The present application claims priority to the Chinese patent application No. 202411295344.X, filed on September 13, 2024, and entitled "A data processing method and apparatus thereof", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the field of artificial intelligence, and in particular to a data processing method and apparatus thereof. BACKGROUND

[0003] Large models, as one of the most important research directions in the current field of artificial intelligence, have accelerated the advancement of neural networks in academia and industry. It is a key step to realize artificial intelligence applications, especially deep learning applications. Large models can help us extract valuable information from complex models and solve a variety of problems such as image recognition, speech recognition, natural language processing, etc.

[0004] The training and inference of large models involve a large number of matrix and vector operators. Matrix operators are generally processed by dedicated matrix computation units, while vector operators are also processed by dedicated vector computation units. Vector operators are diverse and are the source of the non-linear capabilities of large models and an important part of large models.

[0005] The meaning of deterministic computation is to require the same operator to run multiple times, and each time the calculation result is fixed. When the operator involves floating-point number accumulation between multiple cores, the floating-point number accumulation does not have the associative property, so the result cannot be guaranteed to be deterministic.

[0006] For example, in the single-precision floating-point number (fp32) calculation format, (2^23+0.5)+0.5=2^23+0.5=2^23. However, also in the single-precision floating-point number (fp32) calculation format, 2^23+(0.5+0.5)=2^23+1, so different calculation orders will result in different calculation results.

[0007] In large model calculations, data is often divided among different cores for calculation, and eventually involves the accumulation of data on all these cores. At this time, the data of all cores will be accumulated on the same shared memory. Due to the volatility of hardware, the order of accumulation of each core cannot be controlled, so the order of accumulation will be different in different calculation processes, resulting in uncertainty in the accumulation result. SUMMARY

[0008] In a first aspect, the present application provides a data processing method, the method comprising: obtaining a plurality of first data, different first data being data accumulation results obtained by different processing cores, the first data being floating-point numbers; converting the plurality of first data into integers by a first conversion mode to obtain a plurality of second data; accumulating the plurality of second data to obtain a first calculation result; and converting the first calculation result into a floating-point type by a second conversion mode to obtain a second calculation result, wherein the first conversion mode is an inverse conversion of the second conversion mode.

[0009] The root cause of the deterministic calculation problem is the accumulation of floating-point data among multiple cores, and the accumulation of floating-point numbers does not have the commutative law and associative law in general sense, that is, if the accumulation order of floating-point data is changed, the calculation result will change. However, the accumulation of integers is different. Regardless of how the calculation order of integer accumulation is modified, it will always give the same calculation result. Therefore, in the embodiments of the present application, when accumulating floating-point data obtained by multiple processing cores, in order to ensure the determinacy of the calculation result, the floating-point data is first converted into integer data, the integer data is accumulated, and the accumulation result is converted back to the floating-point type, thereby ensuring the determinacy of the accumulation.

[0010] In addition, when the bit width of the converted integer is consistent with that of the floating-point number, the accuracy of the integer accumulation is higher than that of the floating-point number accumulation.

[0011] In a possible implementation, the different first data are data accumulation results obtained by different processing cores for data of a same index sequence number.

[0012] In a possible implementation, the obtaining of the plurality of first data comprises: obtaining the plurality of first data from a general memory (GM).

[0013] The converting the plurality of first data into integers by the first conversion mode to obtain the plurality of second data comprises:

[0014] The vector processing unit converts the plurality of first data into integers by the first conversion mode to obtain the plurality of second data.

[0015] The accumulating the plurality of second data to obtain the first calculation result comprises:

[0016] The accumulating the plurality of second data to obtain the first calculation result comprises:

[0017] The converting the first calculation result into the floating-point type by the second conversion mode to obtain the second calculation result comprises:

[0018] The vector processing unit converts the first calculation result into a floating point type through a second conversion mode to obtain a second calculation result.

[0019] In a possible implementation, the first conversion mode is configured to convert the obtained integer type to not exceed the maximum representation range of the current data type.

[0020] In a possible implementation, each of the first data can be divided by a first data and multiplied by a second value, and an obtained result is subjected to an integer operation, and the second value is a preset power of 2. It is equivalent to converting the floating point number into an integer through the idea of linear quantization.

[0021] In a possible implementation, the conversion of the first calculation result into a floating point type through the second conversion mode to obtain the second calculation result includes multiplying the first calculation result by the first data and dividing the first calculation result by the second value.

[0022] In a possible implementation, the first data is a product result of a quantity of the plurality of first data and an absolute value of a maximum data in the plurality of first data.

[0023] The scale of the conversion of the floating point number into the integer is calculated through the maximum value of the element of each index and the quantity of the element, so that the floating point number is optimally and efficiently converted into the integer, and the risk of overflow does not occur. Meanwhile, the maximum value and the quantity can be obtained in a multi-core parallel manner for each index, and no determinacy problem occurs in this step.

[0024] In a second aspect, the present application provides a data processing apparatus, and the apparatus comprises:

[0025] An acquisition module is configured to acquire a plurality of first data, different first data being data accumulation results obtained by different processing cores, and the first data being floating point numbers.

[0026] A conversion module is configured to convert the plurality of first data into integers through a first conversion mode to obtain a plurality of second data.

[0027] An accumulation module is configured to accumulate the plurality of second data to obtain a first calculation result.

[0028] The conversion module is further configured to convert the first calculation result into a floating point type through a second conversion mode to obtain a second calculation result, and the first conversion mode is an inverse conversion of the second conversion mode.

[0029] In a possible implementation, different first data are data accumulation results obtained by different processing cores for data of a same index number.

[0030] In a possible implementation, the obtaining module is specifically configured to obtain the plurality of first data from a general memory (GM); the converting module belongs to a vector processing unit; and the accumulating module belongs to the general memory.

[0031] In a possible implementation, the first converting mode is configured to convert the obtained integer type to be within a maximum representation range of a current data type.

[0032] In a possible implementation, the converting module is specifically configured to:

[0033] divide each of the first data by the first data, multiply the first data by a second value, and perform an integer operation on a result obtained, the second value being a preset power of 2.

[0034] In a possible implementation, the converting module is specifically configured to:

[0035] multiply the first calculation result by the first data, and divide the first data by the second value.

[0036] In a possible implementation, the first data is a product of a quantity of the plurality of first data and an absolute value of a maximum data in the plurality of first data.

[0037] In a third aspect, an embodiment of the present application provides a data processing apparatus, which can include a memory, a processor, and a bus system, wherein the memory is configured to store a program, and the processor is configured to execute the program in the memory to perform the method in the first aspect and any optional method.

[0038] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, and when the computer program is executed on a computer, the computer is caused to perform the method in the first aspect and any optional method.

[0039] In a fifth aspect, an embodiment of the present application provides a computer program, which, when executed on a computer, causes the computer to perform the method in the first aspect and any optional method.

[0040] In a sixth aspect, the present application provides a chip system, which includes a processor configured to support execution of a data processing apparatus to implement functions involved in the above aspects, for example, sending or processing data involved in the above method; or, information. In a possible design, the chip system further includes a memory, the memory is configured to save necessary program instructions and data for the execution device or the training device. The chip system can be composed of a chip, or can include a chip and other discrete devices. BRIEF DESCRIPTION OF DRAWINGS

[0041] FIG. 1A is a structural schematic diagram of an artificial intelligence main body framework;

[0042] FIG. 1B and FIG. 1C are schematic diagrams of application system frameworks of the present application;

[0043] FIG. 1D is a schematic diagram of an optional hardware structure of a terminal;

[0044] FIG. 2 is a structural schematic diagram of a server;

[0045] FIG. 3 is a schematic diagram of a system architecture of the present application;

[0046] FIG. 4 is a schematic diagram of a system architecture of the present application;

[0047] FIG. 5 is a schematic diagram of a data processing method provided by an embodiment of the present application;

[0048] FIG. 6A, FIG. 6B and FIG. 6C are schematic diagrams of system architectures;

[0049] FIG. 7 is a schematic diagram of a data processing method provided by an embodiment of the present application;

[0050] FIG. 8 is a schematic diagram of a data processing method provided by an embodiment of the present application;

[0051] FIG. 9A to FIG. 9D are schematic diagrams of a data processing process provided by an embodiment of the present application;

[0052] FIG. 10 is a structural schematic diagram of a data processing apparatus provided by an embodiment of the present application;

[0053] FIG. 11 is a structural schematic diagram of an execution device provided by an embodiment of the present application;

[0054] FIG. 12 is a structural schematic diagram of a training device provided by an embodiment of the present application;

[0055] FIG. 13 is a structural schematic diagram of a chip provided by an embodiment of the present application. DETAILED DESCRIPTION

[0056] The embodiments of the present application are described below in conjunction with the accompanying drawings. The terms used in the embodiment part of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0057] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art can know that, as technology develops and new scenarios appear, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0058] The terms "first", "second", and the like, as used in the specification and in the claims of the application, and in the above Description of Embodiments, unless otherwise specified, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments of the application described herein are, for example, capable of use in either order. It is therefore intended that the following claims not be limited to the described order or sequence, except where the description below claims otherwise.

[0059] The terms "substantially", "about", and like terms as used herein are used in a similarly relative sense and are intended to account for variations in measurement values or calculated values that would be recognized by those of ordinary skill in the art as being of no significance. Further, the use of "may" when describing embodiments of the application means, for example, one or more embodiments. The terms "use", "using", and "used" as used herein can be taken in their broadest possible context as being synonymous with the terms "utilize", "utilizing", and "utilized", respectively. Additionally, the term "exemplary" is intended to refer to an instance or illustration.

[0060] First, the overall workflow of the artificial intelligence system is described. Please refer to FIG. 1A, which shows a structural schematic diagram of an artificial intelligence subject framework. The following describes the artificial intelligence subject framework from two dimensions of "intelligent information chain" (horizontal axis) and "IT value chain" (vertical axis). The "intelligent information chain" reflects a series of processes from data acquisition to processing. For example, it can be a general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, intelligent execution and output. In this process, data undergoes a condensation process of "data-information-knowledge-wisdom". The "IT value chain" reflects the value brought by artificial intelligence to the information technology industry from the underlying infrastructure of human intelligence, information (provision and processing technology implementation) to the industrial ecological process of the system.

[0061] (1) Infrastructure

[0062] Infrastructure provides computing power support for artificial intelligence systems, realizes communication with the external world, and realizes support through the underlying platform. Communication with the outside world through sensors; computing power is provided by intelligent chips (CPU, NPU, GPU, ASIC, FPGA, etc. Hardware acceleration chips); the underlying platform includes distributed computing framework and network-related platform support and support, which can include cloud storage and computing, interconnection network, etc. For example, sensors and external communication obtain data, which are provided to intelligent chips in the distributed computing system provided by the underlying platform for calculation.

[0063] (2) Data

[0064] The data of the upper layer of the infrastructure is used to represent the data source in the field of artificial intelligence. Data relates to graphics, images, speech, text, and also relates to Internet of Things data of traditional devices, including business data of existing systems and sensing data such as force, displacement, liquid level, temperature, and humidity.

[0065] (3) Data processing

[0066] Data processing usually includes data training, machine learning, deep learning, search, reasoning, decision, etc.

[0067] Among them, machine learning and deep learning can model, extract, preprocess, train, etc. Symbolic and formalized intelligent information.

[0068] Reasoning refers to the process of simulating human intelligent reasoning methods in computers or intelligent systems, using formalized information to perform machine thinking and solve problems according to reasoning control strategies, and the typical function is search and matching.

[0069] Decision-making refers to the process of decision-making after intelligent information is reasoned, which usually provides functions such as classification, sorting, and prediction.

[0070] (4) General capabilities

[0071] After data is processed as mentioned above, some general capabilities can be formed based on the results of data processing, such as algorithms or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0072] (5) Intelligent products and industry applications

[0073] Intelligent products and industry applications refer to products and applications of artificial intelligence systems in various fields, which are the packaging of the overall solution of artificial intelligence, and realize the application of intelligent information decision-making products. Its application fields mainly include: intelligent terminals, intelligent transportation, intelligent medical treatment, automatic driving, smart city, etc.

[0074] The present application can be applied to the natural language processing field in the field of artificial intelligence. The following will introduce multiple application scenarios landing in products in the natural language processing field.

[0075] First, the application scenarios of the present application are introduced. The present application can be but is not limited to an application program (hereinafter can be referred to as a synthetic application program) of a generative artificial intelligence (artificial intelligence generated content, AIGC) function or a cloud service provided by a cloud-side server, etc., which will be introduced as follows:

[0076] I. Synthetic application program

[0077] The product form of the embodiment of the present application can be a synthetic application program. The synthetic application program can run on a terminal device or a server on the cloud side.

[0078] In one possible implementation, the synthetic application program can implement a data generation task based on input data (such as images, texts, audios, videos, etc.). The synthetic application program can execute the data generation task in response to the input input data (such as images, texts, audios, videos, etc.) to obtain generated data.

[0079] For example, the above data generation task can be but is not limited to:

[0080] Text generation task: various types of text content can be generated, including news reports, blog articles, product descriptions, social media posts, etc. It can generate text with logic and coherence according to the given theme and requirements.

[0081] Image generation task: images can be generated, including illustrations, artwork, design drafts, etc. It can generate image content related to the given description or keywords.

[0082] Audio generation task: voice content can be generated, including reading text, voice assistant answers, etc. It can simulate human voice characteristics and intonation to make the generated voice sound more natural.

[0083] Content summary and summary task: a large amount of text content can be read and a summary or summary can be generated. It can extract key information from the text and present it to the user in a concise manner.

[0084] Language translation task: language translation can be performed to translate text in one language into another language. It can handle multiple language pairs and provide accurate translation results.

[0085] Automatic reply and customer service: can be used to automatically reply to user's questions and provide customer service. It can understand the user's intention and give accurate answers or suggestions.

[0086] In a possible implementation, a user can open a synthetic application installed on a terminal device and input input data (such as an image, text, audio, video, etc.), and the synthetic application can generate data from the input data by the method provided in the embodiments of the present application, and present the generated data to the user (the presentation mode can be, but is not limited to, display, save, upload to the cloud side, etc.).

[0087] In a possible implementation, a user can open a synthetic application installed on a terminal device and input input data, and the synthetic application can send the input data to a server on the cloud side, the server on the cloud side generates data from the input data by the method provided in the embodiments of the present application, and returns the generated data to the terminal device, and the terminal device can present the generated data to the user (the presentation mode can be, but is not limited to, display, save, upload to the cloud side, etc.).

[0088] Next, the synthetic application in the embodiments of the present application is introduced from the functional architecture and the product architecture implementing the function respectively.

[0089] Referring to FIG. 1B, FIG. 1B is a functional architecture diagram of the synthetic application in the embodiments of the present application:

[0090] In a possible implementation, as shown in FIG. 1B, the synthetic application 102 can receive input parameters 101 (for example, containing input data) and generate generated data 103. The synthetic application 102 can be executed on (for example) at least one computer system, and includes computer code that, when executed by one or more computers, causes the computers to perform a natural language model trained by the method provided in the embodiments of the present application.

[0091] Referring to FIG. 1C, FIG. 1C is an entity architecture diagram of running the synthetic application in the embodiments of the present application:

[0092] Referring to FIG. 1C, FIG. 1C shows a system architecture diagram. The system can include a terminal 100 and a server 200. The server 200 can include one or more servers (one server is shown in FIG. 1C as an example), and the server 200 can provide a synthesis function service for one or more terminals.

[0093] The terminal 100 can be installed with a synthetic application or open a webpage related to the synthesis function. The application and the webpage can provide an interface. The terminal 100 can receive parameters input by a user on the synthesis function interface and send the parameters to the server 200. The server 200 can obtain a processing result based on the received parameters and return the processing result to the terminal 100.

[0094] It should be understood that in some optional implementations, the terminal 100 can also complete the action of obtaining a processing result based on received parameters by itself without the cooperation of the server. The embodiments of the present application are not limited.

[0095] Next, the product form of the terminal 100 in FIG. 1C is described.

[0096] The terminal 100 in the embodiments of the present application can be a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), and the like. The embodiments of the present application are not limited in this regard.

[0097] FIG. 1D shows an optional hardware structure schematic diagram of the terminal 100.

[0098] Referring to FIG. 1D, the terminal 100 can include a radio frequency unit 110, a memory 120, an input unit 130, a display unit 140, a camera 150 (optional), an audio circuit 160 (optional), a speaker 161 (optional), a microphone 162 (optional), a processor 170, an external interface 180, a power supply 190, and the like. Those skilled in the art can understand that FIG. 1D is only an example of the terminal or the multifunctional device, and does not constitute a limitation on the terminal or the multifunctional device, and can include more or fewer components than the diagram, or combine certain components, or different components.

[0099] The input unit 130 can be used to receive inputted digital or character information, and to generate key signal inputs related to user settings of the portable multifunctional device and control of functions. Specifically, the input unit 130 can include a touch screen 131 (optional) and / or other input devices 132. The touch screen 131 can collect touch operations of a user thereon or adjacent thereto (such as operations of the user using a finger, a knuckle, a stylus, or any suitable object on or adjacent to the touch screen), and drive corresponding connected devices according to a pre-set program. The touch screen can detect touch actions of the user on the touch screen, convert the touch actions into touch signals and send the touch signals to the processor 170, and can receive commands from the processor 170 and execute the commands; the touch signals at least include touch point coordinate information. The touch screen 131 can provide an input interface and an output interface between the terminal 100 and the user. In addition, the touch screen can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch screen 131, the input unit 130 can also include other input devices 132. Specifically, the other input devices 132 can include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, on-off keys, etc.), trackballs, mice, joysticks, etc.

[0100] The input device 132 can receive inputted input data, etc.

[0101] The display unit 140 can be used to display information inputted by the user or provided to the user, various menus of the terminal 100, interactive interfaces, file display, and / or playing of any kind of multimedia files. In the embodiments of the present application, the display unit 140 can be used to display interfaces of synthetic application programs, generated generated data, etc.

[0102] The storage 120 can be used to store instructions and data. The storage 120 can mainly include a storage instruction area and a storage data area. The storage data area can store various data such as multimedia files, texts, etc.; the storage instruction area can store software units such as operating systems, applications, instructions required by at least one function, etc., or their subsets, expanded sets. It can also include a non-volatile random access memory; provide the processor 170 with software and applications that include management of hardware, software, and data resources in computing processing devices, support control. It is also used for storage of multimedia files, and storage of running programs and applications.

[0103] The processor 170 is the control center of the terminal 100, connects each part of the whole terminal 100 by various interfaces and lines, executes various functions of the terminal 100 and processes data by running or executing the instructions stored in the memory 120 and calling the data stored in the memory 120, thereby overall controlling the terminal device. Optionally, the processor 170 can include one or more processing units; preferably, the processor 170 can integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application program, etc., and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 170. In some embodiments, the processor, the memory, can be implemented on a single chip, and in some embodiments, they can also be implemented on separate chips respectively. The processor 170 can also be used to generate corresponding operation control signals to send to the corresponding components of the computing processing device, read and process the data in the software, especially read and process the data and programs in the memory 120, so that each functional module therein executes the corresponding function, thereby controlling the corresponding components to act according to the requirements of the instructions.

[0104] The memory 120 can be used to store software codes related to the data processing method, and the processor 170 can execute the steps of the data processing method of the chip, or can also schedule other units (such as the above-mentioned input unit 130 and display unit 140) to realize the corresponding functions.

[0105] The RF unit 110 (optional) can be used to receive and send signals in information or communication processes. For example, after receiving the downlink information from the base station, the processor 170 processes the information. In addition, the uplink data is sent to the base station. Generally, the RF circuit includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (LNA), a duplexer, etc. In addition, the RF unit 110 can also communicate with network devices and other devices through wireless communication. The wireless communication can use any communication standard or protocol, including but not limited to Global System for Mobile Communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Messaging Service (SMS), etc.

[0106] In the embodiments of the present application, the RF unit 110 can send input data to the server 200 and receive generated data sent by the server 200.

[0107] It should be understood that the RF unit 110 is optional, which can be replaced by other communication interfaces, for example, a network interface.

[0108] The terminal 100 also includes a power supply 190 (such as a battery) for supplying power to each component. Preferably, the power supply can be logically connected to the processor 170 through a power management system, so as to realize the functions of power management, discharge management, and power consumption management through the power management system.

[0109] The terminal 100 also includes an external interface 180, which can be a standard Micro USB interface or a multi-pin connector. The external interface 180 can be used to connect the terminal 100 to other devices for communication, and can also be used to connect a charger to charge the terminal 100.

[0110] Although not shown, the terminal 100 can also include a flash, a wireless fidelity (WiFi) module, a Bluetooth module, different function sensors, etc., which will not be described here. Some or all of the methods described below can be applied in the terminal 100 as shown in FIG. 1D.

[0111] Next, the product form of the server 200 in FIG. 1C is described;

[0112] FIG. 2 provides a structural schematic diagram of a server 200. As shown in FIG. 2, the server 200 includes a bus 201, a processor 202, a communication interface 203, and a memory 204. The processor 202, the memory 204, and the communication interface 203 communicate through the bus 201.

[0113] The bus 201 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is shown in FIG. 2, but it does not mean that there is only one bus or only one type of bus.

[0114] The processor 202 can be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP), etc.

[0115] The memory 204 can include a volatile memory, such as a random access memory (RAM). The memory 204 can also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a mechanical hard drive (HDD), or a solid state drive (SSD).

[0116] The memory 204 can be used to store software codes related to the data processing method, and the processor 202 can execute the steps of the chip data processing method or schedule other units to realize the corresponding functions.

[0117] It should be understood that the terminal 100 and the server 200 described above can be centralized or distributed devices, and the processors (for example, the processor 170 and the processor 202) in the terminal 100 and the server 200 can be hardware circuits (for example, an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a general-purpose processor, a digital signal processor (DSP), a microprocessor, a microcontroller, or the like) or a combination of the hardware circuits. For example, the processor can be a hardware system with an instruction execution function, such as a CPU, a DSP, or the like, or a hardware system without an instruction execution function, such as an ASIC, an FPGA, or the like, or a combination of the hardware system without an instruction execution function and the hardware system with an instruction execution function.

[0118] It should be understood that the steps related to the model inference process in the embodiments of the present application involve AI-related operations. When performing AI operations, the instruction execution architecture of the terminal device and the server is not limited to the architecture of the processor combined with the memory described above. The system architecture provided by the embodiments of the present application will be described in detail below with reference to FIG. 5.

[0119] FIG. 5 is a schematic diagram of a system architecture provided by the embodiments of the present application. As shown in FIG. 5, the system architecture 500 includes an execution device 510, a training device 520, a database 530, a client device 540, a data storage system 550, and a data collection device 560.

[0120] The execution device 510 includes a computing module 511, an I / O interface 512, a preprocessing module 513, and a preprocessing module 514. The target model / rule 501 can be included in the computing module 511, and the preprocessing module 513 and the preprocessing module 514 are optional.

[0121] The execution device 510 can be a terminal device or a server running a synthetic application program as described above.

[0122] The data collection device 560 is configured to collect training samples. The training samples can be program files (including program codes and program input data) and the like. After collecting the training samples, the data collection device 560 stores the training samples in the database 530.

[0123] The training device 520 can train a neural network to be trained based on the training samples maintained in the database 530 to obtain the target model / rule 501.

[0124] It should be noted that, in actual application, the training samples maintained in the database 530 do not necessarily all come from the collection of the data collection device 560, but can also be received from other devices. In addition, it should be noted that the training device 520 does not necessarily train the target model / rule 501 based on the training samples maintained in the database 530, but can also obtain training samples from the cloud or other places for model training. The above description should not be regarded as a limitation of the embodiments of the present application.

[0125] The target model / rule 501 trained by the training device 520 can be applied to different systems or devices, such as the execution device 510 shown in FIG. 5. The execution device 510 can be a terminal such as a mobile phone terminal, a tablet computer, a notebook computer, an augmented reality (AR) / virtual reality (VR) device, a vehicle-mounted terminal, etc., and can also be a server, etc.

[0126] Specifically, the training device 520 can deliver the trained model to the execution device 510.

[0127] In FIG. 5, the execution device 510 is configured with an input / output (I / O) interface 512 for data interaction with external devices. The user can input data (for example, input data in the embodiments of the present application) to the I / O interface 512 through the client device 540.

[0128] The pre-processing modules 513 and 514 are used for pre-processing the input data received by the I / O interface 512. It should be understood that there can be no pre-processing modules 513 and 514 or only one pre-processing module. When there is no pre-processing module 513 and 514, the input data can be directly processed by the calculation module 511.

[0129] During the pre-processing of the input data by the execution device 510 or during the calculation process performed by the calculation module 511 of the execution device 510, the execution device 510 can call the data, code, etc. in the data storage system 550 for corresponding processing, or store the data, instructions, etc. obtained by the corresponding processing in the data storage system 550.

[0130] Finally, the I / O interface 512 provides the processing result (for example, generated data, etc.) to the client device 540, thereby providing the user.

[0131] In the case shown in FIG. 5, the user can manually give input data, which can be operated through the interface provided by the I / O interface 512. In another case, the client device 540 can automatically send input data to the I / O interface 512, and if the user's authorization is required for the client device 540 to automatically send input data, the user can set the corresponding permission in the client device 540. The user can view the results output by the execution device 510 in the client device 540, and the specific presentation form can be a specific way such as display, sound, action, etc. The client device 540 can also act as a data collection end, collect the input data input into the I / O interface 512 and the output results output from the I / O interface 512 as new sample data, and store them into the database 530. Of course, the collection can also be performed without the client device 540, and the I / O interface 512 can directly store the input data input into the I / O interface 512 and the output results output from the I / O interface 512 as new sample data into the database 530.

[0132] It is worth noting that FIG. 5 is only a schematic diagram of a system architecture provided by an embodiment of the present application, and the positional relationship between the devices, components, modules and the like shown in the figure does not constitute any limitation. For example, in FIG. 5, the data storage system 550 is an external memory relative to the execution device 510, and in other cases, the data storage system 550 can also be placed in the execution device 510. It should be understood that the above execution device 510 can be deployed in the client device 540.

[0133] From the inference side of the model:

[0134] In an embodiment of the present application, the computing module 511 of the execution device 510 can obtain the code stored in the data storage system 550 to implement the steps related to the model inference process in the embodiment of the present application.

[0135] In an embodiment of the present application, the computing module 511 of the execution device 510 can include a hardware circuit (such as an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a general-purpose processor, a digital signal processor (DSP), a microprocessor or a microcontroller, etc.), or a combination of these hardware circuits. For example, the training device 520 can be a hardware system with an execution instruction function, such as a CPU, a DSP, etc., or a hardware system without an execution instruction function, such as an ASIC, an FPGA, etc., or a combination of the hardware system without an execution instruction function and the hardware system with an execution instruction function.

[0136] Specifically, the computing module 511 of the execution device 510 can be a hardware system with an execution instruction function, and the steps related to the model inference process provided in the embodiments of the present application can be software codes stored in the memory. The computing module 511 of the execution device 510 can obtain the software codes from the memory and execute the obtained software codes to implement the steps related to the model inference process provided in the embodiments of the present application.

[0137] It should be understood that the computing module 511 of the execution device 510 can be a combination of a hardware system without an execution instruction function and a hardware system with an execution instruction function, and part of the steps related to the model inference process provided in the embodiments of the present application can also be implemented by the hardware system without an execution instruction function in the computing module 511 of the execution device 510, which is not limited here.

[0138] From the training side of the model:

[0139] In the embodiments of the present application, the training device 520 can obtain the codes stored in the memory (not shown in FIG. 5, which can be integrated in the training device 520 or deployed separately from the training device 520) to implement the steps related to the model training in the embodiments of the present application.

[0140] In the embodiments of the present application, the training device 520 can include hardware circuits (such as application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), general-purpose processors, digital signal processors (DSPs), microprocessors or microcontrollers, etc.), or combinations of these hardware circuits. For example, the training device 520 can be a hardware system with an execution instruction function, such as a CPU, a DSP, etc., or a hardware system without an execution instruction function, such as an ASIC, an FPGA, etc., or a combination of the hardware system without an execution instruction function and the hardware system with an execution instruction function.

[0141] It should be understood that the training device 520 can be a combination of a hardware system without an execution instruction function and a hardware system with an execution instruction function, and part of the steps related to the model training provided in the embodiments of the present application can also be implemented by the hardware system without an execution instruction function in the training device 520, which is not limited here.

[0142] II. Server-provided integration function cloud service:

[0143] In a possible implementation, the server can provide a service of the integration function for the end side through an application programming interface (API).

[0144] The terminal device can send relevant parameters (for example, containing input data) to the server through the API provided by the cloud, and the server can obtain a processing result (for example, generated data) based on the received parameters and return the processing result to the terminal.

[0145] The description of the terminal and the server can be described in the above embodiments, which will not be described here.

[0146] As shown in FIG. 6A, a process of using a cloud service of the integration function provided by a cloud platform is shown.

[0147] 1. Open and purchase a content review service.

[0148] 2. The user can download a software development kit (SDK) corresponding to the content review service. The cloud platform usually provides multiple development versions of the SDK for the user to select according to the needs of the development environment, such as a JAVA version of the SDK, a python version of the SDK, a PHP version of the SDK, an Android version of the SDK, and the like.

[0149] 3. The user downloads the corresponding version of the SDK to the local according to the needs, imports the SDK project to the local development environment, and configures and debugs in the local development environment. The local development environment can also develop other functions, so as to form an application that integrates the integration function type capability.

[0150] 4. During the use of the integration function type application, when the integration function is needed, the API call of the integration function can be triggered. When the application triggers the integration function, an API request is initiated to the running instance of the integration function type service in the cloud environment. The API request carries input data, and the running instance in the cloud environment processes the input data to obtain a processing result (for example, generated data).

[0151] 5. The cloud environment returns the processing result to the application, thereby completing one integration function service call.

[0152] In addition, in addition to the application program and the cloud service, the implementation form of the present application can also be a large model inference acceleration library and a large model application SDK.

[0153] In order to better understand the scheme of the embodiments of the present application, the following takes text generation as an example to briefly introduce the possible application scenarios of the embodiments of the present application in combination with FIGS. 3-4.

[0154] Figure 3 shows a natural language processing system, which includes a user device and a data processing device. The user device includes a smart terminal such as a mobile phone, a personal computer, or an information processing center. The user device is the initiator of natural language data processing, and is the initiator of language query or inquiry. Usually, the user initiates the request through the user device.

[0155] The data processing device can be a cloud server, a network server, an application server, a management server, or other devices or servers with data processing functions. The data processing device receives query statements, voices, texts, etc. from the smart terminal through an interactive interface, and performs language data processing in the form of machine learning, deep learning, searching, reasoning, decision-making, etc. through the storage memory and the data processing processor, and feeds back the processing results to the user device. The storage memory in the data processing device can be a general term, including local storage and database for storing historical data. The database can be on the data processing device or on other network servers.

[0156] In the natural language processing system shown in Figure 3, the user device can receive the user's instructions, for example, the user device can receive a piece of text input by the user, and then initiate a request to the data processing device, so that the data processing device performs natural language processing applications (such as natural language generation, text classification, text reasoning, named entity recognition, translation, etc.) on the piece of text obtained by the user device, thereby obtaining the processing results (such as predicted word results, classification results, reasoning results, named entity recognition results, translation results, etc.) of the corresponding natural language processing applications on the piece of text.

[0157] In the embodiments of the present application, the user device can receive the user's instructions, for example, the user device can receive a piece of text (such as input data) input by the user, and then initiate a request to the data processing device, so that the data processing device performs natural language processing applications (such as text synthesis, etc.) on the piece of text obtained by the user device, thereby obtaining the processing results (such as generated data, etc.) of the corresponding natural language processing applications on the piece of text.

[0158] In Figure 3, the data processing device can process the above-mentioned text data by the method provided in the embodiments of the present application.

[0159] Figure 4 shows another natural language processing system, in which the user device directly serves as the data processing device. The user device can directly receive input from the user and directly process it by the hardware of the user device itself. The specific process is similar to that of Figure 3, and reference can be made to the above description, which will not be repeated here.

[0160] FIG. 4 is a schematic diagram of a device related to natural language processing according to an embodiment of the present application.

[0161] The processor in FIG. 3 and FIG. 4 can train data / machine learn / deep learn through a neural network model or other model, and use the model (e.g., a natural language model in an embodiment of the present application, etc.) finally trained or learned by data to perform natural language processing application (e.g., program synthesis, etc.) on text data (e.g., input data text described in an embodiment of the present application), so as to obtain corresponding processing results.

[0162] Since the embodiments of the present application involve the application of a large number of neural networks, in order to facilitate understanding, the related terms and concepts related to neural networks involved in the embodiments of the present application will be introduced first.

[0163] (1) Neural network

[0164] The neural network can be composed of neural units, and the neural unit can be an operation unit with xs (i.e., input data) and intercept 1 as input. The output of the operation unit can be:

[0165] where s = 1, 2, … n, n is a natural number greater than 1, Ws is the weight of xs, b is the bias of the neural unit. f is the activation function of the neural unit, which is used to introduce non-linear characteristics into the neural network to convert the input signal in the neural unit into the output signal. The output signal of the activation function can be used as the input of the next convolution layer, and the activation function can be a sigmoid function. The neural network is a network formed by connecting a plurality of the above single neural units, that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected to the local receptive field of the previous layer to extract the features of the local receptive field, and the local receptive field can be a region composed of a plurality of neural units.

[0166] (2) Transformer layer

[0167] The neural network comprises an embedding layer and at least one transformer layer, the at least one transformer layer can be N transformer layers (N is an integer greater than 0), wherein each transformer layer comprises, in sequence, an attention layer, an add&norm layer, a feed forward layer and an add&norm layer. In the embedding layer, the current input is embedded to obtain a plurality of embedding vectors; in the attention layer, P input vectors are obtained from the previous layer of the first transformer layer, and a first input vector in the P input vectors is taken as a center, and based on the correlation between each input vector in a preset attention window range and the first input vector, an intermediate vector corresponding to the first input vector is obtained, and thus P intermediate vectors corresponding to the P input vectors are determined; in the pooling layer, the P intermediate vectors are combined into Q output vectors, and the plurality of output vectors obtained by the last transformer layer in the transformer layer are used as the feature representation of the current input.

[0168] (3) Attention mechanism

[0169] The attention mechanism simulates the internal process of biological observation behavior, that is, a mechanism that aligns internal experience and external feeling to increase the observation fineness of a part of the region, and can quickly filter out high-value information from a large amount of information using limited attention resources. The attention mechanism can quickly extract important features of sparse data, and is widely used in natural language processing tasks, especially machine translation. The self-attention mechanism is an improvement of the attention mechanism, which reduces the dependence on external information and is better at capturing the internal correlation of data or features. The essential idea of the attention mechanism can be rewritten as the following formula:

[0170] Wherein, Lx=||Source|| represents the length of Source, the formula means that the elements in Source are imagined to be composed of a series of data pairs, at this time, given an element Query in the target Target, by calculating the similarity or correlation between Query and each Key, the weight coefficient of the Value corresponding to each Key is obtained, and then the Value is weighted and summed, that is, the final Attention value is obtained. Therefore, the essence of Attention mechanism is to weight and sum the Value values of the elements in Source, and Query and Key are used to calculate the weight coefficient of the corresponding Value. Conceptually, Attention can be understood as selectively filtering out a small amount of important information from a large amount of information and focusing on these important information, ignoring a large amount of unimportant information. The focusing process is reflected in the calculation of the weight coefficient, the greater the weight, the more focused on the corresponding Value value, that is, the weight represents the importance of the information, and the Value is the corresponding information. The self-attention mechanism can be understood as intra attention. Attention mechanism occurs between the element Query in the Target and all elements in the Source. The self-attention mechanism refers to the Attention mechanism occurring between the elements in the Source or between the elements in the Target. It can also be understood as the attention mechanism of the special case of Target=Source. The specific calculation process is the same, only the calculation object changes.

[0171] (4) Natural language processing (NLP)

[0172] Natural language is human language, and natural language processing (NLP) is the processing of human language. Natural language processing is a process of systematic analysis, understanding and information extraction of text data in an intelligent and efficient way. By using NLP and its components, we can manage very large blocks of text data, or perform a large number of automated tasks, and solve a variety of problems, such as automatic summarization, machine translation (MT), named entity recognition (NER), relation extraction (RE), information extraction (IE), sentiment analysis, speech recognition, question answering and topic segmentation, etc.

[0173] (5) Pre-trained language model

[0174] A pre-trained language model is a natural language sequence encoder that encodes each word in a natural language sequence into a vector representation for prediction tasks. Its training includes two stages. In the pre-training stage, the model is trained on large-scale unsupervised text for language model tasks, learning a word representation. In the fine-tuning stage, the model uses the parameters learned in the pre-training stage for initialization, and performs a few-step training on text classification, sequence labeling and other downstream tasks, so that the semantic information obtained by pre-training can be successfully transferred to downstream tasks.

[0175] (6) Inference / deployment: the forward computation process of neural networks.

[0176] (7) Large language model (LLM): Large language model refers to natural language processing model trained on large-scale data, usually with tens of billions or hundreds of billions of parameters. These models learn the general features of language through large amounts of text data in the pre-training stage, and can then be fine-tuned on downstream tasks to adapt to specific task requirements.

[0177] (8) Matrix processing unit (Cube unit): A unit in the NPU for calculating matrix multiplication, which can only calculate matrix multiplication of the same type of data, such as integer matrix multiplied by integer matrix or floating-point matrix multiplied by floating-point matrix. Generally, the computing power of the Cube unit is strong.

[0178] (9) Vector processing unit (Vector unit): An NPU unit for calculating vector operations, which can perform a variety of vector operations, but the computing power is weaker than the Cube unit.

[0179] (10) Global memory (GM): Memory for storing data, with a large space. The data involved in the calculation of the Cube unit and the Vector unit needs to be read from the GM, but the bandwidth (reading speed) of reading data from the GM is small.

[0180] (11) L2: L2 cache (second-level cache), used to store data, with a relatively small space and high read-write speed compared to GM. At the beginning of the calculation process, the data is located in the GM, which needs to be entered into the L2 first, and then into the Cube / UB; the data from the Cube / UB can be directly entered into the L2.

[0181] (12) L1: Level 1 cache, i.e., the cache in the Cube unit, with a relatively small space and high read-write speed compared to L2.

[0182] (13) L0A / L0B / L0C: Level 0 cache, i.e., the cache in the Cube unit specifically used to store calculation data, with the smallest space and the highest read-write speed. L0A / B / C is specifically used to store the left matrix, the right matrix, and the calculation result matrix.

[0183] (14) Determinism: The same operator runs multiple times, and each time the calculation result is fixed. When the operator involves floating-point accumulation between multiple cores, the floating-point accumulation does not have the associative property, so it cannot guarantee the determinism of the result, thus posing a challenge to the operator for deterministic calculation.

[0184] (15) Multi-core accumulation: Accumulating data on multiple cores, which generally involves accumulating data on multiple vector cores in the same block of GM memory.

[0185] Large models, as one of the most important research directions in the field of artificial intelligence, have accelerated the advancement of neural networks in academia and industry. It is a key step to realize artificial intelligence applications, especially deep learning applications. Large models can help us extract valuable information from complex models and solve a variety of problems such as image recognition, speech recognition, and natural language processing.

[0186] The training and inference of large models involve a large number of matrix and vector operators, where matrix-type operators are generally processed by dedicated matrix computation units, and vector-type operators are also processed by dedicated vector computation units. Vector-type operators are diverse, are the source of the nonlinear capabilities of large models, and are a major part of large models.

[0187] The meaning of deterministic computation is to require the same operator to run multiple times, and the calculation result obtained each time is fixed. When the operator involves floating-point number accumulation between multiple cores, since floating-point number accumulation does not have associativity, the determinacy of the result cannot be guaranteed.

[0188] For example, in the calculation format of single-precision floating-point numbers (fp32), (2 23 + 0.5) + 0.5 = 2 23 + 0.5 = 2 23. However, also in the calculation format of single-precision floating-point numbers (fp32), 2 23 + (0.5 + 0.5) = 2 23 + 1, so different calculation orders will result in different calculation results.

[0189] In large model calculations, data is often divided among different cores for calculation, and eventually involves accumulation of data on all these cores. At this time, the data of all cores will be accumulated on the same shared memory. Due to the volatility of hardware, the order of accumulation of each core cannot be controlled, so the order of this accumulation will be different in different calculation processes, resulting in uncertainty of the accumulation result.

[0190] Floating-point number accumulation between multiple cores often occurs in vector-type operators such as reduce and scatter.

[0191] Next, taking the Scatter operator as an example, referring to FIG. 6B, the input data is two vectors (value, index), and it is required to accumulate the corresponding value according to the same index to obtain the result shown in FIG. 6C.

[0192] When the cores are divided, the data will not be strictly divided according to the index, but will be directly divided among the cores according to the storage order. Therefore, if no intervention is made, the same index data will inevitably be distributed to different cores, resulting in accumulation between different cores and thus causing the problem of deterministic computation.

[0193] To solve the above problem, an embodiment of the present application provides a data processing method. The data processing method of the embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0194] Referring to FIG. 7, FIG. 7 is a flowchart of a data processing method provided by an embodiment of the present application. As shown in FIG. 7, the data processing method provided by an embodiment of the present application can include steps 701-704, which are described in detail as follows.

[0195] 701. Obtain a plurality of first data, wherein different first data is a data accumulation result obtained by different processing cores, and the first data is a floating point number.

[0196] In a possible implementation, the different first data is a data accumulation result obtained by different processing cores for data of a same index number.

[0197] In a possible implementation, the input data is two vectors (value, index), wherein each core can be responsible for accumulating data of a same index number to obtain a first data. For example, one first data is a result obtained by processing core 1 for accumulating data of an index number 1, and one first data is a result obtained by processing core 2 for accumulating data of the index number 1.

[0198] Then, the plurality of first data obtained by the plurality of processing cores needs to be accumulated.

[0199] In a possible implementation, the plurality of first data can be stored in a dedicated memory, and thus the plurality of first data can be obtained from the dedicated memory (GM).

[0200] Each first data can be a floating point number, for example, the data type of the first data can be fp32 or the like.

[0201] 702. Convert the plurality of first data into integers by using a first conversion mode to obtain a plurality of second data.

[0202] In a possible implementation, the vector processing unit converts the plurality of first data into integers by using the first conversion mode to obtain the plurality of second data.

[0203] An embodiment of the present application proposes a new way of guaranteeing deterministic calculation, that is, using the accumulation of integers to guarantee deterministic calculation. The root cause of the problem of deterministic calculation is the accumulation of floating point data among multiple cores, and the accumulation of floating point numbers does not have the commutative law and associative law in general, that is, if the accumulation order of floating point data is changed, the calculation result will change. However, the accumulation of integers is different, and the accumulation of integers is deterministic. No matter how the calculation order of integer accumulation is modified, it will always give the same calculation result.

[0204] Therefore, in the embodiments of the present application, when accumulating the floating-point data obtained by the plurality of processing cores, in order to ensure the determinacy of the calculation result, the floating-point data is first converted into integer data, the integer data is accumulated, and the accumulated result is converted back to floating-point, so as to ensure the determinacy of the accumulation.

[0205] In addition, when the bit width of the converted integer is consistent with the floating-point number, the precision of the integer accumulation is higher than that of the floating-point accumulation.

[0206] For example, the precision of int32 integer accumulation is higher than that of fp32 (24-bit effective precision) accumulation. If we construct data with a very close data range, the embodiments of the present application will obtain higher calculation precision than the floating-point accumulation.

[0207] In a possible implementation, the first conversion mode is configured to convert the obtained integer to be within the maximum representation range of the current data type.

[0208] Next, how to convert the floating-point into the integer is introduced.

[0209] In a possible implementation, each of the first data can be divided by the first data and multiplied by a second value, and the obtained result is subjected to an integer operation, and the second value is a preset power of 2. It is equivalent to converting the floating-point number into the integer through the idea of linear quantization.

[0210] In a possible implementation, the first data is the product of the number of the plurality of first data and the absolute value of the maximum data in the plurality of first data.

[0211] By the maximum value of the element of each index and the number of elements, the scale of the conversion of the floating-point number into the integer is calculated, so as to ensure the optimal efficiency of the conversion of the floating-point number into the integer and to avoid the risk of overflow. Meanwhile, the maximum value and the number of elements can be calculated in parallel by each index, and the determinacy problem will not occur in this step.

[0212] 703、accumulating the plurality of second data to obtain a first calculation result;

[0213] In a possible implementation, the plurality of second data can be accumulated in the special memory to obtain the first calculation result.

[0214] For example, the absolute value maximum M of the element of the same index in the plurality of first data can be obtained i , the number N of the data of the same index i , and R i =M i *N iFor example, the upper bound R for each index can be obtained from the input data shown in Figure 8. i =M i N i (As shown in Figure 9A). Then, each data point can be transformed... And through the calculation formula The cumulative result for each index is obtained (as shown in Figure 9B).

[0215] 704. The first calculation result is converted into a floating-point type by a second conversion method to obtain a second calculation result, wherein the first conversion method is the inverse conversion of the second conversion method.

[0216] The so-called inverse transformation can be understood as a value that can be obtained as the same or almost the same value after conversion and inverse transformation.

[0217] In one possible implementation, the first calculation result can be converted into a floating-point type by the vector processing unit through a second conversion method to obtain the second calculation result.

[0218] For example, the first calculation result can be multiplied by the first data and divided by the second value.

[0219] Returning integer data as floating-point data involves using the sum of integers obtained in step two and the upper bound of each index obtained in step one (as shown in Figure 9C), calculated using the formula Res. i =I i R i / 2 31 The final calculation result is obtained (as shown in Figure 9D).

[0220] In this embodiment, during the calculation process, whether it's obtaining the maximum value of all elements in the first step or the integer accumulation in the second step, the order of the data is completely irrelevant. However, in existing technologies, i.e., index-based sorting schemes, the calculation result can be related to the order of the input data.

[0221] Referring to Figure 10, which is a schematic diagram of the structure of a data processing apparatus provided in an embodiment of this application, as shown in Figure 10, the data processing apparatus 1000 provided in this embodiment includes:

[0222] The acquisition module 1001 is used to acquire multiple first data, where different first data are the data accumulation results obtained by different processing cores, and the first data are floating-point numbers.

[0223] The conversion module 1002 is used to convert the plurality of first data into integers through a first conversion method to obtain a plurality of second data;

[0224] an accumulation module 1003, configured to accumulate the plurality of second data to obtain a first calculation result;

[0225] The conversion module 1002 is further configured to convert the first calculation result into a floating point type by a second conversion manner to obtain a second calculation result, wherein the first conversion manner is inverse conversion of the second conversion manner.

[0226] In a possible implementation, the different first data are data accumulation results obtained by different processing cores for data of a same index sequence number.

[0227] In a possible implementation, the obtaining module 1001 is specifically configured to obtain the plurality of first data from a general memory (GM); the conversion module 1002 belongs to a vector processing unit; and the accumulation module 1003 belongs to the general memory.

[0228] In a possible implementation, the first conversion manner is configured to convert an obtained integer type to be within a maximum representation range of a current data type.

[0229] In a possible implementation, the conversion module 1002 is specifically configured to:

[0230] divide each of the first data by a first data and multiply by a second value, and perform an integer operation on an obtained result, the second value being a preset power of 2.

[0231] In a possible implementation, the conversion module 1002 is specifically configured to:

[0232] multiply the first calculation result by the first data and divide by the second value.

[0233] In a possible implementation, the first data is a product result of a quantity of the plurality of first data and an absolute value of a maximum data in the plurality of first data.

[0234] Next, a terminal device provided in an embodiment of the present application is introduced. Referring to FIG. 11, FIG. 11 is a structural schematic diagram of a terminal device provided in an embodiment of the present application. The terminal device 1100 can specifically be a virtual reality (VR) device, a mobile phone, a tablet, a notebook computer, a smart wearable device, etc., which is not limited herein. Specifically, the terminal device 1100 includes a receiver 1101, a transmitter 1102, a processor 1103 and a memory 1104 (wherein the number of the processor 1103 in the terminal device 1100 can be one or more, and one processor is taken as an example in FIG. 11). The processor 1103 can include an application processor 11031 and a communication processor 11032. In some embodiments of the present application, the receiver 1101, the transmitter 1102, the processor 1103 and the memory 1104 can be connected through a bus or other means.

[0235] The memory 1104 can include a read-only memory and a random access memory, and provide the processor 1103 with instructions and data. A part of the memory 1104 can also include a non-volatile random access memory (NVRAM). The memory 1104 stores processor and operation instructions, executable modules or data structures, or a subset thereof, or an expanded set thereof, wherein the operation instructions can include various operation instructions for implementing various operations.

[0236] The processor 1103 controls the operation of the device. In a specific application, various components of the device are coupled together through a bus system, which can include a data bus, a power bus, a control bus, and a state signal bus, etc. in addition to a data bus. However, for the sake of clarity, all kinds of buses are referred to as a bus system in the figure.

[0237] The method disclosed in the embodiments of the present application can be applied to the processor 1103 or implemented by the processor 1103. The processor 1103 can be an integrated circuit chip having a signal processing capability. In the implementation process, the steps of the above method can be completed by hardware integrated logic circuits in the processor 1103 or by instructions in the form of software. The processor 1103 described above can be a general processor, a digital signal processor (DSP), a microprocessor or a microcontroller, and can further include an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. The processor 1103 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general processor can be a microprocessor or the processor can also be any conventional processor or the like. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as a hardware code processor for execution, or a combination of hardware and software modules in the code processor for execution. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register or other mature storage medium in the art. The storage medium is located in the storage 1104, and the processor 1103 reads the information in the storage 1104 and combines the hardware to complete the steps related to model training or model inference process in the above method.

[0238] The receiver 1101 can be used to receive input digital or character information, and generate signal input related to the relevant settings and function control of the execution device. The transmitter 1102 can be used to output digital or character information through the first interface; the transmitter 1102 can also be used to send instructions to the disk group through the first interface to modify the data in the disk group; the transmitter 1102 can also include a display device such as a display screen.

[0239] The server provided in the embodiments of the present application can be quite different in configuration or performance, and can include one or more central processing units (CPUs) 1212 (for example, one or more processors) and a memory 1232, one or more storage media 1230 (for example, one or more mass storage devices) storing application programs 1242 or data 1244. The memory 1232 and the storage media 1230 can be temporary or persistent storage. The programs stored in the storage media 1230 can include one or more modules (not shown in the figure), each of which can include a series of instructions operated in the server. Further, the central processing unit 1212 can be configured to communicate with the storage media 1230 and execute the series of instructions operated in the storage media 1230 on the server 1200.

[0240] The server 1200 can also include one or more power supplies 1226, one or more wired or wireless network interfaces 1250, one or more input and output interfaces 1258, or one or more operating systems 1241, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, and the like.

[0241] In the embodiments of the present application, the central processing unit 1212 is configured to perform the actions related to model training or model inference in the above embodiments.

[0242] The embodiments of the present application also provide a computer program product including a computer program for performing the steps executed by the execution device as described above, or for performing the steps executed by the training device as described above, when the computer program product is run on a computer.

[0243] The embodiments of the present application also provide a computer readable storage medium having a program for performing signal processing stored therein, which causes a computer to perform the steps executed by the execution device as described above, or causes the computer to perform the steps executed by the training device as described above, when the computer readable storage medium is run on the computer.

[0244] The execution device, the training device or the terminal device provided in the embodiments of the present application can specifically be a chip, which comprises a processing unit, for example, a processor, and a communication unit, for example, an input / output interface, a pin or a circuit, etc. The processing unit can execute computer execution instructions stored in a storage unit, so that the chip in the execution device executes the data processing method described in the above embodiments, or so that the chip in the training device executes the data processing method described in the above embodiments. Alternatively, the storage unit is a storage unit in the chip, such as a register, a cache, etc., and the storage unit can also be a storage unit outside the chip in the wireless access device, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.

[0245] Specifically, please refer to FIG. 13, which is a structural schematic diagram of a chip provided in the embodiments of the present application. The chip can be represented as a neural network processor NPU 1300, which is mounted on a host CPU (Host CPU) as a coprocessor and is assigned tasks by the Host CPU. The core part of the NPU is an operation circuit 1303, which extracts matrix data in a memory and performs multiplication operation under the control of a controller 1304.

[0246] In some implementations, the operation circuit 1303 internally includes a plurality of processing units (PEs). In some implementations, the operation circuit 1303 is a two-dimensional systolic array. The operation circuit 1303 can also be a one-dimensional systolic array or other electronic circuits capable of performing mathematical operations such as multiplication and addition. In some implementations, the operation circuit 1303 is a general-purpose matrix processor.

[0247] For example, it is assumed that there are an input matrix A, a weight matrix B and an output matrix C. The operation circuit takes corresponding data of the matrix B from the weight memory 1302 and caches it on each PE in the operation circuit. The operation circuit takes the matrix A data from the input memory 1301 and performs matrix operation with the matrix B, and the partial result or final result of the obtained matrix is saved in an accumulator 1308.

[0248] The unified memory 1306 is used to store input data and output data. The weight data is transferred to the weight memory 1302 through a direct memory access controller (DMAC) 1305. The input data is also transferred to the unified memory 1306 through the DMAC.

[0249] BIU is Bus Interface Unit, which is used for the interaction between AXI bus and DMAC and instruction fetch buffer (IFB) 1309.

[0250] The bus interface unit 1310 (Bus Interface Unit, referred to as BIU) is used for the instruction fetch buffer 1309 to obtain instructions from the external memory, and is also used for the storage unit access controller 1305 to obtain the original data of the input matrix A or the weight matrix B from the external memory.

[0251] The DMAC is mainly used to carry the input data in the external memory DDR to the unified memory 1306, or carry the weight data to the weight memory 1302, or carry the input data to the input memory 1301.

[0252] The vector calculation unit 1307 includes a plurality of operation processing units, which further process the output of the operation circuit 1303 as needed, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc. It is mainly used for non-convolution / full connection layer network calculation in neural network, such as batch normalization, pixel-level summation, upsampling of feature plane, etc.

[0253] In some implementations, the vector calculation unit 1307 can store the processed output vector to the unified memory 1306. For example, the vector calculation unit 1307 can apply a linear function; or, a nonlinear function to the output of the operation circuit 1303, such as linear interpolation on the feature plane extracted by the convolution layer, and again, for example, a vector of accumulated values, to generate an activation value. In some implementations, the vector calculation unit 1307 generates normalized values, pixel-level summation values, or both. In some implementations, the processed output vector can be used as an activation input to the operation circuit 1303, for example, for use in subsequent layers in the neural network.

[0254] The controller 1304 is connected to the instruction fetch buffer 1309, which is used to store instructions used by the controller 1304;

[0255] The unified memory 1306, the input memory 1301, the weight memory 1302, and the instruction fetch buffer 1309 are all on-chip memories. The external memory is private to the NPU hardware architecture.

[0256] The processor mentioned in any of the above can be a general central processing unit, a microprocessor, an ASIC, or one or more integrated circuits for controlling execution of the above programs.

[0257] It should be further noted that the above-described apparatus embodiments are merely illustrative, and the units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e., they can be located in one place or distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment. In addition, the connection relationship between the modules in the apparatus embodiment provided in the present application indicates that there is a communication connection between them, which can be implemented as one or more communication buses or signal lines.

[0258] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software and the necessary general hardware, and of course it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. Generally, functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure for implementing the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better embodiment. Based on this understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer's floppy disk, U disk, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., including a plurality of instructions for making a computer device (which can be a personal computer, training device, or network device, etc.) execute the methods described in various embodiments of the present application.

[0259] In the above embodiments, all or part can be realized by software, hardware, firmware or any combination thereof. When realized by software, it can be realized in the form of a computer program product in whole or in part.

[0260] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from one website, computer, training device or data center to another website, computer, training device or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be stored by the computer or a data storage device such as a training device, a data center, etc. integrated with one or more available media sets. The available media can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)), etc.

Claims

1. A data processing method, characterized by, The method comprises: obtaining a plurality of first data, different first data being data accumulation results obtained by different processing cores, the first data being floating-point numbers; converting the plurality of first data into integers by a first conversion mode to obtain a plurality of second data; accumulating the plurality of second data to obtain a first calculation result; converting the first calculation result into a floating-point type by a second conversion mode to obtain a second calculation result, wherein the first conversion mode is inverse conversion of the second conversion mode.

2. The method of claim 1, wherein, The method comprises: obtaining a plurality of first data from a special memory (GM); the vector processing unit converts the plurality of first data into integers by a first conversion mode to obtain a plurality of second data; accumulating the plurality of second data in the special memory to obtain a first calculation result; the vector processing unit converts the first calculation result into a floating-point type by a second conversion mode to obtain a second calculation result. The method comprises: multiplying the first calculation result by the first data and dividing the first calculation result by the second value.

3. The method according to claim 1 or 2, characterized in that, The first data is a product of a quantity of the plurality of first data and an absolute value of a maximum data in the plurality of first data. The first conversion mode is configured to convert the obtained integer to be within a maximum representation range of a current data type.

4. The method of claim 3, wherein, The method comprises:

5. The method according to any one of claims 1 to 4, characterized in that, dividing each first data by a first data and multiplying the first data by a second value, and performing an integer operation on the obtained result, the second value being a preset power of 2.

6. The method according to any one of claims 1 to 5, characterized in that, The device comprises:

7. The method according to any one of claims 1 to 6, characterized in that, an obtaining module configured to obtain a plurality of first data, different first data being data accumulation results obtained by different processing cores, the first data being floating-point numbers; a conversion module configured to convert the plurality of first data into integers by a first conversion mode to obtain a plurality of second data; 8. A data processing apparatus, characterized by, an accumulating module configured to accumulate the plurality of second data to obtain a first calculation result; the conversion module is further configured to convert the first calculation result into a floating-point type by a second conversion mode to obtain a second calculation result, wherein the first conversion mode is inverse conversion of the second conversion mode. The first data is a product of a quantity of the plurality of first data and an absolute value of a maximum data in the plurality of first data. The obtaining module is specifically configured to obtain a plurality of first data from a special memory (GM); the conversion module belongs to a vector processing unit, and the accumulating module belongs to the special memory. ​ 9. The apparatus of claim 8, wherein, ​ 10. The apparatus of claim 8 or 9, wherein, ​ 11. The apparatus of any one of claims 8 to 10, wherein, The first conversion mode is configured to convert the obtained integer type to not exceed the maximum representation range of the current data type.

12. The apparatus of any one of claims 8 to 11, wherein, The conversion module is specifically configured to: divide each of the first data by the first data, multiply by a second value, and perform an integer operation on the obtained result, the second value being a preset power of 2.

13. The apparatus of claim 12, wherein, The conversion module is specifically configured to: multiply the first calculation result by the first data, and divide by the second value.

14. The apparatus of claim 12 or 13, wherein, The first data is a product of the number of the plurality of first data and the absolute value of the largest data in the plurality of first data.

15. A computer storage medium, comprising, The computer storage medium stores one or more instructions, which, when executed by one or more computers, cause the one or more computers to perform the operations of the method of any one of claims 1 to 7.

16. A computer program product, characterised in that, The computer storage medium stores one or more instructions, which, when executed by one or more computers, cause the one or more computers to perform the operations of the method of any one of claims 1 to 7.

17. A system, comprising: The computer storage medium stores one or more instructions, which, when executed by one or more computers, cause the one or more computers to perform the operations of the method of any one of claims 1 to 7. The computer storage medium stores one or more instructions, which, when executed by one or more computers, cause the one or more computers to perform the operations of the method of any one of claims 1 to 7. The computer storage medium stores one or more instructions, which, when executed by one or more computers, cause the one or more computers to perform the operations of the method of any one of claims 1 to 7.

18. A chip comprising a processor, characterized in that The computer storage medium stores one or more instructions, which, when executed by one or more computers, cause the one or more computers to perform the operations of the method of any one of claims 1 to 7. The computer storage medium stores one or more instructions, which, when executed by one or more computers, cause the one or more computers to perform the operations of the method of any one of claims 1 to 7.

Images