Data output method and device, storage medium and program product

Abstract

The invention provides a data output method and device, a storage medium and a program product, and relates to the field of data processing. The data output method can be applied to the combined processing device. The combined processing device comprises a computing device, an interface device, a processing device and a storage device. The computing device is configured to execute a data output task, is mainly implemented as a single-core intelligent processor or a multi-core intelligent processor, is used for executing deep learning or machine learning computation, and can interact with the processing device through the interface device to jointly complete the data output task. The interface device is used for transmitting data and control instructions between the computing device and the processing device. As a general processing device, the processing device performs basic control including but not limited to data handling, starting and / or stopping of the computing device, and the like. The storage device is used for storing data to be processed. According to the scheme, the data output efficiency can be improved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data output method, device, storage medium, and program product. Background Technology

[0002] An AI processor is a module specifically designed to handle the large amounts of computation in AI applications. When programming an AI processor, it is often necessary to print some of the data obtained from the computation on the AI ​​processor to the screen so that programmers can debug the code or analyze the calculation results.

[0003] Currently, the common way to print data onto a screen is to have an AI processor transmit output parameters to a general-purpose processor, which then prints the data to be output onto the screen. However, the traditional method cannot specify the output content, which is not only inefficient but also lacks flexibility. Summary of the Invention

[0004] This application provides a data output method, device, storage medium, and program product to improve the diversity and flexibility of data output and increase data processing efficiency by parsing the output parameters through an artificial intelligence processor, thus avoiding the problem that general-purpose processors can only output string constants due to their inability to parse relevant parameters.

[0005] In a first aspect, this application provides a data output method applied to an artificial intelligence processor, wherein the artificial intelligence processor is provided with a memory buffer, the method comprising: receiving a data output task sent by a general-purpose processor; parsing the output parameters of the data output task to obtain basic information of the content to be output; writing the content to be output into the memory buffer according to the basic information of the content to be output; and transmitting the content to be output from the memory buffer to the general-purpose processor.

[0006] Secondly, this application provides a data output method applied to a general-purpose processor, comprising: responding to a data output instruction, requesting a memory buffer on the side of an artificial intelligence processor for a data output task to be processed; sending the data output task to the artificial intelligence processor; obtaining content to be output from the memory buffer, the content to be output being written into the memory buffer by the artificial intelligence processor after parsing the output parameters of the data output task; parsing the content to be output, and outputting the content to be output to a designated device according to the parsing result.

[0007] Thirdly, this application provides a data output device applied to an artificial intelligence processor, wherein the artificial intelligence processor is provided with a memory buffer, and the device includes:

[0008] The receiving module is used to receive data output tasks sent by the general-purpose processor;

[0009] The first parsing module is used to parse the output parameters of the data output task to obtain the basic information of the content to be output.

[0010] The writing module is used to write the content to be output into the memory buffer according to the basic information of the content to be output;

[0011] A transmission module is used to transmit the content to be output from the memory buffer to the general-purpose processor.

[0012] Fourthly, this application provides a data output device applied to a general-purpose processor, the device comprising:

[0013] The request module is used to request a memory buffer on the AI ​​processor side for the data output task to be processed in response to the data output command;

[0014] The sending module is used to send the data output task to the artificial intelligence processor;

[0015] The acquisition module is used to acquire the content to be output from the memory buffer. The content to be output is written into the memory buffer by the artificial intelligence processor after parsing the output parameters of the data output task.

[0016] The second parsing module is used to parse the content to be output and output the content to be output to the designated device according to the parsing result.

[0017] Fifthly, this application provides an electronic device, including a general-purpose processor and an artificial intelligence processor, wherein: the general-purpose processor is configured to, in response to a data output instruction, request a memory buffer on the side of the artificial intelligence processor for a data output task to be processed; and send the data output task to the artificial intelligence processor; the artificial intelligence processor is configured to receive the data output task sent by the general-purpose processor; parse the output parameters of the data output task to obtain basic information of the content to be output; and write the content to be output into the memory buffer according to the basic information of the content to be output; the general-purpose processor is further configured to retrieve the content to be output from the memory buffer; parse the content to be output; and output the content to be output to a designated device according to the parsing result.

[0018] Sixthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the method described in any of the above aspects.

[0019] In a seventh aspect, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the methods described in any of the above aspects.

[0020] The data output method, device, storage medium, and program product provided in this application, when receiving a data output task sent by a general-purpose processor, respond to the task by parsing the output parameters of the task in an artificial intelligence processor. Based on the basic information of the content to be output obtained from the parsing, the processor writes the content to be output into a pre-defined memory buffer, and then transmits the content to be output from the memory buffer to the general-purpose processor, enabling the general-purpose processor to output the content to a designated device. The artificial intelligence processor can parse parameters in the output parameters that the general-purpose processor cannot parse, thus expanding the output stream beyond string constants. In this way, by parsing the output parameters using the artificial intelligence processor, the problem of only being able to output string constants due to the general-purpose processor's inability to parse relevant parameters is avoided, improving the diversity and flexibility of data output and increasing data processing efficiency. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0022] Figure 1 This application provides a schematic diagram of the structure of an electronic device.

[0023] Figure 2 A flowchart illustrating a data output method provided in an embodiment of this application;

[0024] Figure 3 A flowchart illustrating a data output method provided in an embodiment of this application;

[0025] Figure 4 A flowchart illustrating the parsing of fmt and %s is provided for an embodiment of this application;

[0026] Figure 5 This is a schematic diagram of the partitioning of fmt provided in this embodiment;

[0027] Figure 6 A schematic diagram illustrating a process for writing content to be output into a buffer based on different rewind scenarios, provided for an embodiment of this application;

[0028] Figure 7 A schematic diagram illustrating the process of copying string parameters to a buffer in a rewind component scenario, as provided in an embodiment of this application;

[0029] Figure 8This application provides a schematic diagram illustrating the actual form of buffer writing during rewind component copying.

[0030] Figure 9 This application provides a schematic diagram of buffer maintenance under the per_ctx_per_core method, as shown in the embodiments of this application.

[0031] Figure 10 This application provides a schematic diagram of buffer maintenance under the per_ctx_shared_fifo method.

[0032] Figure 11 A flowchart illustrating a data output method provided in an embodiment of this application;

[0033] Figure 12 A signaling interaction diagram illustrating a data output method provided in an embodiment of this application;

[0034] Figure 13 A signaling interaction diagram illustrating a data output method provided in an embodiment of this application;

[0035] Figure 14A A structural block diagram of a data output device provided in an embodiment of this application;

[0036] Figure 14B A structural block diagram of a data output device provided in an embodiment of this application;

[0037] Figure 15 A structural diagram of a circuit board provided in an embodiment of this application;

[0038] Figure 16 A structural diagram of a combined processing apparatus provided in an embodiment of this application;

[0039] Figure 17 This application provides a schematic diagram of the internal structure of a single-core computing device.

[0040] Figure 18 This application provides a schematic diagram of the internal structure of a multi-core computing device.

[0041] Figure 19 This is a schematic diagram of the internal structure of another single-core computing device provided in an embodiment of this application.

[0042] The accompanying drawings have illustrated specific embodiments of this disclosure, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this disclosure to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0043] The technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, not all of them. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0044] It should be understood that the terms "first," "second," etc., in the claims, specification, and drawings of this disclosure are used to distinguish different objects, not to describe a specific order. The terms "comprising" and "including" used in the specification and claims of this disclosure indicate the presence of the described features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or sets thereof.

[0045] It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this disclosure. As used in this disclosure and claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this disclosure and claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations.

[0046] As used in this specification and claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0047] To clearly describe the technical solutions of the embodiments of this application, the terms involved in this application are first defined as follows:

[0048] Printf: refers to the formatted output function, whose main function is to output information to the output device in a specified format.

[0049] `fmt_str`: Format string. In programming, a format string is a string containing text and special placeholders that can be replaced with the values ​​of variables at runtime. Format strings are commonly used to generate dynamic text output, such as log entries and user interface displays.

[0050] GDRAM: Global Dynamic Random-Access Memory.

[0051] GPR: General Purpose Register.

[0052] IPU: Intelligent Processing Unit.

[0053] MPU: Memory Processing Unit.

[0054] NRAM: Neuron Random-Access Memory.

[0055] SRAM: Shared Random-Access Memory.

[0056] The data output method of this application embodiment can be applied to heterogeneous computer systems, thereby enabling the output of any required data from the device side of the heterogeneous computer system to a designated device. For example, a heterogeneous computer system may include a general-purpose processor and an artificial intelligence processor or graphics processing unit (GPU). The general-purpose processor can act as the host side, and the artificial intelligence processor or GPU can act as the device side. The processor on the device side has a different instruction set architecture than the general-purpose processor on the host side. For example, the processor on the device side is an artificial intelligence processor, such as... Figure 1 As shown, this application provides an electronic device 1 based on a heterogeneous computer system, including: at least one general-purpose processor 11, at least one artificial intelligence processor 12, and a memory 13. Figure 1 The example uses a general-purpose processor 11 and an artificial intelligence processor 12. The general-purpose processor 11, the artificial intelligence processor 12, and the memory 13 are connected via a bus 10. The general-purpose processor 11 can be a CPU; the structure of the artificial intelligence processor can be found in [reference needed]. Figures 18-19 The artificial intelligence processor may include an IPU (Integrated Processing Unit) and a MPU (Multi-Processing Unit). The MPU includes, but is not limited to, Global Dynamic Random Access Memory (GDRAM), Shared RAM (SRAM), Neuron RAM (NRAM), and Weight RAM (WRAM). The memory 13 stores instructions that can be executed by the general-purpose processor 11 and / or the artificial intelligence processor 12.

[0057] In heterogeneous environments, it is often necessary to print certain data obtained from computations on the device side to the screen so that programmers can debug code or analyze calculation results. Traditionally, data is printed to the screen by transmitting the device's output parameters to a general-purpose processor, which then prints the data to be output. However, this traditional method cannot specify the output content, resulting in low efficiency and poor flexibility.

[0058] In relevant embodiments, the `printf` function can print device-side variables within the software stack. The `printf` function is called in the form `printf(fmt_str, arg1, arg2…)`, where `fmt_str` points to the string `fmt`, which is the information to be output to the screen. `fmt` can contain different type labels, each corresponding to a string parameter following `fmt_str`. During actual output, the type label is replaced by the corresponding string parameter. Common type labels include `%d`, `%f`, and `%s`. The strings pointed to by the character pointers of `fmt` and the type label `%s` in the `printf` function must be string constants; outputting strings from other address spaces is not supported. In traditional techniques, the output stream of the `printf` function is limited to standard output and does not support user-specified output streams.

[0059] like Figure 2 As shown, this is a data output method in a heterogeneous environment. Here, the device side can refer to the AI ​​processor side, and the host side can refer to the general-purpose processor side. Assume the basic layers in the heterogeneous programming model, from highest to lowest, are Context (threads), Queue (queues), and Kernel (kernel functions). A context can have multiple queues, and a queue can have multiple kernels. The kernel function is the device-side entry point function, and the main device-side code is located within the kernel function. The total number of executions and the minimum number of parallel executions of the kernel function can be configured by the user. Figure 2 The content within square brackets [] represents key time points. The operation following each bracket is the printf operation performed after that time point. NA indicates no operation. For example... Figure 2 As shown, taking an AI processor as an example on the device side, the data output process based on the Printf function can include the following steps:

[0060] Step 201: Create a Context on the host side.

[0061] Step 202: Before invoking the Kernel (sending the kernel function), the host side checks whether the Kernel function contains the printf function. If so, the host-side driver allocates a device-side buffer for the printf function, and then the host side sends the Kernel function to the device side.

[0062] Specifically, when the printf function is used in a kernel function in the device-side code, the host-side driver allocates a space on GDRAM (Global Dynamic Random-Access Memory) for all kernel functions before the kernel function is executed. This space is called the printf buffer and is used to temporarily store the data to be printed. Each kernel function has its own independent space on the printf buffer, and each kernel function's independent space is divided into N records. Each record is 160 bytes in size. Every time the printf function is executed in a kernel function, the output of the currently running printf function is stored in a record. The first 8 bytes of the record are called the index, which indicates how many times the printf function has been executed in this record.

[0063] Step 203: After the Printf buffer is allocated, the Kernel function begins execution on the device side. During Kernel function execution, it reads the Printf parameters and writes them to the buffer. Specifically, the Kernel function first reads the Printf parameters and stores them in the GPR. Then, it uses the built-in variable coreId (processor core identifier) ​​to determine whether the printf function is running on the IPU (Intelligent Processing Unit) or the MPU (Memory Processing Unit). When running on the IPU, the parameters stored in the GPR are first stored in NRAM (Neuron Random-Access Memory), and then the parameters stored in NRAM are stored in the corresponding record on GDRAM. When running on the MPU, the parameters stored in the GPR are first stored in SRAM (Shared Random-Access Memory), and then the parameters stored in SRAM are stored in the corresponding record on GDRAM. Each kernel function has a fixed number of records that can be written in its dedicated memory space. Once the last record in that space is written, subsequent `printf` calls will start writing from the first record in that space, overwriting any overwritten records. The `printf` function only copies the arguments; it does not perform any other parsing of the arguments.

[0064] There is no data interaction between the host side and the device side during the execution of the kernel function.

[0065] Step 204: After the Kernel function finishes running, the host-side driver copies the data written in the printf buffer to the host side for parsing and output, then releases the allocated device-side printf buffer, and finally the host-side code prints the data to be output to the screen.

[0066] Step 205: Perform synchronization operations such as Queue sync on the host side.

[0067] Step 206: Destroy the context on the host side.

[0068] In this embodiment, the string pointed to by `fmt_str` in the `printf` function and the string pointed to by the character pointer corresponding to the type label `%s` must be string constants. Outputting strings from other memory addresses will result in an error. An example is shown below:

[0069] __mlu_device__char p_gdram[4]={'a','b','c','\0'};

[0070] __mlu_entry__void Kernel()

[0071] {

[0072] / / "abc" is a constant string and can be output normally.

[0073] printf("%s\n","abc"); / / abc

[0074] / / The string pointed to by fmt_str is located in GDRAM and is not a string constant, so it cannot be output normally.

[0075] printf(p_gdram); / / No output

[0076] The string pointed to by p_gdram is located in GDRAM and is not a string constant, so it cannot be output normally.

[0077] printf("%s\n",p_gdram); / / Segmentation fault(core dumped)

[0078] }

[0079] To address at least one of the aforementioned problems, embodiments of this application provide a data output method. When a data output task is received from a general-purpose processor, in response to the task, an artificial intelligence processor parses the output parameters in the task and writes the content to be output into a pre-defined memory buffer based on the basic information of the content to be output obtained from the parsing. Then, the content to be output is transmitted from the memory buffer to the general-purpose processor, so that the general-purpose processor outputs the content to be output to a designated device. In this way, by parsing the output parameters with the artificial intelligence processor, the problem of only being able to output string constants due to the general-purpose processor's inability to parse relevant parameters is avoided, thereby improving the diversity and flexibility of data output and improving data processing efficiency.

[0080] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Where there is no conflict between the embodiments, the following embodiments and features can be combined with each other. Furthermore, the timing of the steps in the following method embodiments is merely an example and not a strict limitation.

[0081] Please refer to Figure 3 This is a data output method according to an embodiment of the present application, which enables the parsing of output parameters by an artificial intelligence processor. This avoids the problem that general-purpose processors cannot parse relevant parameters, resulting in the output of only string constants, thereby improving the diversity and flexibility of data output and increasing data processing efficiency. This embodiment uses an artificial intelligence processor 22 as the execution end, and the method includes the following steps:

[0082] Step 301: Receive the data output task sent by the general-purpose processor.

[0083] In this step, the AI ​​processor can receive data output tasks from the general-purpose processor in real time. The AI ​​processor has a memory buffer that can be accessed by both the general-purpose processor and the AI ​​processor. This memory buffer can be a buffer requested by the general-purpose processor on the device side through a driver. For example, the device side can receive a Kernel function transmitted by the general-purpose processor. The general-purpose processor program checks whether the Kernel function contains a data output task. If it does, the general-purpose processor, through the driver, requests a memory buffer on the AI ​​processor side for the data output task. This memory buffer is used to temporarily store the output content of the data output task. The data output task can be implemented by a function with data stream output capabilities, such as the printf function.

[0084] Specifically, when creating a context on the host side, the program checks whether the kernel function contains a printf function. If it does, the host side allocates a memory buffer on the device side for the printf function through the driver. This memory buffer is used to temporarily store the output of the printf function.

[0085] Step 302: Parse the output parameters of the data output task to obtain the basic information of the content to be output.

[0086] In this step, the AI ​​processor can parse the output parameters in the received data output task to obtain basic information about the content to be output. These output parameters include, but are not limited to, string constants and string type labels. Users can specify the output stream by setting the output parameters. Through the AI ​​processor's parsing, output parameters that general-purpose processors cannot parse can be pre-parsed, thus ensuring that the final output content is not limited to string constants and improving the flexibility of data output.

[0087] Step 303: Write the content to be output into the memory buffer based on the basic information of the content to be output.

[0088] In this step, the AI ​​processor can write the content to be output into a pre-defined memory buffer based on the basic information obtained from the parsing. This allows the general-purpose processor to read the content from the memory buffer and output it to the specified device. This memory buffer can be jointly maintained by the AI ​​processor and the general-purpose processor.

[0089] Step 304: Transfer the content to be output from the memory buffer to the general-purpose processor.

[0090] In this step, the AI ​​processor can parse parameters in the output parameters that the general-purpose processor cannot parse, so that the output stream is not limited to string constants. The parsed output content is written to the allocated memory buffer, so that the general-purpose processor can read the output content from the memory buffer and output it. In this way, by parsing the output parameters by the AI ​​processor, the problem of only being able to output string constants due to the inability of the general-purpose processor to parse relevant parameters is avoided, thereby improving the diversity and flexibility of data output and improving data processing efficiency.

[0091] In one embodiment, the device can parse the target type tag in the output parameters to determine the attribute information of the string parameter corresponding to the target type tag. The basic information of the content to be output includes the attribute information of the string parameter.

[0092] In this embodiment, the output parameters include one or more type labels, such as %d, %f, and %s, which are common type labels in the printf function. %s is a string placeholder used to insert text information; %d is an integer placeholder used to insert numbers; and %f, as a placeholder, is used to insert a specific floating-point value into the string.

[0093] The target type label can refer to the type label %s. The string parameter corresponding to the target type label is the string parameter stored at the address pointed to by the character pointer corresponding to the target type label. This string parameter can include one or more of the following attribute information: the alignment of the string parameter (left or right alignment), output width, output precision, etc.

[0094] Assuming the string parameter corresponding to the address pointed to by the target type label %s is a%b, %s will be replaced with a%b in the actual output. For example, the printf function supports specifying whether the format of the relevant string parameter in the output content is left-aligned or right-aligned in the type label %s, and supports setting the width and precision of the string parameter, etc. In this disclosed solution, the artificial intelligence processor can recognize the target type label %s and parse it to obtain the attribute information of the string parameter, thereby copying and printing the output content according to the attribute information of the string parameter.

[0095] Furthermore, the AI ​​processor can sequentially scan each character in the format string of the data output task. A format string is a string containing text and special placeholders (such as type labels), which can be replaced with the values ​​of variables (such as the string parameter corresponding to a type label) at runtime. Format strings are commonly used to generate dynamic text output, describing the characteristics of the content to be output. For example... Figure 5 As shown, the format string can be fmt=12345%8.3s12345%s……12345%s12345.

[0096] If a target type tag is found in the format string, each character indicated by the target type tag is parsed sequentially to obtain the attribute information of the string parameter, including a pointer to the string parameter. Since general-purpose processors cannot read some contents in the on-chip storage area of ​​the AI ​​processor, and therefore cannot parse the type tag %s pointing to the on-chip storage area of ​​the AI ​​processor, the AI ​​processor can parse only the type tag %s. This not only allows the output content to include string parameters, no longer limited to string constants, but also saves computational resources.

[0097] Taking the `printf` function in the output task as an example, assuming the target type label is `%s`, for the format string `fmt`, the `printf` function scans each character in `fmt` sequentially. When it encounters ordinary characters other than the target type label `%s`, it skips them. After encountering the target type label `%s`, it parses each character indicated by `%s` to obtain the attribute information of the string parameter corresponding to `%s`. This attribute information includes, but is not limited to, a pointer to the string parameter corresponding to `%s`. The string parameter corresponding to `%s` can then be retrieved through the address pointed to by this pointer. When writing the content to be output to the memory buffer, `%s` in `fmt` is replaced with the corresponding string parameter.

[0098] In an optional embodiment, the attribute information of the string parameter corresponding to %s may further include one or more of the following: output flag, output width, output precision, output size, and type. The output flag of the string parameter is used to indicate the output alignment of the string parameter, such as left-aligned or right-aligned. The output width of the string parameter represents the minimum output length of the string parameter; if insufficient, spaces can be used for padding (so the string parameter may include both spaces and the string parameter itself in the final output). The output precision represents the maximum output length of the string parameter; the string parameter can be constrained by precision first and then by width. In practical scenarios, the corresponding attribute information can be recorded according to requirements.

[0099] like Figure 4 The diagram illustrates a flowchart of parsing format strings `fmt` and `%s` according to an embodiment of this application. Taking the `printf` function as an example, assuming the target type label is `%s`, for the format string `fmt`, if `fmt_str` is empty, it means the content of the format string is empty, and parsing ends. If `fmt_str` is not empty, it means the content of the format string contains at least one character. In this case, the `printf` function can be used to scan each character in the format string `fmt` sequentially. The specific scanning process is as follows:

[0100] Step 401: Determine if the current character is not '\0'. If it is, continue to determine if the current character is "%".

[0101] Step 402: If a normal character other than % is encountered during scanning, skip it and read the next character.

[0102] Step 403: If a % is detected, enter parsing mode. For example... Figure 4 As shown, the parsing process can be as follows: five fields are parsed sequentially: Flags, Width, Precision, Size, and Type. In practical scenarios, all fields except Type may be empty if the input is valid. Therefore, during the scanning process, the system jumps to the corresponding field based on the encountered character and performs the appropriate processing.

[0103] Step 404: When the Type is parsed as 's', record the relevant attribute information such as the width and precision of the string parameter to achieve targeted information recording and save computing and storage resources.

[0104] The recorded information can be used in subsequent processes. For example, when fmt = "%-8.3hs", the information recorded after scanning can be as follows: the character pointer corresponding to the current %s, the output format is left-aligned, the width is 8, and the precision is 3. Since size is invalid for %s, h does not need to be recorded. Simultaneously, based on the above attribute information, the required write size of the string parameter corresponding to the current %s in the buffer can be calculated. This attribute information will be stored in a structure for use in subsequent processes. When other types of tags (such as %d) are scanned, they can be processed as ordinary characters; other types of tags will be processed during the second parsing process on the host side.

[0105] Step 405: Skip illegal characters if the input is invalid, until the terminating character '\0' is encountered, at which point the parsing ends. After scanning, the attribute information of fmt can be obtained, such as the number of string arguments and the size of fmt. This information can be used as basic information for the output content in subsequent steps. Because the printf function parses fmt and the %s type in fmt on the device side, the string output through these two methods can reside in any valid memory location.

[0106] In one embodiment, the method of this disclosure may further include: determining the memory requirement of the content to be output based on the basic information of the content to be output; determining whether the free memory balance of the memory buffer is greater than or equal to the memory requirement of the content to be output; and if the free memory balance of the memory buffer is greater than or equal to the memory requirement of the content to be output, writing the content to be output into the memory buffer.

[0107] The foregoing Figure 2 In the corresponding implementation, data loss can easily occur. When the number of times the `printf` function is executed within the `Kernel` function exceeds the number of records, the old records will be overwritten by the newly generated records, and the old records will be lost and unable to be output. For example, when the number of times the `printf` function is executed within the `Kernel` function is N+1, the record generated by the first execution of `printf` will be overwritten by the record generated by the (N+1)th execution of `printf`, and the output of the first execution of `printf` cannot be output.

[0108] The method in this embodiment can calculate the memory write size required for the content to be output based on the basic information of the content to be output, i.e. the memory requirement of the content to be output; then compare the memory write size required for the content to be output with the free memory balance of the memory buffer. If the free memory balance of the memory buffer is greater than or equal to the memory requirement, the content to be output is written to the memory buffer, avoiding data loss caused by data overwriting and improving the accuracy of data writing.

[0109] Taking the `printf` function as an example, the device side can obtain the attribute information of the target type tag `%s` during the parsing of `fmt` and `%s`, such as the number of string parameters to be written and their write size. Based on this information, it calculates the write size required for the output content of the `printf` function. Next, the device side reads the information of the memory buffer (where the memory buffer information refers to the memory usage information of the memory buffer, which can be determined through the data write index, as detailed later). When the size of the free memory in the memory buffer is less than the write size of the content to be output, it can wait for the general-purpose processor to process the existing valid content in the buffer. After the general-purpose processor processes the content, the size of the free memory in the memory buffer will increase. When the size of the free memory in the memory buffer is greater than or equal to the write size of the content to be output, it can begin writing the content to be output into the memory buffer, thus avoiding data loss.

[0110] Optionally, the host and device sides can jointly maintain the memory buffer. For example, when the host side creates a context, the driver checks if the kernel function contains a `printf` function. If it does, the driver allocates a memory buffer for `printf` at the beginning of the context. This memory buffer is used to temporarily store the output of `printf`. Furthermore, the device side can obtain the data write index and data processing index of the memory buffer before writing the content to be output, thus gaining information about the memory buffer. Moreover, after writing the content to be output into the memory buffer, the device side can update the data write index in the memory buffer, facilitating subsequent processing by the AI ​​processor and the general-purpose processor based on the updated data write index, thus jointly maintaining the memory buffer.

[0111] In this embodiment of the disclosure, the artificial intelligence processor includes multiple cores, such as... Figure 18 As shown, the AI ​​processor includes 16 processor cores, with 4 cores forming a processor cluster. In scenarios where the device-side processor is a multi-core processor, the maintenance of the device-side memory buffer can be done in different ways. Based on these different maintenance methods, the method for determining the free memory balance of the memory buffer also differs. Specifically, there are different ways to request a memory buffer:

[0112] Method 1 is called per_ctx_per_core. In this method, each core on the device side allocates a memory buffer, called a sub-buffer. Each sub-buffer corresponds to at least one core of the AI ​​processor. Each core uses the corresponding sub-buffer to record the output content of that core. Each core maintains its corresponding sub-buffer information separately.

[0113] Method 2 is called per_ctx_shared_fifo. In this method, all cores on the device side request a shared memory buffer. All cores share a memory buffer to record their output content, and all cores jointly maintain the information of the shared memory buffer.

[0114] In Method 1, the steps for determining the free memory balance of the memory buffer may specifically include: obtaining the data write index and data processing index in the sub-buffer corresponding to the data output task; determining the free memory balance of the sub-buffer based on the data write index and data processing index, so as to obtain information about at least one sub-buffer.

[0115] like Figure 9 The diagram illustrates one method of memory buffer maintenance provided in this application. Assume the data write index in the sub-buffer is `tail`, and the data processing index in the sub-buffer is `head`. In this method, each core uses its corresponding sub-buffer for output. This minimizes the overhead of the `printf` function and guarantees sequential output within a core, but not between cores. A sub-buffer is a device-side storage space used to temporarily store output content. The sub-buffer is divided into multiple entries, and output content can occupy one or more entries. Each core maintains two variables, `tail` and `head`, to describe the buffer state. `tail` represents the index of the starting entry used by the device-side `printf` function when writing to the memory buffer, and `head` represents the index of the entry processed by the host-side driver in the memory buffer. `head` is always less than or equal to `tail`. Entries between `head` and `tail` are occupied, while the remaining entries are free. Figure 9 As shown, in the initial state, head = 0, tail = 0.

[0116] Before the device starts writing the content to be output into the memory buffer, it can read tail and head. After the device reads tail and head, head = 0 and tail = 0.

[0117] The device can calculate the starting address of the printf function's write to the memory buffer based on `tail`. It can also determine whether the free space in the memory buffer is greater than or equal to the required write size for the output content based on `tail` and `head`. After writing the output content to the memory buffer, the device updates `tail` according to the write size and synchronizes it with the host side. Figure 9 As shown, after writing an entry and updating the tail on the device side, head = 0 and tail = 1.

[0118] During kernel operation, the host-side driver periodically reads the tail and head of each core. When the head is smaller than the tail, it indicates that there is unprocessed content in the memory buffer. The host-side driver then copies this unprocessed content back to the host for processing, updates the head, and synchronizes it with the device side. Figure 9 As shown, after the host side copies back and parses an entry and updates the head, head = 1 and tail = 1.

[0119] In Method 2, the memory buffer includes a shared memory buffer, which at least one core of the AI ​​processor can use for output. In this case, the step of determining the free memory balance of the memory buffer can further include: obtaining the current data write index and data processing index in the shared memory buffer; updating the current data write index according to the number of memory entries required for the current content to be written; and determining the free memory balance of the shared memory buffer based on the updated data write index and data processing index, thereby obtaining information about the shared memory buffer.

[0120] like Figure 10 The diagram illustrates memory buffer maintenance in a second mode according to an embodiment of this application. It assumes the current data write index in the shared memory buffer is `head`, and the data processing index is `consumer`. In mode two, all cores on the device side share a single buffer for output and jointly maintain the basic information of the memory buffer. This incurs more overhead than mode one, but all cores can output in sequence. The memory buffer in mode two is also divided into many entries. All cores jointly maintain three variables: `tail`, `head`, and `consumer` to describe the buffer state. `head` represents the index of the starting entry used by the current `printf` command to write to the memory buffer on the device side; `tail` represents the index of the ending entry used by the current `printf` command to write to the memory buffer on the device side; and `consumer` represents the index of the entry processed by the host-side driver in the buffer. Figure 10 As shown, in the initial state, head = 0, tail = 0, consumer = 0.

[0121] Before the current printf function begins writing to the memory buffer, the device reads the head and updates it atomically to the head plus the number of entries needed for the current write. This is to ensure that the head read by other printf functions when writing to the memory buffer is also the index of the starting entry used by those other printf functions when writing to the memory buffer. Figure 10As shown, after the device reads tail and head, head=1, tail=0, and consumer=0.

[0122] The device side can calculate the starting address for the current printf function to write to the memory buffer based on the head. Based on the consumer and head, it can determine whether the free size of the memory buffer is greater than or equal to the size of the output content to be written. After writing to the memory buffer, the device side will update the tail to the head and synchronize it with the host side. For example... Figure 10 As shown, after writing an entry and updating the tail on the device side, head=1, tail=1, and consumer=0.

[0123] During kernel operation, the host-side driver periodically reads `tail` and `consumer`. When `consumer` is less than `tail`, it indicates unprocessed content in the memory buffer. The host-side driver then copies this unprocessed content back to the host for processing, updates `consumer`, and synchronizes it with the device side. Figure 10 As shown, after the host side copies back and parses an entry and updates the consumer, head=1, tail=1, and consumer=1.

[0124] In the above embodiments, there are several ways for the host side and the device side to jointly maintain the memory buffer, each corresponding to a different method of allocating the memory buffer, such as method one (per_ctx_per_core) and method two (per_ctx_shared_fifo). Method one (per_ctx_per_core) is suitable for situations where not all computing cores have atomic capabilities, while method two (per_ctx_shared_fifo) is suitable for situations where all computing cores have atomic capabilities. Different methods can be flexibly selected according to actual needs to improve data processing efficiency and flexibility.

[0125] In one embodiment, the step of writing the content to be output into the memory buffer may further include: determining the remaining space of a first memory region between the current write address and the end address of the memory buffer; and determining whether the remaining space of the first memory region is less than the memory requirement of the content to be output. If the remaining space of the first memory region is greater than or equal to the memory requirement of the content to be output, the content to be output is written into the first memory region. If the remaining space of the first memory region is less than the memory requirement of the content to be output, the remaining space of a second memory region starting from the start address of the memory buffer is determined. If the sum of the remaining space of the first memory region and the remaining space of the second memory region is greater than or equal to the memory requirement of the content to be output, the content to be output is written into the first memory region and the second memory region step by step. Here, the first memory region refers to the memory region between the current write address and the end address of the memory buffer, and the second memory region refers to the memory region starting from the start address of the memory buffer.

[0126] In most cases, the output of a `printf` statement is stored in a contiguous block of memory in a buffer. However, the size of the buffer is limited, and as `printf` continues to write, the address at which it writes to the buffer increases, getting closer and closer to the end of the buffer. When the remaining space in the first memory region is less than the memory required for the output, the `printf` function can first write the first part of the output to the first memory region, and then continue writing the rest of the output from the beginning of the buffer. Therefore, it's possible for the first part of the output of a `printf` statement to be stored at the end of the buffer (i.e., the first memory region) and the second part at the beginning of the buffer (i.e., the second buffer). This situation is called a rewind of the `printf` statement.

[0127] First, the method in this embodiment compares the remaining space in the first memory area with the memory requirement for writing the content to be output into the buffer to determine whether a rewind will occur. If the remaining space in the first memory area is greater than or equal to the memory requirement of the content to be output, it indicates that a rewind will not occur, and the string constants and string parameters in the content to be output can be copied normally in sequence. If the remaining space in the first memory area is less than the memory requirement of the content to be output, it indicates that a rewind has occurred. In this case, if the sum of the remaining space in the first memory area and the remaining space in the second memory area is greater than or equal to the memory requirement of the content to be output, it indicates that there is sufficient memory in the buffer. The first part of the content to be output can be written into the first memory area first, and the remaining part of the output content will continue to be written into the second memory area starting from the beginning address of the buffer to ensure the accuracy of the output content.

[0128] In one embodiment, if a rewind is determined to have occurred on the device side, and the sum of the remaining space in the first memory area and the remaining space in the second memory area is greater than or equal to the memory requirement of the content to be output, the method of this disclosure can write the content to be output into the first memory area and the second memory area in stages. Specifically, the method of this disclosure embodiment may include:

[0129] If the sum of the remaining space in the first memory area and the remaining space in the second memory area is greater than or equal to the memory requirement of the content to be output, the type of the starting character to be written in the second memory area is determined. The type of the starting string can be a string constant or a string parameter; this starting character is the character at the rewind location. In this embodiment, when the content to be output is rewinded, the accuracy of the output content is further ensured by classifying whether the rewind occurs in a string constant or a string parameter.

[0130] like Figure 5 As shown, `fmt` can be divided into at least one group, and the content of each group can be a string constant or a string parameter. In this embodiment, the group containing the initial write character can be designated as the rewind group, and the rewind group can be classified according to whether its content is a string constant or a string parameter, and different rewind groups can be processed accordingly. Specifically, if the initial write character is a string constant, it indicates that rewind occurs within a string constant. The string constants appearing consecutively before and after the initial write character are sequentially written to the first memory area and the second memory area, i.e., a two-step method is used to copy the string constants in the rewind group (e.g., ...). Figure 8 (As shown in Case 1). Here, the rewind group can refer to the FMT group containing the character at the rewind location. For FMT grouping principles, please refer to [link to relevant documentation]. Figure 5 The illustrated embodiment. Specifically, the method of copying string constants in a two-step process using the rewind group can be as follows: first, copy the first part of the string constants in the rewind group to the remaining space in the buffer, and then copy the second part of the string constants in the rewind group starting from the beginning address of the buffer.

[0131] If the type of the initial character to be written is a string parameter, meaning the content of the rewind group is a string parameter, then the content pointed to by the target string parameter corresponding to the initial character is written step-by-step to the first and second memory areas according to the number of string parameters corresponding to the target type label, the output flag, and the output width. If the type of the initial character to be written is a string parameter, it means that rewind occurs for string parameters. The content pointed to by the target string parameter corresponding to the initial character is written step-by-step to the first and second memory areas. That is, the content pointed to by the string parameter is copied to the memory buffer according to the number of string parameters, alignment, output width, etc., of the rewind group. The number of string parameters corresponding to the target type label, the output flag, and the output width are used. The output flag of the string parameter indicates the output alignment of the string parameter, such as left or right alignment. The output width of the string parameter represents the minimum output length of the string parameter; spaces can be used to pad if necessary.

[0132] Specifically, if the type of the initial character to be written is a string parameter, the output alignment of the target string parameter is determined based on the output flag of the target string parameter corresponding to the initial character. The output alignment of the target string parameter can be left-aligned or right-aligned. In this embodiment, the target string parameter refers to the string parameter of the rewind group. If the type of the character rewinding is a string parameter, the number of output spaces required for the string parameter of the rewind group can be further determined based on the alignment and output width of the string parameter.

[0133] If the string parameter in the rewind group is right-aligned, the required spaces are first copied to the first and / or second memory areas based on the number of string parameters and the number of spaces needed. Then, the string parameter itself is copied to the first and / or second memory areas. This ensures not only the accuracy of data writing but also that the output content is formatted according to the specified format, enriching the diversity of data output. Specifically, if the target string parameter is right-aligned, the current remaining space in the first memory area is determined before the target string parameter is written. Based on the number of target string parameters and the output width, the number of output spaces required by the target string parameter is determined. If the current remaining space is less than the number of output spaces and is not zero, the output spaces of the target string parameter are written to the first and second memory areas in two steps, and the target string parameter itself is written to the second memory area. If the current remaining space is equal to the number of output spaces, the output spaces of the target string parameter are written to the first memory area, and the target string parameter itself is written to the second memory area. If the current margin is greater than the number of output spaces, write the output spaces of the target string parameter to the first memory area, and write the target string parameter itself to the first memory area and the second memory area in stages.

[0134] If the string parameter in the rewind group is left-aligned, further, based on the number of string parameters and the required number of spaces, first copy the string parameter itself to the first and / or second memory areas, and then copy the required spaces to the first and / or second memory areas as needed. This not only ensures the accuracy of data writing but also guarantees that the output content is output in the specified format, enriching the diversity of data output. Specifically, if the output alignment of the target string parameter is left-aligned, determine the write size of the target string parameter itself. If the current margin is zero, write the target string parameter itself and the output spaces sequentially to the second memory area. If the current margin is not zero and the current margin is less than the write size of the target string parameter itself, write the target string parameter itself step by step to the first and second memory areas, and write the output spaces of the target string parameter to the second memory area. If the current margin is equal to the write size of the target string parameter itself, write the target string parameter itself to the first memory area, and write the output spaces of the target string parameter to the second memory area. If the current margin is greater than the write size of the target string parameter itself, write the target string parameter itself to the first memory area, and write the output spaces of the target string parameter to the first memory area and the second memory area in stages.

[0135] In the process of writing the content to be output into the memory buffer, the content to be output can be pre-grouped, and the data can be written according to whether the rewind occurs in a string constant or a string parameter, based on the grouping, to avoid data loss and improve the accuracy of data writing.

[0136] like Figure 5 The diagram illustrates a partitioning of the fmt in this embodiment. The fmt is grouped according to the string type %s. Each %s string is grouped with the preceding string constant and with the last string constant in another group. These are designated as the first group (first_gruop), middle group (middle_gruop), and last group (last_gruop). Note that the string constants in each group may be empty. When there is no %s string in the fmt, the fmt only contains the last group. For example, %8.3s represents the width and 3 represents the precision.

[0137] Based on the above classification of FMT, rewind can be divided into four cases:

[0138] (1) The number of string parameters is not 0, and the first group causes rewind.

[0139] (2) The number of string parameters is not 0, and the middle group causes rewind.

[0140] (3) The number of string parameters is not 0, and the tail group causes rewind.

[0141] (4) The number of string parameters is 0, and the tail group causes rewind.

[0142] When rewind occurs in cases (1) and (2), it can be further divided into two cases: rewind caused by a string constant within the group and rewind caused by a string parameter within the group. Different rewind cases are handled accordingly. The main cases involved in writing to the buffer will be introduced below.

[0143] like Figure 6 As shown in the illustration, this application provides a flowchart illustrating the process of writing content to be output into a buffer based on different rewind scenarios. This flowchart is used to elaborate and explain in detail the scheme for writing content to be output into a memory buffer involved in the aforementioned embodiments. Figure 5 Taking the grouping in the example, the method includes the following steps:

[0144] Step 601: Write the output content to the memory buffer.

[0145] Step 602: Determine if a rewind has occurred. If yes, proceed to step 603; otherwise, proceed to step 604.

[0146] Step 603: Determine the location of the rewind. If the rewind occurs in the first or middle group, proceed to step 605. If the rewind occurs in the last group, proceed to step 609.

[0147] Step 604: All groups copy string constants normally, all groups copy string parameters normally, end.

[0148] Step 605: Determine if a string constant caused the rewind. If so, proceed to step 606; otherwise, proceed to step 607.

[0149] Step 606: The rewind group copies the string constant in two steps, and the rewind group copies the string parameters normally. Then proceed to step 608. Here, the rewind group refers to the fmt group where the character at the rewind position is located.

[0150] Step 607: The rewind group copies string constants normally, and the rewind group copies string parameters according to different cases, and then proceeds to step 608.

[0151] Step 608: Copy the string constants normally for the remaining groups, copy the string parameters normally for the remaining groups, and then end.

[0152] Step 609: Determine if the number of string parameters is not equal to 0. If yes, proceed to step 610; otherwise, proceed to step 612.

[0153] Step 610: The rewind group (tail group) copies the string constants in two steps, and then proceeds to step 611.

[0154] Step 611: Copy the string constants normally for the remaining groups, copy the string parameters normally for the remaining groups, and then end.

[0155] Step 612: Copy the string constant in two steps using the rewind group (tail group), then finish.

[0156] One method for copying a string constant normally is to copy the string constant from its address space to the corresponding location in the buffer.

[0157] A method for copying string parameters normally in any group can be to first determine whether the string parameter format is right-aligned or left-aligned. When spaces are required to ensure alignment, for right alignment, write the spaces first and then copy the string parameter itself; for left alignment, copy the string parameter itself first and then write the spaces. When copying the string parameter itself, it can be divided into multiple segments based on the % symbol within the string parameter and copied segment by segment. The % symbol within the string parameter itself can be written as %%, in the buffer. This is to prevent the host from misinterpreting the % symbol within the string parameter itself as some kind of parameter when parsing the output. This improves data precision.

[0158] The rewind group can be used to copy string constants in two steps. First, the first part of the string constant in the rewind group can be copied to the remaining space in the buffer. Then, the second part of the string constant in the rewind group can be copied starting from the beginning address of the buffer.

[0159] In an optional embodiment, the rewind component copies string parameters depending on the situation, and can be handled on a case-by-case basis.

[0160] To further clarify the scheme for copying string parameters to the buffer in the rewind component case in the foregoing embodiments, please refer to [link to relevant documentation]. Figure 7 and Figure 8 Corresponding implementation examples. For example... Figure 7 The diagram shown is a schematic representation of a process for copying string parameters to a memory buffer in a rewind component configuration, according to an embodiment of this application. Figure 8 This is a schematic diagram illustrating the actual writing of the buffer during the copying of a rewind component, as provided in an embodiment of this application. Figure 8 Taking the first group of rewinds as an example, assuming that the string parameter corresponding to the address space pointed to by the pointer of %s is "a%b", fmt starts writing from the 8th byte from the end of the buffer (the starting address of fmt is the 8th byte from the end of the buffer, that is, the remaining space of the first memory area includes 8 bytes). The starting address of the buffer and the memory area after it are called the second memory area. Figure 8 In this text, a "□" represents a space.

[0161] Case 1: If the type of the initial character to be written is a string constant, write the string constants that appear consecutively before and after the initial character to the first memory area and the second memory area in two steps.

[0162] exist Figure 8 In the example corresponding to Case 1, the input format is: fmt = "123456789%s". The string constant "9" is rewinded, and the string constant "12345678" is directly written to the first memory area. The string constant "9" is written to the second memory area from the starting address of the buffer. The remaining string parameters "a%%b" are written to the second memory area in sequence.

[0163] If the type of the initial character to be written is a string parameter, the output alignment of the target string parameter is first determined based on the output flag of the target string parameter corresponding to the initial character to be written. For example, it is determined whether the string parameter format is right-aligned. If it is right-aligned, the current space before_rewind_size of the first memory area is compared with the number of output spaces space_size, which may include the following triggering conditions:

[0164] Case 2: Right alignment, before_rewind_size < space_size. Copy the spaces in two steps, and then copy the string parameter itself. Here, before_rewind_size represents the current remaining amount in the first memory area of the buffer before copying the rewind group string parameter; space_size represents the number of output spaces in the string parameter.

[0165] For example, the calculation method of the number of spaces is as follows: Suppose fmt = "1234%7s", and assume that the string parameter corresponding to the address space pointed to by the pointer of %s is "a%b", and the content actually written into the buffer is "a%%b". Since the number of bytes of "a%b" is 3 and its output width is 7, the calculation method of the corresponding space_size can be 7 - 3 = 4.

[0166] In Figure 8 the corresponding example of Case 2, the input form: fmt = "1234%8s", before_rewind_size = 4, space_size = 5. The spaces of the string parameter undergo rewind. The string constant "!234" is directly written into the first memory area, the first 4 spaces of the string parameter are written into the first memory area, the 5th space of the string parameter is written into the second memory area from the starting address of the buffer, and the remaining string parameter "a%%b" is written into the second memory area in sequence.

[0167] Case 3: Right alignment, before_rewind_size = space_size. First copy the spaces, and then copy the string parameter itself.

[0168] In Figure 8 the corresponding example of Case 3, the input form: fmt = "1234%7s", before_rewind_size = 4, space_size = 4. The spaces exactly fill the buffer. The string constant "1234" is directly written into the first memory area, the 4 spaces of the string parameter are written into the first memory area, and the remaining string parameter "a%%b" is written into the second memory area from the starting address of the buffer.

[0169] Case 4: Right alignment, before_rewind_size > space_size. Further judge the specific situation of rewind. If rewind occurs after the second % in the string parameter, first copy the spaces, and then copy the string parameter itself in different cases.

[0170] In Figure 8Example corresponding to Case 4: Input format: fmt = "1234%4s", before_rewind_size = 4, space_size = 1. The string parameter itself is copied step by step. The string constant "1234" is directly written to the first memory area, one space of the string parameter is written to the first memory area, the string parameter "a%%" is written to the first memory area, and the remaining string parameter "b" is written to the second memory area from the starting address of the buffer.

[0171] Case 5: Right alignment, before_rewind_size > space_size, rewind occurs after the first % in the string parameter. First copy the spaces, then copy the string parameter itself depending on the case.

[0172] exist Figure 8 Example corresponding to Case 5: Input format: fmt = "123456%s", before_rewind_size = 2, space_size = 0. The string parameters themselves are copied step by step. The string constant "123456" is directly written to the first memory area. No spaces are needed in the string parameters. The string parameter "a%" is written to the first memory area. The remaining string parameter "%b" is written to the second memory area from the starting address of the buffer.

[0173] Case 6: Right alignment, before_rewind_size > space_size, rewind occurs after other characters in the string parameter. First copy the spaces, then copy the string parameter itself depending on the case.

[0174] exist Figure 8 Example corresponding to Case 6: Input format: fmt = "1234567%s", before_rewind_size = 1, space_size = 0. A rewind occurs after 'a' in the string parameter. The string parameter itself is copied step by step. The string constant "1234567" is directly written to the first memory area. No spaces are needed in the string parameter. The string parameter "a" is written to the first memory area. The remaining string parameter "%%b" is written to the second memory area from the starting address of the buffer.

[0175] like Figure 7 and Figure 8 As shown, if the string parameter is left-aligned, comparing the current remaining space of the first memory area before_rewind_size with the memory requirement of the content to be output real_str_arg_size + per_sign_num can include the following triggering conditions:

[0176] Case 7: Left alignment, before_rewind_size = 0. First, copy the string argument itself, and then copy the space.

[0177] Figure 8 For the example corresponding to Case 7, the input form is: fmt = "12345678%-4s", before_rewind_size = 0. Directly write the string constant "12345678" into the first memory area, write the string argument "a%%b" from the starting address of the buffer into the second memory area, and write the 1 space required by the string argument into the second memory area.

[0178] Case 8: Left alignment. If before_rewind_size!= 0, further judge the situation of rewind. If before_rewind_size < real_str_arg_size + per_sign_num, rewind occurs after the second % in the string argument. First, copy the string argument itself in different cases, and then copy the space. Here, real_str_arg_size: represents the size of the string argument itself. per_sign_num: represents the number of % in the string argument. real_str_arg_size + per_sign_num is the writing size required by the string argument itself in the buffer, that is, the memory requirement for the content to be output.

[0179] Figure 8 For the example corresponding to Case 8, the input form is: fmt = "12345%-s", before_rewind_size = 3, real_str_arg_size + per_sign_num = 4. Directly write the string constant "12345" into the first memory area, write the string argument "a%%" into the first memory area, and write the remaining string argument "b" from the starting address of the buffer into the second memory area.

[0180] Case 9: Left alignment, before_rewind_size!= 0, before_rewind_size < real_str_arg_size + per_sign_num, rewind occurs after the first % in the string argument. First, copy the string argument itself in different cases, and then copy the space.

[0181] Figure 8For the example corresponding to Case 9, the input form is: fmt = "123456%-s", before_rewind_size = 2, real_str_arg_size + per_sign_num = 4. The string constant "123456" is directly written into the first memory area, the string argument "a%" is written into the first memory area, and the remaining string argument " %b" is written into the second memory area starting from the buffer start address.

[0182] Case 10: Left alignment, before_rewind_size!= 0, before_rewind_size < real_str_arg_size + per_sign_num, rewind occurs after other characters in the string argument. First, copy the string argument itself in different cases, and then copy the spaces.

[0183] Figure 8 For the example corresponding to Case 10, the input form is: fmt = "1234567%-4s", before_rewind_size = 1, real_str_arg_size + per_sign_num = 4, rewind occurs after 'a' in the string argument. The string constant "1234567" is directly written into the first memory area, the string argument "a" is written into the first memory area, and the remaining string argument " %%b" is written into the second memory area starting from the buffer start address.

[0184] Case 11: Left alignment, before_rewind_size = real_str_arg_size + per_sign_num. First, copy the string argument itself, and then copy the spaces.

[0185] Figure 8 For the example corresponding to Case 11, the input form is: fmt = "1234%-8s", before_rewind_size = 4, real_str_arg_size + per_sign_num = 4. The string constant "1234" is directly written into the first memory area, the string argument "a%%b" is written into the first memory area, and the 4 spaces of the string argument are written into the second memory area starting from the buffer start address.

[0186] Case 12: Left alignment, before_rewind_size > real_str_arg_size + per_sign_num. First, copy the string argument itself, and then copy the spaces in two steps.

[0187] Figure 8Example corresponding to case 12, input format: fmt = "123%-8s", before_rewind_size = 5, real_str_arg_size + per_sign_num = 4, directly write the string constant "123" into the first memory area, write the string parameter "a%%b" into the first memory area, write the first space of the string parameter into the first memory area, and write the last 4 spaces of the string parameter into the second memory area from the starting address of the buffer.

[0188] Please refer to Figure 11 This is a data output method according to an embodiment of the present application. It aims to improve the diversity and flexibility of data output and increase data processing efficiency by parsing the output parameters using an artificial intelligence processor, thus avoiding the problem of only being able to output string constants due to the inability of a general-purpose processor to parse relevant parameters. This embodiment uses a general-purpose processor 21 as the execution end, and the method includes the following steps:

[0189] Step 1101: In response to the data output command, request a memory buffer on the device side for the data output task to be processed.

[0190] In this step, the data output instruction can be triggered by the user or by specific conditions, such as when the general-purpose processor detects a data output function. In response to this instruction, the general-purpose processor requests a memory buffer from the AI ​​processor side for the data output task. Both the general-purpose processor and the AI ​​processor can access this memory buffer. The buffer is used to temporarily store the output content of the data output task.

[0191] Taking a general-purpose processor as the host side and an AI processor as the device side as an example, the host side can check whether the kernel function contains the `printf` function when creating the context. If the `printf` function is present, the host-side driver will allocate a memory buffer for `printf` at the beginning of the context. The memory buffer is used to temporarily store the output of `printf`.

[0192] In one embodiment, step 1101 may specifically include: requesting a memory buffer on the AI ​​processor side for the thread to which the data output task belongs.

[0193] In this application, when a user calls a kernel function containing the `printf` function multiple times, if each call involves allocating, copying, and releasing memory on the device side, the numerous memory operations would incur significant time overhead. Furthermore, the memory occupied by these kernel functions on the device side would not be released until the kernel functions using `printf` finished executing. When there are many kernel functions using `printf`, this could lead to large spikes in memory usage and situations where device-side storage space cannot be allocated. This application addresses this by allocating memory buffers for data output tasks at the thread level. For example, the `printf` function only allocates buffers during the context creation phase, changing the memory management granularity from the kernel level to the context level. When `printf` is called within a context, a pre-defined memory buffer is allocated only at the beginning of the context, regardless of the number of kernel functions calling `printf`. This not only reduces the number of memory operations, lowering time overhead, but also keeps memory overhead to less than 0.1% of total memory usage, thus reducing overall memory consumption.

[0194] In this embodiment, the host side can check whether the kernel function contains a printf function. If it does, the driver can allocate a memory buffer for the printf function, which is used to temporarily store the output of printf.

[0195] There are different ways to allocate buffers. One method is called per_ctx_per_core. In this method, the general-purpose processor can allocate a sub-buffer of a preset size separately for each core of the AI ​​processor. Each core uses the corresponding sub-buffer for output, and each core maintains its own sub-buffer information independently.

[0196] Method two, known as per_ctx_shared_fifo, involves the general-purpose processor allocating a shared memory buffer of a preset size for the multiple cores of the AI ​​processor. All cores share this memory buffer to record their output and jointly maintain information about it.

[0197] For detailed information, please refer to the relevant descriptions in the foregoing embodiments.

[0198] Step 1102: Send the data output task to the artificial intelligence processor.

[0199] Step 1103: Retrieve the content to be output from the memory buffer. The content to be output is written to the memory buffer by the AI ​​processor after parsing the output parameters of the data output task. Optionally, during the parsing process of the AI ​​processor, the host side can periodically read the information in the memory buffer, and retrieve the content to be output from the memory buffer when it is determined that there is unprocessed content to be output in the memory buffer.

[0200] In this step, while the AI ​​processor is writing the content to be output into the memory buffer, the general-purpose processor can read the content to be output from the memory buffer.

[0201] Taking a general-purpose processor as the host side and an AI processor as the device side as an example, during the process of writing output content to the memory buffer on the device side, the driver running on the host side can read the information in the memory buffer. If it is determined that there is newly written content in the memory buffer, the driver on the host side will copy the newly written content in the memory buffer back to the host side and perform a second parsing. For example, when the host side performs synchronization operations such as queue sync, the driver will read the memory buffer information and determine whether there is any unprocessed content in the memory buffer. If so, it will copy the unprocessed content in the memory buffer back to the host side for parsing and output, and then update the memory buffer information.

[0202] In the foregoing Figure 2 In the illustrated embodiment, the output of `Printf` is only processed by the host side after the kernel finishes. If the kernel does not finish, no output can be generated, resulting in output lag. In this embodiment, the general-purpose processor can read the memory buffer information at fixed intervals to check if there is any new content to be written. If so, it copies it back to the general-purpose processor for parsing and output. The periodic reading process can be performed simultaneously with the process of the AI ​​processor writing the content to be output to the memory buffer. That is, during the execution of the AI ​​processor's kernel function, the general-purpose processor can periodically read the content of the memory buffer for processing without waiting for the kernel function to finish, thus avoiding output lag and improving data processing efficiency. Step 1104: Parse the content to be output and output it to the specified device according to the parsing result.

[0203] In this embodiment, the host-side driver can read the memory buffer information. If it is determined that there is newly written content in the memory buffer, the host-side driver will copy the newly written content in the memory buffer back to the host side and perform a second parsing. Taking the printf function in the previous embodiment as an example, the second parsing will also scan each character in fmt sequentially. After determining the parameter corresponding to each type descriptor (%type) in fmt, the snprintf function (which formats the variable number of parameters into a string according to format) can be used to finally output the content to be output to the screen. During the second parsing process, the content corresponding to the type tag %s that has already been parsed by the device side will be directly output, without the need for the host side to perform a second parsing of %s. Therefore, the problem of only being able to output string constants due to the host's inability to parse %s is avoided, improving the diversity and flexibility of data output and improving data processing efficiency.

[0204] As mentioned above, the memory buffer is jointly maintained by the host side and the device side. Based on the two different memory buffer partitioning methods described above (Method 1 and Method 2), the host side of the embodiments of this disclosure may have different behaviors.

[0205] When the memory buffer uses Method 1, the host side can read the sub-buffer information corresponding to each core of the AI ​​processor. If it determines that there is unprocessed content to be output in a sub-buffer, it retrieves the content to be output from the sub-buffer and outputs it to the designated device. Furthermore, after outputting the content to be output to the designated device, the host side can also update the data processing index in the memory buffer.

[0206] In this embodiment, if the general-purpose processor allocates a sub-buffer of a preset size for each core of the AI ​​processor, the general-purpose processor can read the sub-buffer information corresponding to each core when reading the buffer content, and then obtain the content to be output from the sub-buffer. That is, if the memory buffer allocation method is method one per_ctx_per_core, the host-side driver will read the tail and head of each core at regular intervals during kernel operation. When the head is less than the tail, it can be determined that there is unprocessed content in the memory buffer. At this time, the host-side driver will copy the unprocessed content in the memory buffer back to the host side for processing. After processing, the head will be updated and synchronized to the device side. For details, please refer to the relevant descriptions in the foregoing embodiments, which will not be repeated here.

[0207] In Method 2, the host can read the shared memory buffer information. When it determines that there is unprocessed content to be output in the shared memory buffer, it retrieves the content to be output from the shared memory buffer and outputs it to the designated device. Furthermore, after outputting the content to be output to the designated device, the host can also update the data processing index in the memory buffer.

[0208] In this embodiment, if the general-purpose processor requests a shared memory buffer of a preset size for multiple cores of the AI ​​processor, the general-purpose processor can directly read the shared memory buffer information when reading the buffer content, and then obtain the content to be output from the shared memory buffer. That is, if the memory buffer is requested using method two, per_ctx_shared_fifo, for the case of multiple cores sharing the buffer, the host-side driver will read tail and consumer at regular intervals during kernel operation. When consumer is less than tail, it can be determined that there is unprocessed content in the memory buffer. At this time, the host-side driver will copy the unprocessed content in the memory buffer back to the host side for processing. After processing, it will update consumer and synchronize with the device side. For details, please refer to the relevant descriptions in the foregoing embodiments, which will not be repeated here.

[0209] In the foregoing Figure 2 In the illustrated embodiment, when many kernel functions containing the `printf` function have not finished executing, the memory occupied by these kernel functions on the device side will not be released, resulting in a large peak in memory overhead and a situation where device-side storage space cannot be allocated. In this embodiment, after parsing is completed, the host-side driver updates the memory buffer information, resetting the corresponding part of the memory buffer to free, so that the device side can continue to write new content into the free memory buffer, improving memory resource utilization.

[0210] In one embodiment, the method further includes: releasing the memory buffer when the host destroys the thread to which the data output task belongs.

[0211] In the foregoing Figure 2 In the illustrated embodiment, when a user calls the Kernel function containing the printf function multiple times, each call to the Kernel function involves allocating device-side memory, copying device-side memory, and releasing device-side memory. These multiple memory operations result in significant time overhead. In this embodiment, when the host destroys the context, it releases the device-side memory buffer. The printf function only performs the memory buffer release operation at the end of the context's execution, reducing the number of memory operations and lowering the time overhead.

[0212] In the foregoing Figure 2In the illustrated embodiment, the printf function is implemented internally by the compiler, and its logic is difficult to expose, making debugging and optimization challenging. The printf function is implemented using macro syntax sugar, not the standard variable argument implementation.

[0213] In an optional embodiment of this application, the Printf function can be implemented using the standard variable parameter method, and the basic code for the implementation is shown below:

[0214] printf(const char*fmt,...){

[0215] va_list args; / / Create a variable of type va_list

[0216] va_start(args,fmt); / / Initialize the va_list variable to a list of arguments.

[0217] va_arg(args, type); / / Access each item in the parameter list

[0218] The rest of the code

[0219] va_end(args); / / Clean up the memory assigned to the va_list variable

[0220] }

[0221] The "..." in the function parameters indicates a list of variable arguments, which are read sequentially using the relevant interfaces for variable arguments within the function.

[0222] The printf function is implemented using a standard variable argument method, which makes it highly readable and allows for source-level debugging using a debugger, facilitating debugging and optimization.

[0223] The data output method in this application implements an improved printf function that can print device-side variables to the screen. The essence of this improved printf function implementation is the implementation of fprintf (a common function in many programming languages, whose main function is to write formatted output to a specified file or output stream). The improved printf function calls fprintf and specifies the output stream as stdout (standard output).

[0224] The foregoing Figure 2 In the embodiment shown, the printf function cannot obtain detailed information about the output during execution, so the printf function has no return value and cannot provide the user with specific execution details of the function.

[0225] Compared to Figure 2The `printf` function shown in this embodiment of the application parses `fmt` on the device side, so it can return a meaningful integer value to help the user analyze the data. Normally, the function returns the number of parsed parameters other than `fmt_ptr`. When `fmt_ptr` is NULL, the function returns -1. The following two tables explain the function's return value under different conditions:

[0226] Table 1: Handling of different cores when fmt_ptr points to different RAMs

[0227]

[0228] Table 2: Processing of string parameters in different cores when they are located in different RAM locations

[0229]

[0230] The data output method of this application parses `fmt` and `%s` on the device side. The string parameters pointed to by the character pointers corresponding to `fmt` and `%s` can be located in any legal space. It supports specifying whether the string parameter format is left-aligned or right-aligned in `%s`, and supports setting the width and precision of the string parameter. The width represents the minimum output length of the string, and spaces are used to pad if it is insufficient. Therefore, the string parameter can include both spaces and the string itself in the final output. The improved `printf` function does not have the problems of data loss and output lag, and it can specify the output stream. At the same time, there are also certain optimizations in terms of time and space overhead, which can help users better complete the function of printing device-side variables.

[0231] For details of each step of the above method, please refer to the relevant descriptions in the above embodiments, which will not be repeated here.

[0232] Please refer to Figure 12 This is an interactive schematic diagram of a data output method according to an embodiment of this application. It aims to improve the diversity and flexibility of data output and increase data processing efficiency by parsing the output parameters through an artificial intelligence processor, thus avoiding the problem of only being able to output string constants due to the inability of a general-purpose processor to parse relevant parameters. This embodiment takes the interaction between a general-purpose processor 21 and an artificial intelligence processor 22 as an example. The method includes the following steps:

[0233] Step 1201: In response to the data output instruction, the general-purpose processor requests a memory buffer on the AI ​​processor side for the data output task to be processed.

[0234] Step 1202: The general-purpose processor sends the data output task to the artificial intelligence processor.

[0235] Step 1203: The artificial intelligence processor receives the data output task sent by the general-purpose processor.

[0236] Step 1204: The artificial intelligence processor parses the output parameters of the data output task to obtain the basic information of the content to be output.

[0237] Step 1205: The AI ​​processor writes the content to be output into the memory buffer based on the basic information of the content to be output.

[0238] Step 1206: The general-purpose processor retrieves the content to be output from the memory buffer. The content to be output is written into the memory buffer after the artificial intelligence processor parses the output parameters of the data output task.

[0239] Step 1207: The general-purpose processor parses the content to be output and outputs the content to the specified device according to the parsing result.

[0240] It should be noted that the execution order between step 1206 and steps 1204-1205 is only an example. Step 1206 can be executed before steps 1204-1205, or after steps 1204-1205. Step 1206 can also be executed simultaneously with steps 1204-1205. The execution order is not limited in this embodiment.

[0241] For details of each step of the above method, please refer to the relevant descriptions in the above embodiments, which will not be repeated here.

[0242] Please refer to Figure 13 This is an interactive schematic diagram of a data output method according to an embodiment of this application. Compared with the previous embodiment, this embodiment uses a general-purpose processor 21 as the host and an artificial intelligence processor 22 as the device. Taking the interaction execution process between the host and the device as an example, the method includes the following steps:

[0243] Step 1301: The host side checks whether the kernel function contains the printf function. If so, the driver allocates a memory buffer on the device side for printf at the beginning of the context.

[0244] Step 1302: The host sends the Kernel function to the device.

[0245] Step 1303: During the execution of the Kernel function on the device side, fmt and %s are parsed, necessary information is stored, the write size required for the output content is calculated, and the information of the memory buffer is read. When the device side determines that the free size of the memory buffer is greater than or equal to the write size of the output content, the output content is written to the memory buffer, and the information of the memory buffer is updated after writing.

[0246] Step 1304: The host side and the device side jointly maintain a memory buffer information. For details, please refer to the relevant description in the foregoing embodiments, which will not be repeated here.

[0247] Step 1305: At fixed intervals, the host-side driver actively reads the information in the memory buffer to check if there is any new content written to it. If so, the driver copies the contents of the memory buffer back to the host side for parsing and output, and then updates the information in the memory buffer.

[0248] Step 1306: The host side confirms that the kernel function execution has ended.

[0249] Step 1307: When the host performs synchronization operations such as queue sync, the driver reads the information in the memory buffer and checks if there is any new content written to the memory buffer. If so, the driver copies the contents of the memory buffer back to the host for parsing and output, and then updates the memory buffer information.

[0250] Step 1308: The host destroys the context, and at the same time, the driver releases the memory buffer.

[0251] For details of each step of the above method, please refer to the relevant descriptions in the above embodiments, which will not be repeated here.

[0252] Please refer to Figure 14A This is a data output device 1400 according to an embodiment of this application. The device 1400 can be applied to... Figure 1 The artificial intelligence processor in the illustrated electronic device 1 is designed to parse output parameters, avoiding the problem of only being able to output string constants due to the inability of general-purpose processors to parse relevant parameters. This improves the diversity and flexibility of data output and enhances data processing efficiency. The device includes: a receiving module 1401, a first parsing module 1402, a writing module 1403, and a transmission module 1404. The functional principles of each module are as follows:

[0253] The receiving module 1401 is used to receive data output tasks sent by the general-purpose processor.

[0254] The first parsing module 1402 is used to parse the output parameters of the data output task to obtain the basic information of the content to be output.

[0255] The writing module 1403 is used to write the content to be output into the memory buffer based on the basic information of the content to be output.

[0256] The transmission module 1404 is used to transfer the content to be output from the memory buffer to the general-purpose processor.

[0257] In one embodiment, the output parameters include one or more type tags. The first parsing module 1402 is used to parse the target type tags in the output parameters, determine the attribute information of the string parameters corresponding to the target type tags, and the basic information of the content to be output includes the attribute information of the string parameters.

[0258] In one embodiment, the first parsing module 1402 is used to sequentially scan each character in the format string of the data output task. If a target type label is scanned, each character indicated by the target type label is parsed sequentially to obtain the attribute information of the string parameter, which includes a pointer to the string parameter.

[0259] In one embodiment, the writing module 1403 is used to determine the memory requirement of the content to be output based on its basic information. It determines whether the free memory remaining in the memory buffer is greater than or equal to the memory requirement. If the free memory remaining in the memory buffer is greater than or equal to the memory requirement, the content to be output is written to the memory buffer.

[0260] In one embodiment, the write module 1403 is used to determine the remaining amount of a first memory region between the current write address and the end address of the memory buffer. It determines whether the remaining amount of the first memory region is less than the memory requirement of the content to be output. If the remaining amount of the first memory region is greater than or equal to the memory requirement of the content to be output, the content to be output is written to the first memory region. If the remaining amount of the first memory region is less than the memory requirement of the content to be output, the remaining amount of a second memory region starting from the start address of the memory buffer is determined. If the sum of the remaining amounts of the first and second memory regions is greater than or equal to the memory requirement of the content to be output, the content to be output is written to the first and second memory regions sequentially in stages.

[0261] In one embodiment, the basic information includes one or more of the following: the number of string parameters corresponding to the target type label, the output flag, and the output width. The writing module 1403 is used to determine the type of the starting character to be written in the second memory area if the sum of the remaining space in the first memory area and the remaining space in the second memory area is greater than or equal to the memory requirement of the content to be output. If the type of the starting character to be written is a string constant, the string constants appearing consecutively before and after the starting character are written to the first and second memory areas in two steps. If the type of the starting character to be written is a string parameter, the content pointed to by the target string parameter corresponding to the starting character is written to the first and second memory areas in steps according to the number of string parameters corresponding to the target type label, the output flag, and the output width.

[0262] In one embodiment, the writing module 1403 is configured to, if the type of the initial write character is a string parameter, determine the output alignment of the target string parameter based on the output flag of the target string parameter corresponding to the initial write character. If the output alignment of the target string parameter is right alignment, determine the current remaining space in the first memory area of ​​the memory buffer before the target string parameter is written, and determine the number of output spaces required by the target string parameter based on the number of target string parameters and the output width. If the current remaining space is less than the number of output spaces and the current remaining space is not zero, write the output spaces of the target string parameter into the first memory area and the second memory area in two steps, and write the target string parameter itself into the second memory area. If the current remaining space is equal to the number of output spaces, write the output spaces of the target string parameter into the first memory area, and write the target string parameter itself into the second memory area in two steps. If the current remaining space is greater than the number of output spaces, write the output spaces of the target string parameter into the first memory area, and write the target string parameter itself into the first memory area and the second memory area in two steps.

[0263] In one embodiment, the writing module 1403 is further configured to determine the write size of the target string parameter itself if the output alignment of the target string parameter is left-aligned. If the current margin is zero, the target string parameter itself and the output space are written to the second memory area sequentially. If the current margin is not zero and the current margin is less than the write size of the target string parameter itself, the target string parameter itself is written to the first memory area and the second memory area in steps, and the output space of the target string parameter is written to the second memory area. If the current margin is equal to the write size of the target string parameter itself, the target string parameter itself is written to the first memory area, and the output space of the target string parameter is written to the second memory area. If the current margin is greater than the write size of the target string parameter itself, the target string parameter itself is written to the first memory area, and the output space of the target string parameter is written to the first memory area and the output space of the target string parameter is written to the second memory area in steps.

[0264] In one embodiment, the memory buffer includes at least one sub-buffer, each sub-buffer corresponding to at least one core of the artificial intelligence processor. The apparatus further includes a determining module for acquiring a data write index and a data processing index in the sub-buffer corresponding to the data output task. Based on the data write index and the data processing index, the free memory balance of the sub-buffer is determined.

[0265] In one embodiment, the memory buffer includes a shared memory buffer corresponding to at least one core of the artificial intelligence processor. The apparatus further includes a determining module for obtaining a current data write index and a data processing index in the shared memory buffer. The current data write index is updated based on the number of memory entries required for the currently written content. The remaining free memory in the shared memory buffer is determined based on the updated data write index and data processing index.

[0266] In one embodiment, it further includes: a first update module, configured to update the data write index in the memory buffer after writing the content to be output into the memory buffer.

[0267] For a detailed description of the data output device 1400, please refer to the description of the relevant method steps in the above embodiments. The implementation principle and technical effect are similar, and will not be repeated here.

[0268] Please refer to Figure 14B This is a data output device 1500 according to an embodiment of this application. The device 1500 can be applied to... Figure 1 The general-purpose processor in the electronic device 1 shown is used to parse the output parameters by an artificial intelligence processor, avoiding the problem of only being able to output string constants due to the general-purpose processor's inability to parse relevant parameters. This improves the diversity and flexibility of data output and enhances data processing efficiency. The device includes: an application module 1501, a sending module 1502, an acquisition module 1503, and a second parsing module 1504. The functional principles of each module are as follows:

[0269] Application module 1501 is used to request a memory buffer on the AI ​​processor side for the data output task to be processed in response to a data output instruction.

[0270] The sending module 1502 is used to send data output tasks to the artificial intelligence processor.

[0271] The acquisition module 1503 is used to acquire the content to be output from the memory buffer. The content to be output is written into the memory buffer after the artificial intelligence processor parses the output parameters of the data output task.

[0272] The second parsing module 1504 is used to parse the content to be output and output the content to be output to the specified device according to the parsing result.

[0273] In one embodiment, the artificial intelligence processor includes multiple cores. An application module 1501 is used to apply for a sub-buffer of a preset size for each core of the artificial intelligence processor. An acquisition module 1503 is used to read the sub-buffer information corresponding to each core of the artificial intelligence processor, and when it is determined that there is unprocessed content to be output in the sub-buffer, to acquire the content to be output from the sub-buffer.

[0274] In one embodiment, the artificial intelligence processor includes multiple cores. The application module 1501 is used to apply for a shared memory buffer of a preset size for the multiple cores of the artificial intelligence processor. The acquisition module 1503 is used to read the shared memory buffer information, and when it is determined that there is unprocessed content to be output in the shared memory buffer, it acquires the content to be output from the shared memory buffer.

[0275] In one embodiment, the acquisition module 1503 is configured to periodically read information from the memory buffer during the parsing process performed by the artificial intelligence processor, and acquire the unprocessed content to be output from the memory buffer when it is determined that there is unprocessed content to be output in the memory buffer. And / or, the device further includes: a second update module, configured to update the data processing index in the memory buffer after parsing the content to be output.

[0276] In one embodiment, the application module 1501 is used to apply for a memory buffer on the AI ​​processor side for the thread to which the data output task belongs. And / or, the device further includes a release module, used to release the memory buffer when the thread to which the data output task belongs is destroyed.

[0277] For a detailed description of the data output device 1500, please refer to the description of the relevant method steps in the above embodiments. The implementation principle and technical effect are similar, and will not be repeated here.

[0278] In one possible implementation, a chip is also disclosed that includes the data output device provided in any of the above embodiments.

[0279] In one possible implementation, a board is also disclosed that can serve as a device side in a heterogeneous computing system. Figure 15 This is a structural diagram of a board according to an embodiment of this application, such as... Figure 15As shown, board 500 includes chip 501, which is a system-on-chip (SoC) for artificial intelligence. It integrates one or more combined processing units, which are artificial intelligence computing units used to support various deep learning and machine learning algorithms, meeting the intelligent processing needs of complex scenarios in fields such as computer vision, speech, natural language processing, and data mining. In particular, deep learning technology is widely used in cloud intelligence. A significant characteristic of cloud intelligence applications is the large volume of input data, placing high demands on the platform's storage and computing capabilities. Board 500 in this embodiment is suitable for cloud intelligence applications, possessing massive off-chip storage, on-chip storage, and substantial computing power.

[0280] Chip 501 is connected to external device 503 via external interface device 502. External device 503 may be, for example, a server, computer, camera, monitor, mouse, keyboard, network card, or Wi-Fi interface. Optionally, in this embodiment, it may be various vision sensors. Data to be processed can be transmitted from external device 503 to chip 501 via external interface device 502. The calculation results of chip 501 can be transmitted back to external device 503 via external interface device 502. Depending on the application scenario, external interface device 502 may have different interface forms, such as a PCIe interface.

[0281] The board 500 also includes a storage device 504 for storing data, which includes one or more memory cells 505. The storage device 504 is connected to and transmits data with the controller 506 and the artificial intelligence chip 501 via a bus. The controller 506 in the board 500 is configured to regulate the state of the artificial intelligence chip 501. Therefore, in one application scenario, the controller 506 may include a microcontroller (MCU).

[0282] In one possible implementation, a combined processing device is also provided. Figure 16 A structural diagram of the combined processing device in chip 501 of this embodiment is shown. Figure 16 As shown, the combined processing device 60 includes a computing device 601, an interface device 602, a processing device 603, and a storage device 604.

[0283] The computing device 601 is configured to perform user-specified operations. It is mainly implemented as a single-core intelligent processor or a multi-core intelligent processor to perform deep learning or machine learning calculations. It can interact with the processing device 603 through the interface device 602 to jointly complete the user-specified operations.

[0284] Interface device 602 is used to transmit data and control commands between computing device 601 and processing device 603. For example, computing device 601 can obtain input data from processing device 603 via interface device 602 and write it to on-chip storage device of computing device 601. Further, computing device 601 can obtain control commands from processing device 603 via interface device 602 and write them to on-chip control cache of computing device 601. Alternatively or optionally, interface device 602 can also read data from storage device of computing device 601 and transmit it to processing device 603.

[0285] The processing device 603, as a general-purpose processing device, performs basic controls including but not limited to data transfer and starting / stopping the computing device 601. Depending on the implementation, the processing device 603 may be one or more types of processors, such as a central processing unit (CPU), a graphics processing unit (GPU), or other general-purpose and / or special-purpose processors. These processors include, but are not limited to, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., and their number can be determined according to actual needs. As mentioned above, the computing device 601 of this application can be considered as having a single-core structure or a homogeneous multi-core structure. However, when the computing device 601 and the processing device 603 are considered together, they are considered to form a heterogeneous multi-core structure.

[0286] Storage device 604 is used to store data to be processed. It may be DRAM 604, which is DDR memory, typically 16G or larger in size, and is used to store data of computing device 601 and / or processing device 603.

[0287] Figure 17 The diagram below shows the internal structure of a single-core computing device. The single-core computing device 700 is used to process input data for computer vision, speech, natural language processing, data mining, etc. The single-core computing device 700 includes three main modules: a control module 701, a processing module 702, and a storage module 703.

[0288] The control module 701 coordinates and controls the operation of the computation module 702 and the storage module 703 to complete the deep learning task. It includes an instruction fetch unit (IFU) 7011 and an instruction decode unit (IDU) 7012. The instruction fetch unit 7011 fetches instructions from the processing device, while the instruction decode unit 7012 decodes the fetched instructions and sends the decoding result as control information to the computation module 702 and the storage module 703.

[0289] The computation module 702 includes a vector operation unit 7021 and a matrix operation unit 7022. The vector operation unit 7021 is used to perform vector operations and can support complex operations such as vector multiplication, addition, and nonlinear transformations. The matrix operation unit 7022 is responsible for the core computations of deep learning algorithms, namely matrix multiplication and convolution.

[0290] Storage module 703 is used to store or move relevant data, including neuron RAM (NRAM) 7031, weight RAM (WRAM) 7032, and direct memory access (DMA) module 7033. NRAM 7031 stores input neurons, output neurons, and intermediate results. WRAM 7032 stores the convolution kernels, i.e., weights, of the deep learning network. DMA 7033 is connected to DRAM 604 via bus 704 and is responsible for data transfer between the single-core computing device 700 and DRAM 604.

[0291] Figure 18 The diagram shows the internal structure of a multi-core computing device. The multi-core computing device 800 adopts a hierarchical design. As a system-on-a-chip, the multi-core computing device 800 includes at least one cluster, and each cluster includes multiple processor cores. In other words, the multi-core computing device 800 is constructed in a hierarchical structure of system-on-a-chip, cluster, and processor cores.

[0292] From the perspective of system-on-a-chip hierarchy, such as Figure 18 As shown, the multi-core computing device 800 includes an external storage controller 801, a peripheral communication module 802, an on-chip interconnect module 803, a synchronization module 804, and multiple clusters 805.

[0293] There can be multiple external storage controllers 801; two are shown as an example in the figure. These controllers are used to respond to access requests from the processor core to access external storage devices, such as… Figure 18The DRAM 604 in the chip allows data to be read from or written to external sources. The peripheral communication module 802 receives control signals from the processing device via the interface device 602 to initiate tasks. The on-chip interconnect module 803 connects the external storage controller 801, the peripheral communication module 802, and multiple clusters 805 to transmit data and control signals between modules. The synchronization module 804 is a global barrier controller (GBC) used to coordinate the working progress of each cluster and ensure information synchronization. The multiple clusters 805 are the computing cores of the multi-core computing device 800. Four are shown exemplary in the figure; however, with hardware development, the multi-core computing device 800 of this application can also include 8, 16, 64, or even more clusters 805. The clusters 805 are used to efficiently execute deep learning algorithms.

[0294] From the perspective of cluster hierarchy, such as Figure 18 As shown, each cluster 805 includes multiple processor cores (IPU cores) 806 and one memory core (MEM core) 807. For example, each cluster 805 includes four processor cores and one memory, which can be DRAM 604.

[0295] Four processor cores 806 are shown in the figure as an example, but this application does not limit the number of processor cores 806. Each processor core 806 is similar to Figure 19 The single-core computing device also includes three main modules: a control module 901, an arithmetic module 902, and a storage module 903. The functions and structures of the control module 901, arithmetic module 902, and storage module 903 are largely the same as those of the control module 701, arithmetic module 702, and storage module 703. The control module 901 includes an instruction fetch unit 9011 and an instruction decode unit 9012. The arithmetic module 902 includes a vector operation unit 9021 and a matrix operation unit 9022. Further details are omitted. It should be noted that the storage module 903 includes an input / output direct memory access (IODMA) module 9033 and a move direct memory access (MVDMA) module 9034. The IODMA 9033 controls memory access between the NRAM 9031 / WRAM 9032 and the DRAM 604 via the broadcast bus 809. The MVDMA 9034 is used to control memory access to the NRAM 9031 / WRAM 9032 and the SRAM 808.

[0296] Back Figure 18The storage core 807 is primarily used for storage and communication, namely storing shared data or intermediate results among processor cores 806, and performing communication between cluster 805 and DRAM 604, communication between clusters 805, and communication between processor cores 806. In other embodiments, the storage core 807 has scalar operation capabilities and is used to perform scalar operations.

[0297] The storage core 807 includes an SRAM 808, a broadcast bus 809, a cluster direct memory access (CDMA) module 810, and a global direct memory access (GDMA) module 811. The SRAM 808 acts as a high-performance data relay station. Data multiplexed between different processor cores 806 within the same cluster 805 does not need to be obtained from the DRAM 604 by each processor core 806. Instead, it is relayed between processor cores 806 via the SRAM 808. The storage core 807 only needs to quickly distribute the multiplexed data from the SRAM 808 to multiple processor cores 806 to improve inter-core communication efficiency and greatly reduce on-chip and off-chip input / output access.

[0298] Broadcast bus 809, CDMA 810, and GDMA 811 are used to perform communication between processor cores 806, communication between clusters 805, and data transfer between cluster 805 and DRAM 604, respectively. These will be explained below.

[0299] The broadcast bus 809 is used to complete high-speed communication between the processor cores 806 within the cluster 805. In this embodiment, the broadcast bus 809 supports inter-core communication methods including unicast, multicast, and broadcast. Unicast refers to point-to-point (e.g., single processor core to single processor core) data transmission. Multicast is a communication method that transmits a piece of data from SRAM 808 to several specific processor cores 806. Broadcast is a communication method that transmits a piece of data from SRAM 808 to all processor cores 806, and is a special case of multicast.

[0300] CDMA 810 is used to control SRAM 808 access between different clusters 805 within the same computing device.

[0301] The GDMA 811 works in conjunction with the external memory controller 801 to control memory access from the SRAM 808 to the DRAM 604 in the cluster 805, or to read data from the DRAM 604 into the SRAM 808. As mentioned above, communication between the DRAM 604 and the NRAM 9031 or WRAM 9032 can be achieved through two channels. The first channel is a direct connection between the DRAM 604 and the NRAM 9031 or WRAM 9032 via the IODAM 9033. The second channel involves first transferring data between the DRAM 604 and SRAM 808 via the GDMA 811, and then transferring data between the SRAM 808 and the NRAM 9031 or WRAM 9032 via the MVDMA 9034. Although the second channel appears to require more components and has a longer data flow, in some embodiments, the bandwidth of the second channel is actually much greater than that of the first channel. Therefore, communication between the DRAM604 and the NRAM 9031 or WRAM 9032 may be more efficient through the second channel. Embodiments of this application may select the data transmission channel based on their hardware capabilities.

[0302] In other embodiments, the functions of GDMA 811 and IODMA 9033 can be integrated into the same component. For ease of description, this application treats GDMA 811 and IODMA 9033 as different components. For those skilled in the art, any component that implements functions and achieves technical effects similar to this application falls within the scope of protection of this application. Furthermore, the functions of GDMA 811, IODMA 9033, CDMA 810, and MVDMA 9034 can also be implemented by the same component.

[0303] This application also provides a computer-readable storage medium storing a computer program, which, when executed by at least one processor, implements the method provided in any of the above embodiments.

[0304] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the method of any of the foregoing embodiments.

[0305] The foregoing may be better understood in view of the following clauses:

[0306] Clause 1. A data output method applied to an artificial intelligence processor, the artificial intelligence processor having a memory buffer, the method comprising:

[0307] Receive data output tasks sent by the general-purpose processor.

[0308] The output parameters of the data output task are parsed to obtain basic information about the content to be output.

[0309] Based on the basic information of the content to be output, write the content to be output into the memory buffer.

[0310] The content to be output is transferred from the memory buffer to the general-purpose processor.

[0311] Clause 2, following the method in Clause 1, the output parameters include one or more types of labels. Parse the output parameters of the data output task to obtain basic information about the content to be output, including:

[0312] Parse the target type label in the output parameters to determine the attribute information of the string parameter corresponding to the target type label. The basic information of the content to be output includes the attribute information of the string parameter.

[0313] Clause 3. Following the method in Clause 2, parse the target type label in the data output task to determine the attribute information of the string parameter corresponding to the target type label, including:

[0314] Scan each character in the format string of the data output task in sequence.

[0315] If a target type label is detected, each character indicated by the target type label is parsed sequentially to obtain the attribute information of the string parameter, which includes a pointer to the string parameter.

[0316] Clause 4. According to the method in Clause 1, based on the basic information of the content to be output, write the content to be output into the memory buffer, including:

[0317] Based on the basic information of the content to be output, determine the memory requirements of the content to be output.

[0318] Determine whether the amount of free memory in the memory buffer is greater than or equal to the memory requirement.

[0319] If the free memory in the memory buffer is greater than or equal to the memory requirement, the content to be output will be written to the memory buffer.

[0320] Clause 5. Write the content to be output to a memory buffer according to any one of Clauses 1-4, including:

[0321] Determine the remaining space in the first memory region between the current write address and the end address of the memory buffer.

[0322] Determine whether the remaining space in the first memory area is less than the memory requirement of the content to be output.

[0323] If the remaining space in the first memory area is greater than or equal to the memory requirement of the content to be output, the content to be output is written into the first memory area.

[0324] If the remaining space in the first memory area is less than the memory requirement of the content to be output, determine the remaining space in the second memory area starting from the beginning address of the memory buffer.

[0325] If the sum of the remaining space in the first memory area and the remaining space in the second memory area is greater than or equal to the memory requirement of the content to be output, the content to be output will be written into the first memory area and the second memory area in stages.

[0326] Clause 6. According to the method in Clause 5, the basic information includes one or more of the following: the number of string parameters corresponding to the target type tag, output flags, and output width. If the sum of the remaining space in the first memory area and the remaining space in the second memory area is greater than or equal to the memory requirement of the content to be output, the content to be output is written to the first memory area and the second memory area in steps, including:

[0327] If the sum of the remaining space in the first memory area and the remaining space in the second memory area is greater than or equal to the memory requirement of the content to be output, determine the type of the starting character to be written in the second memory area.

[0328] If the type of the initial character to be written is a string constant, the string constants that appear consecutively before and after the initial character will be written to the first memory area and the second memory area in two steps.

[0329] If the type of the initial character to be written is a string parameter, the content pointed to by the target string parameter corresponding to the initial character is written to the first memory area and the second memory area in steps, according to the number of string parameters corresponding to the target type label, the output flag, and the output width.

[0330] Clause 7. According to the method in Clause 6, if the type of the initial character to be written is a string parameter, the content pointed to by the target string parameter corresponding to the initial character to be written is written step by step to the first memory area and the second memory area according to the number of string parameters corresponding to the target type label, the output flag, and the output width, including:

[0331] If the type of the initial character to be written is a string parameter, the output alignment of the target string parameter is determined according to the output flag of the target string parameter corresponding to the initial character to be written.

[0332] If the output alignment of the target string parameter is right-aligned, determine the current remaining space in the first memory area of ​​the memory buffer before the target string parameter is written, and determine the number of output spaces required for the target string parameter based on the number of target string parameters and the output width.

[0333] If the current margin is less than the number of output spaces and the current margin is not zero, write the output spaces of the target string parameter into the first memory area and the second memory area in two steps, and write the target string parameter itself into the second memory area.

[0334] If the current margin equals the number of output spaces, write the output spaces of the target string parameter to the first memory area, and write the target string parameter itself to the second memory area.

[0335] If the current margin is greater than the number of output spaces, write the output spaces of the target string parameter to the first memory area, and write the target string parameter itself to the first memory area and the second memory area in stages.

[0336] Clause 8. According to the method in Clause 7, if the type of the initial write character is a string parameter, the content pointed to by the target string parameter corresponding to the initial write character is written step by step into the first memory area and the second memory area according to the number of string parameters corresponding to the target type label, the output flag, and the output width, and further includes:

[0337] If the output alignment of the target string parameter is left-aligned, determine the write size of the target string parameter itself.

[0338] If the current margin is zero, write the target string parameter itself and the output space into the second memory area in sequence.

[0339] If the current margin is not zero and the current margin is less than the write size of the target string parameter itself, write the target string parameter itself into the first memory area and the second memory area in steps, and write the output space of the target string parameter into the second memory area.

[0340] If the current margin is equal to the write size of the target string parameter itself, write the target string parameter itself to the first memory area, and write the output space of the target string parameter to the second memory area.

[0341] If the current margin is greater than the write size of the target string parameter itself, write the target string parameter itself to the first memory area, and write the output spaces of the target string parameter to the first memory area and the second memory area in stages.

[0342] Clause 9. According to the method of Clause 4, the memory buffer includes at least one sub-buffer, each sub-buffer corresponding to at least one core of the artificial intelligence processor. The steps for determining the free memory balance of the memory buffer include:

[0343] Get the data write index and data processing index in the sub-buffer corresponding to the data output task.

[0344] The amount of free memory remaining in the sub-buffer is determined based on the data write index and the data processing index.

[0345] Clause 10. According to the method of Clause 4, the memory buffer includes a shared memory buffer, which corresponds to at least one core of the artificial intelligence processor. The steps for determining the free memory balance of the memory buffer include:

[0346] Get the current data write index and data processing index in the shared memory buffer.

[0347] Update the current data write index based on the number of memory entries required for the current content to be written.

[0348] The amount of free memory remaining in the shared memory buffer is determined based on the updated data write index and the data processing index.

[0349] Clause 11. After writing the content to be output to the memory buffer according to any one of Clauses 1-10, the method further includes:

[0350] Update the data in the memory buffer to the write index.

[0351] Clause 12. A data output method applied to a general-purpose processor, comprising:

[0352] In response to a data output command, request a memory buffer on the AI ​​processor side for the data output task to be processed.

[0353] Send the data output task to the artificial intelligence processor.

[0354] The content to be output is obtained from the memory buffer. The content to be output is written into the memory buffer after the AI ​​processor parses the output parameters of the data output task.

[0355] Parse the content to be output and output it to the specified device based on the parsing result.

[0356] Clause 13. According to the method in Clause 12, the AI ​​processor includes multiple cores. A memory buffer on the AI ​​processor side is requested for the data output task to be processed, including:

[0357] Each core of the AI ​​processor is allocated a separate sub-buffer of a preset size.

[0358] Retrieve the content to be output from the memory buffer, including:

[0359] The sub-buffer information corresponding to each core of the AI ​​processor is read separately. When it is determined that there is unprocessed content to be output in the sub-buffer, the content to be output is obtained from the sub-buffer.

[0360] Clause 14. According to the method in Clause 12, the AI ​​processor includes multiple cores. A memory buffer on the AI ​​processor side is requested for the data output task to be processed, including:

[0361] Allocate a shared memory buffer of a preset size for multiple cores of the AI ​​processor.

[0362] Retrieve the content to be output from the memory buffer, including:

[0363] Read the shared memory buffer information, and if it is determined that there is unprocessed content to be output in the shared memory buffer, retrieve the content to be output from the shared memory buffer.

[0364] Clause 15. Obtain the content to be output from the memory buffer according to any one of Clauses 12-14, including:

[0365] During the parsing process executed by the AI ​​processor, information in the memory buffer is periodically read. When it is determined that there is unprocessed content to be output in the memory buffer, the content to be output is retrieved from the memory buffer. And / or,

[0366] After parsing the content to be output, it also includes:

[0367] Update the data processing index in the memory buffer.

[0368] Clause 16. In accordance with the method of Clause 12, request a memory buffer on the AI ​​processor side for the data output task to be processed, including:

[0369] Allocate a memory buffer on the AI ​​processor side for the thread to which the data output task belongs. And / or,

[0370] Release the memory buffer when the thread to which the data output task belongs is destroyed.

[0371] Clause 17. An electronic device comprising a general-purpose processor and an artificial intelligence processor, wherein:

[0372] A general-purpose processor, in response to data output instructions, requests a memory buffer on the AI ​​processor side for the data output task to be processed, and sends the data output task to the AI ​​processor.

[0373] The artificial intelligence processor receives data output tasks from the general-purpose processor. It parses the output parameters of the data output task to obtain basic information about the content to be output. Based on this basic information, it writes the content to be output into a memory buffer.

[0374] The general-purpose processor is also used to retrieve content to be output from the memory buffer, parse the content to be output, and output the content to the specified device based on the parsing result.

[0375] Clause 18. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the method as described in any of the above embodiments.

[0376] Clause 19. A computer program product comprising a computer program that, when executed by a processor, implements the method as described in any of the above embodiments.

[0377] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0378] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0379] It should be understood that the above-described device embodiments are merely illustrative, and the device of this application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.

[0380] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0381] When integrated units / modules are implemented in hardware, the hardware can be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, an AI processor can be any suitable hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, storage units can be any suitable magnetic or magneto-optical storage medium, such as resistive random access memory (RRAM), dynamic random access memory (DRAM), shared random access memory (SRAM), enhanced dynamic random access memory (EDRAM), high-bandwidth memory (HBM), hybrid memory cube (HMC), etc.

[0382] If the integrated unit / module is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0383] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0384] The embodiments of this disclosure have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this disclosure. The descriptions of the embodiments above are only for the purpose of helping to understand the methods and core ideas of this disclosure. Furthermore, any changes or modifications made by those skilled in the art based on the ideas of this disclosure, and on the specific implementation methods and application scope of this disclosure, are all within the scope of protection of this disclosure. Therefore, the content of this specification should not be construed as a limitation of this disclosure.

Claims

1. A data output method, characterized in that, Applied to an artificial intelligence processor, the artificial intelligence processor having a memory buffer, the method includes: Receive data output tasks sent by a general-purpose processor; The output parameters of the data output task are analyzed to obtain basic information about the content to be output. Based on the basic information of the content to be output, the content to be output is written into the memory buffer; The content to be output is transferred from the memory buffer to the general-purpose processor.

2. The method according to claim 1, characterized in that, The output parameters include one or more types of tags; parsing the output parameters of the data output task to obtain basic information about the content to be output includes: The target type tag in the output parameters is parsed to determine the attribute information of the string parameter corresponding to the target type tag. The basic information of the content to be output includes the attribute information of the string parameter.

3. The method according to claim 2, characterized in that, The step of parsing the target type label in the data output task and determining the attribute information of the string parameter corresponding to the target type label includes: Scan each character in the format string of the data output task sequentially; If the target type tag is detected, each character indicated by the target type tag is parsed sequentially to obtain the attribute information of the string parameter, which includes a pointer to the string parameter.

4. The method according to claim 1, characterized in that, The step of writing the content to be output into the memory buffer based on the basic information of the content to be output includes: Based on the basic information of the content to be output, determine the memory requirement of the content to be output; Determine whether the free memory balance in the memory buffer is greater than or equal to the memory requirement; If the free memory balance in the memory buffer is greater than or equal to the memory requirement, the content to be output is written into the memory buffer.

5. The method according to any one of claims 1 to 4, characterized in that, The step of writing the content to be output into the memory buffer includes: Determine the remaining amount of the first memory region between the current write address and the end address of the memory buffer; Determine whether the remaining space in the first memory area is less than the memory requirement of the content to be output; If the remaining space in the first memory area is greater than or equal to the memory requirement of the content to be output, the content to be output is written into the first memory area; If the remaining space in the first memory area is less than the memory requirement of the content to be output, determine the remaining space in the second memory area starting from the starting address of the memory buffer; If the sum of the remaining space in the first memory area and the remaining space in the second memory area is greater than or equal to the memory requirement of the content to be output, the content to be output is written into the first memory area and the second memory area in steps.

6. The method according to claim 5, characterized in that, The basic information includes one or more of the following: the number of string parameters corresponding to the target type tag, output flags, and output width; if the sum of the remaining space in the first memory area and the remaining space in the second memory area is greater than or equal to the memory requirement of the content to be output, the content to be output is written into the first memory area and the second memory area in steps, including: If the sum of the remaining space in the first memory area and the remaining space in the second memory area is greater than or equal to the memory requirement of the content to be output, determine the type of the starting character to be written in the second memory area; If the type of the initial write character is a string constant, the string constants that appear consecutively before and after the initial write character are written into the first memory area and the second memory area in two steps. If the type of the initial write character is a string parameter, the content pointed to by the target string parameter corresponding to the initial write character is written to the first memory area and the second memory area in steps according to the number of string parameters corresponding to the target type label, the output flag bit and the output width.

7. The method according to claim 6, characterized in that, If the type of the initial write character is a string parameter, the content pointed to by the target string parameter corresponding to the initial write character is written step by step into the first memory area and the second memory area according to the number of string parameters corresponding to the target type label, the output flag, and the output width, including: If the type of the starting character to be written is a string parameter, the output alignment of the target string parameter is determined according to the output flag of the target string parameter corresponding to the starting character; If the output alignment of the target string parameter is right-aligned, determine the current remaining amount of the first memory area in the memory buffer before the target string parameter is written, and determine the number of output spaces required for the target string parameter based on the number of target string parameters and the output width; If the current remaining space is less than the number of output spaces and the current remaining space is not zero, the output spaces of the target string parameter are written to the first memory area and the second memory area in two steps, and the target string parameter itself is written to the second memory area. If the current remaining amount is equal to the number of output spaces, write the output spaces of the target string parameter into the first memory area, and write the target string parameter itself into the second memory area; If the current remaining amount is greater than the number of output spaces, the output spaces of the target string parameter are written to the first memory area, and the target string parameter itself is written to the first memory area and the second memory area in steps.

8. The method according to claim 7, characterized in that, If the type of the initial write character is a string parameter, the content pointed to by the target string parameter corresponding to the initial write character is written step by step into the first memory area and the second memory area according to the number of string parameters corresponding to the target type label, the output flag, and the output width, and the method further includes: If the output alignment of the target string parameter is left-aligned, determine the write size of the target string parameter itself; If the current remaining amount is zero, the target string parameter itself and the output space are written sequentially into the second memory area; If the current margin is not zero and the current margin is less than the write size of the target string parameter itself, the target string parameter itself is written to the first memory area and the second memory area in steps, and the output space of the target string parameter is written to the second memory area. If the current remaining space is equal to the write size of the target string parameter itself, write the target string parameter itself into the first memory area, and write the output space of the target string parameter into the second memory area; If the current margin is greater than the write size of the target string parameter itself, the target string parameter itself is written to the first memory area, and the output spaces of the target string parameter are written to the first memory area and the second memory area in steps.

9. The method according to claim 4, characterized in that, The memory buffer includes at least one sub-buffer, each sub-buffer corresponding to at least one core of the artificial intelligence processor; the step of determining the free memory balance of the memory buffer includes: Obtain the data write index and data processing index in the sub-buffer corresponding to the data output task; The amount of free memory remaining in the sub-buffer is determined based on the data write index and the data processing index.

10. The method according to claim 4, characterized in that, The memory buffer includes a shared memory buffer, which corresponds to at least one core of the artificial intelligence processor; The step of determining the free memory balance of the memory buffer includes: Obtain the current data write index and data processing index in the shared memory buffer; Update the current data write index according to the number of memory entries required for the current content to be written; The amount of free memory remaining in the shared memory buffer is determined based on the updated data write index and the data processing index.

11. The method according to any one of claims 1-10, characterized in that, After writing the content to be output into the memory buffer, the method further includes: Update the data write index in the memory buffer.

12. A data output method, characterized in that, Applied to a general-purpose processor, the method includes: In response to a data output command, request a memory buffer on the AI ​​processor side for the data output task to be processed; Send the data output task to the artificial intelligence processor; The content to be output is obtained from the memory buffer. The content to be output is written into the memory buffer after the artificial intelligence processor parses the output parameters of the data output task. The content to be output is parsed, and the content to be output is output to the specified device according to the parsing result.

13. The method according to claim 12, characterized in that, The artificial intelligence processor includes multiple cores; the step of requesting a memory buffer on the artificial intelligence processor side for the data output task to be processed includes: Each core of the AI ​​processor is allocated a separate sub-buffer of a preset size; The step of obtaining the content to be output from the memory buffer includes: The sub-buffer information corresponding to each core of the artificial intelligence processor is read respectively. When it is determined that there is unprocessed content to be output in the sub-buffer, the content to be output is obtained from the sub-buffer.

14. The method according to claim 12, characterized in that, The artificial intelligence processor includes multiple cores; the step of requesting a memory buffer on the artificial intelligence processor side for the data output task to be processed includes: Allocate a shared memory buffer of a preset size for the multiple cores of the artificial intelligence processor; The step of obtaining the content to be output from the memory buffer includes: Read the shared memory buffer information, and when it is determined that there is unprocessed content to be output in the shared memory buffer, obtain the content to be output from the shared memory buffer.

15. The method according to any one of claims 12-14, characterized in that, The step of obtaining the content to be output from the memory buffer includes: During the parsing process executed by the AI ​​processor, information in the memory buffer is periodically read. When it is determined that there is unprocessed content to be output in the memory buffer, the content to be output is retrieved from the memory buffer; and / or, After parsing the content to be output, the following is also included: Update the data processing index in the memory buffer.

16. The method according to claim 12, characterized in that, The process of requesting a memory buffer on the AI ​​processor side for the data output task includes: Allocate a memory buffer on the AI ​​processor side for the thread to which the data output task belongs; and / or, When the thread to which the data output task belongs is destroyed, the memory buffer is released.

17. An electronic device, characterized in that, Including general-purpose processors and artificial intelligence processors, among which: The general-purpose processor is configured to, in response to a data output instruction, request a memory buffer on the AI ​​processor side for a data output task to be processed; and send the data output task to the AI ​​processor. The artificial intelligence processor is used to receive data output tasks sent by the general-purpose processor; parse the output parameters of the data output task to obtain basic information of the content to be output; and write the content to be output into the memory buffer according to the basic information of the content to be output. The general-purpose processor is also used to obtain the content to be output from the memory buffer; parse the content to be output, and output the content to be output to a designated device according to the parsing result.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the method as described in any one of claims 1-16.

19. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method as described in any one of claims 1-16.

Images