Control method, control device, compute-in-memory system and electronic device

Abstract

The application provides a control method, a control device, a storage-computation system and an electronic device. The storage-computation system comprises: a first storage area configured to store first weight data of a first linear layer of a first model block of a neural network model, the first model block being a last model block of a forward propagation path of the model, and the first linear layer being a last linear layer of the forward propagation path of the first model block; a second storage area configured to store weight data of a second linear layer of the first model block; and at least one processing circuit configured to control the second storage area to perform first computation in an analog domain based on a weight parameter of the second linear layer, and to control reading out of the first weight data from the first storage area and performing second computation in a digital domain based on the read-out first weight data. The above scheme can balance the running efficiency and reliability of the artificial intelligence model.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and more specifically, to control methods, control devices, memory computing systems, and electronic devices. Background Technology

[0002] With the rapid development of artificial intelligence (AI) technology, AI models (such as neural network models) are playing an increasingly important role in various intelligent applications. In these applications, AI models are deployed on hardware devices, utilizing hardware resources for computation. The computational demands of AI models are enormous; therefore, how to improve the operational efficiency and reliability of results while utilizing limited hardware resources has become a pressing issue. Summary of the Invention

[0003] This application provides a control method, control device, in-memory computing system, and electronic device that can balance the operating efficiency and reliability of AI models.

[0004] In a first aspect, a storage computing system is provided, comprising: a first storage area for storing first weight data of a first linear layer of a first model block of an artificial intelligence model, wherein the forward propagation path of the artificial intelligence model includes multiple model blocks, the first model block is the last model block in the forward propagation path of the artificial intelligence model, and the forward propagation path of the first model block includes multiple linear layers, wherein the first linear layer is the last linear layer in the forward propagation path of the first model block; a second storage area for storing weight data of a second linear layer of the first model block; and at least one processing circuit for controlling the second storage area to perform a first calculation in the analog domain based on the weight data of the second linear layer; and for controlling the first weight data to be read from the first storage area, and performing a second calculation in the digital domain based on the read first weight data.

[0005] In conjunction with the first aspect, in some implementations of the first aspect, the result of the first calculation is used as the input for the second calculation.

[0006] In conjunction with the first aspect, in some implementations of the first aspect, the weight data corresponding to the weight parameters of the first linear layer includes first weight data and second weight data; a second storage area is also used to store the second weight data; at least one processing circuit is also used to control the second storage area to perform a third calculation of the analog domain based on the second weight data; wherein the result of the first calculation is used as the input of the third calculation, and the result of the third calculation is used as the input of the second calculation.

[0007] In conjunction with the first aspect, in some implementations of the first aspect, the first weight data includes a first array, the second weight data includes a second array, the weight data corresponding to the weight parameters of the first linear layer includes a third array, and the first array and the second array are determined based on the third array.

[0008] In conjunction with the first aspect, in some implementations of the first aspect, the first array and the second array are obtained by decomposing the third array using singular value decomposition.

[0009] In conjunction with the first aspect, in some implementations of the first aspect, the first and second dimensions of the first array are equal to the second dimension of the third array, the first dimension of the second array is equal to the first dimension of the third array, and the second dimension of the second array is equal to the second dimension of the third array.

[0010] In conjunction with the first aspect, in some implementations of the first aspect, the second array is equal to the product of the first part and the second part, the first dimension and the second dimension of the first part are equal to the first dimension of the third array, the first dimension of the second part is equal to the first dimension of the third array, and the second dimension of the second part is equal to the second dimension of the third array.

[0011] In conjunction with the first aspect, when the first dimension of the weight data corresponding to the weight parameters of the first linear layer is less than the second dimension, the weight data corresponding to the weight parameters of the first linear layer is stored in the first storage area; or, when the first dimension of the weight data corresponding to the weight parameters of the first linear layer is greater than the second dimension, the first weight data and the second weight data corresponding to the weight parameters of the first linear layer are stored in the first storage area and the second storage area respectively; or, when the first dimension of the weight data corresponding to the weight parameters of the first linear layer is equal to the second dimension, the first weight data and the second weight data corresponding to the weight parameters of the first linear layer are stored in the first storage area and the second storage area respectively; or, the weight data corresponding to the weight parameters of the first linear layer is stored in the first storage area.

[0012] In conjunction with the first aspect, in some implementations of the first aspect, the first storage region and the second storage region are located in the same storage circuit, or the first storage region and the second storage region are located in different storage circuits.

[0013] In conjunction with the first aspect, in some implementations of the first aspect, the second storage area is further used to store the weight parameters of the linear layer of the second model block among multiple model blocks; at least one processing circuit is further used to control the second storage area to perform a fourth calculation of the simulation domain based on the weight parameters of the linear layer of the second model block.

[0014] Secondly, an electronic device is provided, including a storage computing system as described in any of the first aspects.

[0015] Thirdly, a control method is provided, comprising: acquiring weight data of an artificial intelligence model; storing the first weight data of the first linear layer of a first model block of the artificial intelligence model in a first storage area of ​​an in-memory computing system, wherein the forward propagation path of the artificial intelligence model includes multiple model blocks, the first model block is the last model block in the forward propagation path of the artificial intelligence model, the forward propagation path of the first model block includes multiple linear layers, and the first linear layer is the last linear layer in the forward propagation path of the first model block; storing the weight data of the second linear layer of the first model block in a second storage area of ​​the in-memory computing system; controlling the second storage area to perform a first calculation in the analog domain based on the weight data of the second linear layer; reading the first weight data from the first storage area, and performing a second calculation in the digital domain based on the read first weight data.

[0016] In conjunction with the third aspect, in some implementations of the third aspect, the control method further includes: reading out the result of the first calculation; performing a second calculation in the digital field based on the read out first weight data includes: performing a second calculation in the digital field based on the read out first calculation result and the read out first weight data.

[0017] In conjunction with the third aspect, in some implementations of the third aspect, the weight data corresponding to the weight parameters of the first linear layer includes first weight data and second weight data. The control method further includes: storing the second weight data in a second storage area; inputting the result of the first calculation into the second storage area; controlling the second storage area to perform a third calculation in the analog domain based on the result of the first calculation and the second weight data; reading out the result of the third calculation; and performing a second calculation in the digital domain based on the read first weight data, which includes performing a second calculation in the digital domain based on the read third calculation result and the read first weight data.

[0018] In conjunction with the third aspect, in some implementations of the third aspect, the first weight data includes a first array, the second weight data includes a second array, the weight data corresponding to the weight parameters of the first linear layer includes a third array, and the first array and the second array are determined based on the third array.

[0019] In conjunction with the third aspect, in some implementations of the third aspect, the first and second arrays are obtained by decomposing the third array using singular value decomposition.

[0020] In conjunction with the third aspect, in some implementations of the third aspect, the first and second dimensions of the first array are equal to the second dimension of the third array, the first dimension of the second array is equal to the first dimension of the third array, and the second dimension of the second array is equal to the second dimension of the third array.

[0021] In conjunction with the third aspect, in some implementations of the third aspect, the second array is equal to the product of the first part and the second part, the first dimension and the second dimension of the first part are equal to the first dimension of the third array, the first dimension of the second part is equal to the first dimension of the third array, and the second dimension of the second part is equal to the second dimension of the third array.

[0022] In conjunction with the third aspect, in some implementations of the third aspect, when the first dimension of the weight data corresponding to the weight parameters of the first linear layer is less than the second dimension, the weight data corresponding to the weight parameters of the first linear layer is stored in the first storage area; or, when the first dimension of the weight data corresponding to the weight parameters of the first linear layer is greater than the second dimension, the first weight data and the second weight data corresponding to the weight parameters of the first linear layer are stored in the first storage area and the second storage area respectively; or, when the first dimension of the weight data corresponding to the weight parameters of the first linear layer is equal to the second dimension, the first weight data and the second weight data corresponding to the weight parameters of the first linear layer are stored in the first storage area and the second storage area respectively; or, the weight data corresponding to the weight parameters of the first linear layer is stored in the first storage area.

[0023] In conjunction with the third aspect, in some implementations of the third aspect, the control method further includes: storing the weight data of the linear layer of the second model block among multiple model blocks in a second storage area; and controlling the second storage area to perform a fourth calculation of the simulation domain based on the weight data of the linear layer of the second model block.

[0024] Fourthly, a control device is provided, comprising: a processing circuit for executing a control method as described in any of the third aspects; and an interface circuit for signal connection with at least one storage circuit, the at least one storage circuit including a first storage region and a second storage region.

[0025] All of the above aspects can improve the efficiency of AI model operation while ensuring its reliability. Attached Figure Description

[0026] Figure 1 A schematic diagram of an in-memory computing system according to an exemplary embodiment of this application is shown.

[0027] Figure 2 A schematic diagram of the structure of a model block of a Mamba series model according to an exemplary embodiment of this application is shown.

[0028] Figure 3 A schematic diagram illustrating the change in PPL of different model blocks of a Mamba series model after adding noise, according to an exemplary embodiment of this application, is shown.

[0029] Figure 4A schematic diagram illustrating the change in PPL of a Mamba series model after adding noise to different linear layers according to an exemplary embodiment of this application is shown.

[0030] Figure 5 A schematic diagram showing the change in PPL of a Mamba series model after adding noise according to an exemplary embodiment of this application is shown.

[0031] Figure 6 A schematic diagram showing the change in PPL after adding noise to another Mamba series model according to an exemplary embodiment of this application is shown.

[0032] Figure 7 A schematic flowchart of a control method according to an exemplary embodiment of this application is shown.

[0033] Figure 8 A schematic diagram of a linear layer splitting according to an exemplary embodiment of this application is shown.

[0034] Figure 9 A schematic diagram of another in-memory computing system according to an exemplary embodiment of this application is shown.

[0035] Figure 10 A schematic diagram of yet another storage system according to an exemplary embodiment of this application is shown.

[0036] Figure 11 A schematic diagram of a control device according to an exemplary embodiment of this application is shown.

[0037] Figure 12 A schematic diagram of an electronic device according to an exemplary embodiment of this application is shown. Detailed Implementation

[0038] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0039] To keep the drawings concise, the figures in this application only schematically show the parts related to the corresponding embodiments, and they do not represent the actual structure of the product. In addition, to make the drawings concise and easy to understand, some figures only schematically show some structures or components, and there may actually be more or fewer identical or similar structures or components.

[0040] The business scenarios described in the embodiments of this application are for illustrative purposes only and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0041] In this application, unless otherwise expressly specified and limited, "connection" includes direct or indirect connection between objects: connected objects may be directly connected through a medium (e.g., wires, traces, etc.), or indirectly connected through other components, or may be an internal connection. "Coupling" includes signal connection between objects, which may be achieved directly through a medium (e.g., wires, traces, etc.), or through other components. "Grounding" includes direct grounding or indirect grounding, with indirect grounding including, for example, grounding through other components.

[0042] In this application, unless otherwise expressly specified and limited, ordinal numbers, such as "first," "second," etc., are used only to distinguish the objects being described and should not be construed as indicating or implying the relative importance or order between the objects being described. Furthermore, ordinal numbers do not represent the quantity of the objects being described. "Multiple" includes two or more, and other quantifiers are similar. "Or," "and / or," etc., are used to describe the relationship between objects, indicating a non-exclusive inclusion. For example, "A and / or B," "A or B" can include: "A alone," "B alone," or "A and B." Similarly, "A, B, and / or C," "A, B, or C" can include: "A alone," "B alone," "C alone," "A and B," "A and C," "B and C," or "A, B, and C." Additionally, the " / " in this application is used to indicate an "or" relationship between preceding and following objects. The meaning of "one or more of A and B" or "at least one of A and B" in this application is the same as the meaning of "A and / or B" or "A or B" above. "One or more of A, B and C" or "at least one of A, B and C" has the same meaning as "A, B and / or C" or "A, B or C" above.

[0043] With the rapid development of artificial intelligence (AI) technology, AI models (such as neural network models) are playing an increasingly important role in various intelligent applications. In these applications, AI models are deployed on hardware devices, utilizing hardware resources for computation. The computational demands of AI models are enormous; therefore, how to improve the operational efficiency and reliability of results while utilizing limited hardware resources has become a pressing issue.

[0044] In traditional computing paradigms, such as the von Neumann architecture, storage and computation are physically separated. When processing data using this paradigm, data is frequently transferred between storage devices and computing devices, leading to data transfer latency and energy consumption. In-memory computing architectures can physically integrate storage and computation. This integration can include, for example, integrating storage and computation components close together through packaging processes (known as near-memory computing); integrating processing circuitry within memory to achieve in-memory processing integration; or implementing computation through storage devices (known as in-memory computing), achieving tight integration of storage and computation. In-memory computing architectures can reduce data transfer requirements, lower transmission latency and energy consumption, and significantly improve data processing efficiency.

[0045] This application utilizes an in-memory computing architecture to effectively address the latency and power consumption issues caused by frequent data transfers between storage devices and computing devices. For example, accelerating the execution of AI models through an in-memory computing architecture can improve the efficiency of AI model operation. Furthermore, this application analyzes the impact of the in-memory computing architecture on the operation of AI models, optimizing the deployment of AI models to improve both operational efficiency and reliability.

[0046] For example, Figure 1 A schematic diagram of an in-memory computing system 100 according to an exemplary embodiment of this application is shown. This in-memory computing system 100 can be used to implement one or more computations during the operation of an AI model. For example... Figure 1 As shown, the in-memory computing system 100 may include at least one storage circuit 110 and at least one processing circuit 120.

[0047] The deployment of an AI model may include storing the weight parameters of the AI ​​model in storage circuit 110. The storage of weight parameters may include storing encoded data of the weight parameters, which can be referred to as weight data. That is, storage circuit 110 can be used to store weight data, such as weight data obtained after encoding the weight parameters of the AI ​​model. At least one storage circuit 110 may include storage area 111 and storage area 112. In the AI ​​model deployment of this application embodiment, the weight parameters of the linear layer are stored in storage area 112 in the form of weight data to achieve in-memory computation within storage area 112, thereby accelerating the operation of the model.

[0048] In some embodiments of this application, the in-memory computing system 100 further includes a storage area 111, where linear calculations related to the stored weight data can be performed in the digital domain. For example, the storage area 111 can output the weight data stored therein in response to a read command, and the read weight data can be used to perform digital domain calculations within a processing circuit in at least one processing circuit 120. Optionally, the processing circuit can be integrated closely with the storage area 111 (e.g., through 3D encapsulation) to achieve near-memory computing, thereby further improving the operating efficiency of the AI ​​model.

[0049] According to some embodiments of this application, storage area 111 and storage area 112 may be located in the same storage circuit of the in-memory computing system 100, or they may be located in different storage circuits of the in-memory computing system 100.

[0050] According to some embodiments of this application, the in-memory computing system 100 may include a first storage circuit and a second storage circuit. The first storage circuit may include a storage region 111, and the second storage circuit may include a storage region 112. The storage media of the first storage circuit and the second storage circuit may be of the same or different types. For example, the first storage circuit or the second storage circuit may include, but is not limited to, non-volatile memory (NVM) or volatile memory (VM). Volatile memory may include, but is not limited to, static random access memory (SRAM) or dynamic random access memory (DRAM); non-volatile memory may include, but is not limited to, flash memory, resistive random access memory (RRAM), magnetic random access memory (MRAM), ferroelectric memory (FeRAM), or phase change memory (PCM), etc.

[0051] At least one processing circuit 120 can be used to control the operating state of at least one storage circuit 110, such as controlling one or more operations of the at least one storage circuit 110, such as programming (i.e., writing data), reading, erasing, and calculating. For example, at least one processing circuit 120 can control the reading of weight data stored in storage area 111, and can also perform digital domain calculations based on the weight data read from storage area 111; as another example, at least one processing circuit 120 can convert input data into input signals and provide input signals to storage area 112, and control storage area 112 to convert and output the input signals based on the stored weight data to realize analog domain calculations.

[0052] While analog-domain computation can accelerate the execution of AI models, it is inevitably affected by device noise. For example, semiconductor devices have inherent non-ideals, which can interfere with stored weight data, causing disturbances in the weight parameters of the AI ​​model. These disturbances can affect the inference accuracy of the AI ​​model and even negate the advantages of hardware acceleration. This application analyzes the perturbation sensitivity of weight parameters and proposes the above-mentioned in-memory computing system, which combines the advantages of hardware acceleration with improved reliability of AI model execution results (e.g., inference). Perturbation sensitivity refers to the degree to which weight parameters affect the output reliability of the AI ​​model under device noise interference. Under the same noise interference, the higher the perturbation sensitivity, the greater the impact of weight parameters on the output reliability of the AI ​​model; conversely, the lower the perturbation sensitivity, the lower the impact of weight parameters on the output reliability of the AI ​​model.

[0053] In the in-memory computing system provided in this application embodiment, weight parameters in the AI ​​model that are more sensitive to disturbances (i.e., weight parameters with high disturbance sensitivity) can be stored in storage area 111, and calculations related to these disturbance-sensitive weight parameters can be performed through digital domain computation. Conversely, weight parameters in the AI ​​model that are less sensitive to disturbances can be stored in storage area 112, and calculations for these less sensitive weight parameters (i.e., weight parameters with low disturbance sensitivity) can be performed through analog domain computation. Therefore, hardware acceleration in the analog domain can improve the model's operating efficiency, while reducing the impact of disturbances in weight parameters caused by noise in analog circuits on the AI ​​model's operating results. By combining the disturbance sensitivity of weight parameters with analog and digital domains to deploy the AI ​​model, the advantages of in-memory computing in terms of energy consumption and latency can be balanced, accelerating the AI ​​model's operation. Furthermore, by leveraging the reliability advantages of digital domain computation, the reliability of the AI ​​model's operating results can be improved while simultaneously achieving hardware acceleration.

[0054] According to some embodiments of this application, storage area 111 is used to store the first weight data of the first linear layer of the first model block of the artificial intelligence model. The forward propagation path of the artificial intelligence model includes multiple model blocks, and the first model block is the last model block in the forward propagation path of the artificial intelligence model. The forward propagation path of the first model block includes multiple linear layers, and the first linear layer is the last linear layer in the forward propagation path of the first model block.

[0055] This application embodiment, through research and analysis of AI models, discovers that perturbations in weight parameters accumulate and propagate over the sequence length. In the last model block of the forward propagation path, the perturbation sensitivity of the weight parameters significantly increases, requiring higher robustness. For the last linear layer in the last model block, the perturbation sensitivity of the weight parameters plays a major role. This application embodiment deploys the last linear layer in the forward propagation path of the last model block in storage area 112 and performs calculations related to the perturbation-sensitive weight parameters through digital domain computation, achieving a good balance between the running efficiency and reliability of the AI ​​model. The forward propagation path refers to using the output of the previous layer as the input of the next layer and calculating the output of the next layer until the output layer is reached.

[0056] According to some embodiments of this application, the AI ​​model may include, but is not limited to, a state space model (SSM). SSM models may include, but are not limited to, structured state space models (e.g., S4 model, S4D model), selective state space models (e.g., Mamba series models, S6 model), and visual state space models (e.g., VMamba model, Spatial-Mamba model). SSMs possess recursive characteristics and can be applied to long-sequence processing tasks, such as energy-efficient computing tasks like natural language processing or time series analysis. Therefore, optimizing the deployment of SSM model hardware resources based on the embodiments of this application can balance the operational efficiency and reliability of the SSM model, thereby significantly improving the computing efficiency and reliability of edge devices or data centers. Furthermore, the solutions provided in the embodiments of this application can also be applied to the deployment of other AI models, such as neural network models like transformers; this application does not specifically limit their application to these. According to some embodiments of this application, during the deployment of the AI ​​model, the weight parameters stored in storage area 111 and storage area 112 can be determined based on the sensitivity of the weight parameters of the linear layer of the AI ​​model's blocks to noise.

[0057] In some embodiments of this application, robustness tests on the AI ​​model revealed that the larger the number of parameters, the stronger the robustness of the AI ​​model. Furthermore, the last linear layer in the last model block is most sensitive to noise; for example, the output projection layer of the last model block in the Mamba series models is the most sensitive. This embodiment can extract the last linear layer of the last model block, deploy all or part of it in storage area 111, and perform computation in digital circuits. This improves the noise resistance and accuracy of the AI ​​model while maintaining compatibility with the original hardware architecture.

[0058] This application does not limit the robustness testing method; testing can be based on hardware circuits or software simulation. For example, in one robustness testing process, noise can be injected into the weight parameters to perform in-memory computational noise sensitivity analysis on the AI ​​model, identifying model blocks and / or linear layers that are more sensitive to hardware noise.

[0059] As an example, an AI model can be used to perform multiple computational tasks based on the same input data, adding noise to some of the weight parameters of the AI ​​model in each computational task. The parts with added noise can be different in different computational tasks. For example, noise can be added to different blocks to determine the differences in noise sensitivity between different blocks; or noise can be added to different layers of the same block to determine the differences in noise sensitivity between different layers.

[0060] The simulation results of the AI ​​model are analyzed to identify sensitive parts of the model, such as model blocks and / or linear layers that are sensitive to hardware noise. For example, the reliability of the output of an AI model with added noise can be determined based on the output of the computational task.

[0061] As an example, output accuracy can be used to represent output reliability. The output of a computational task can be compared to the standard label (or expected output) corresponding to the input data to obtain the output accuracy of the computational task relative to the standard label (or expected output). Higher output accuracy indicates higher output reliability; lower output accuracy indicates lower output reliability. For another example, neural network models can include language models, and the perplexity (PPL) of the language model can be used to represent output reliability. The output of a computational task can include the probability distribution of the language model predicting the next token, and the perplexity can be calculated based on the probability distribution. Higher perplexity indicates lower output reliability; lower perplexity indicates higher output reliability. When the injected noise causes a significant decrease in output reliability, it can be determined that the weight parameters of that part of the AI ​​model are highly sensitive to noise.

[0062] As an example, the sensitivity of the weight parameters of each part of an AI model to noise can be negatively correlated with the output reliability of the output obtained when noise is added to the weight parameters of that part during testing. That is, the higher the output reliability, for example, the lower the PPL, the lower the sensitivity of the corresponding weight parameters to noise; the lower the output reliability, for example, the higher the PPL, the higher the sensitivity of the corresponding weight parameters to noise.

[0063] As an example, taking the Mamba series model as the test object, please refer to... Figure 2 This illustrates a schematic diagram of the structure of a model block for a Mamba series model according to an exemplary embodiment of this application. As an example, the structure of a model block for a Mamba series model can be as follows: Figure 2 As shown, following the forward propagation path, the linear layers of a model block can include an input projection layer, a convolution layer, an SSM core layer, and an output projection layer. The output projection layer is the last linear layer in the forward propagation path of the model block.

[0064] As an example, Figure 3 A schematic diagram illustrating the change in PPL of different model blocks of a Mamba series model after adding noise, according to an exemplary embodiment of this application, is shown. Figure 3 The vertical axis represents PPL (Programmable Logic Block), and the horizontal axis represents the model block number. The number corresponds to the position of the model block on the forward propagation path of the AI ​​model; the larger the number, the later the position. For example... Figure 3 As shown, taking the Mamba series model as an example, tests were conducted under different noise standard deviations (0.00, 0.01, 0.02, and 0.03 for added noise, where a noise standard deviation of 0.00 indicates no added noise). The PPL of different model blocks along the forward propagation path of the AI ​​model was obtained. (Refer to...) Figure 3 It can be seen that the PPL of the last model block is significantly improved after noise is added. That is, the decrease in output reliability of the last model block after noise is added is significantly greater than the decrease in output reliability of other model blocks after noise is added. Thus, it can be concluded that the weight parameters of the last model block in the forward propagation path of the AI ​​model are highly sensitive to noise.

[0065] As another example, Figure 4 A schematic diagram illustrating the change in PPL of a Mamba series model after adding noise to different linear layers according to an exemplary embodiment of this application is shown. Figure 4The vertical axis represents PPL, and the horizontal axis represents the noise standard deviation. For example... Figure 4 As shown, tests were conducted under different noise standard deviations (0.00, 0.01, 0.02, 0.03, and 0.04 for added noise, where a noise standard deviation of 0.00 indicates no added noise), to obtain the PPL of different linear layers along the forward propagation path of the model block. (Refer to...) Figure 4 It can be seen that, with the same noise standard deviation, the PPL of the output projection layer is significantly improved after adding noise. That is, with the same noise standard deviation, the decrease in output reliability after adding noise to the output projection layer compared to the state without noise is significantly greater than the decrease in output reliability after adding noise to other layers compared to the state without noise. This indicates that the weight parameters of the last linear layer in the forward propagation path of the AI ​​model block are highly sensitive to noise.

[0066] As an example, Figure 5 A schematic diagram showing the change in PPL of a Mamba series model after adding noise according to an exemplary embodiment of this application is shown. Figure 6 A schematic diagram illustrating the change in PPL after adding noise to another Mamba series model according to an exemplary embodiment of this application is shown. Figure 5 and Figure 6 As shown, the test objects are Mamba1 and Mamba2 models with different parameter amounts in the Mamba series models, respectively. The tests are carried out under test conditions with different noise standard deviations (the noise standard deviations with added noise are 0.00, 0.01, 0.02, 0.03, 0.04 and 0.05, respectively, where a noise standard deviation of 0.00 with added noise indicates no added noise). Figure 5 The test results for the Mamba1 model with parameter values ​​of 130M, 370M, 790M, and 1.4B are shown in the figure. Figure 6 The test results for the Mamba2 model with parameter values ​​of 130M, 370M, 780M, and 1.3B are shown. (Refer to...) Figure 6 It can be seen that, with the same noise standard deviation, for the same model, the larger the number of parameters, the less the PPL of the model is affected by noise, that is, the higher the output reliability and the lower the sensitivity to noise.

[0067] Based on further testing, it can be found that the larger the number of parameters in an AI model, the stronger its robustness; the last model block in the forward propagation path has the highest sensitivity; and the last linear layer (e.g., the output projection layer) in a module block has the highest sensitivity. Further testing revealed that performing noisy inference on the last linear layer in the last model block significantly improves the robustness of the entire AI model. Therefore, the model deployment scheme provided in the above embodiments can significantly improve the operating efficiency and reliability of the AI ​​model.

[0068] According to some embodiments of this application, the weight data stored in storage area 111 may be determined based on the weight parameters of the last linear layer of the last model block in the forward propagation path of the AI ​​model.

[0069] According to some embodiments of this application, the above tests can also be performed on other AI models with added noise to determine the sensitivity of different model blocks or linear layers of other AI models to noise.

[0070] According to some embodiments of this application, the in-memory computing system 100 may include multiple processing circuits, such as a first processing circuit that controls the operating state of the storage circuit, also known as a control circuit 121; or, for example, the in-memory computing system may include a second processing circuit that performs linear calculations in the digital domain on the read weight data, also known as a calculation circuit 122. For example, the control circuit 121 may control one or more operations such as programming, reading, erasing, and calculation in the storage areas 111 and 112. Optionally, the control circuit 121 may also control the calculations of the calculation circuit 122. Optionally, the in-memory computing system 100 may also include a third processing circuit that can be used to control the calculations of the calculation circuit 122. Optionally, the third processing circuit may act as the main controller of the in-memory computing system 100, controlling or coordinating the operation of the control circuit and the calculation circuit.

[0071] According to some embodiments of this application, the in-memory computing system 100 may further include a third storage circuit, such as a cache circuit, for caching data, such as weight data from storage area 111. The computing circuit 122 can read the weight data from the cache circuit and perform calculations using the weight data.

[0072] This application embodiment can also provide a control method that can be used in the above-mentioned in-memory computing system, for example, executed by at least one processing circuit of the above-mentioned in-memory computing system. The at least one processing circuit can be located in a control device, which can be further described in subsequent embodiments.

[0073] For example, Figure 7 A schematic flowchart of a control method according to an exemplary embodiment of this application is shown. This control method 700 can be used for example... Figure 1 The storage system 100 shown is used to control the storage of weight parameters of the AI ​​model. It can also further control the calculations related to the weight parameters. This control method 700 can be executed by a control device.

[0074] like Figure 7 As shown, the control method 700 may include the following steps.

[0075] S710: Obtain weight data for the AI ​​model.

[0076] S720: Store the first weight data of the first linear layer of the first model block of the AI ​​model in the first storage area of ​​the in-memory computing system (e.g., storage area 111).

[0077] S730: The weight data of the second linear layer of the first model block is stored in the second storage area of ​​the in-memory computing system (e.g., storage area 112).

[0078] S740: Controls the second storage area to perform the first calculation of the simulation domain based on the weight data of the second linear layer.

[0079] S750: Read the first weight data from the first storage area, and perform a second calculation in the digital domain based on the read first weight data. The first linear layer of the first model block can be a part of the AI ​​model that is highly sensitive to perturbations. According to some embodiments of this application, the first model block can be the last model block among multiple model blocks in the forward propagation path of the AI ​​model, and the first linear layer can be the last linear layer among multiple linear layers in the forward propagation path of the first model block. Optionally, the first linear layer of the first model block can also include other parts of the AI ​​model that are highly sensitive to perturbations. For AI models with different structures, the first linear layer of the first model block can be located in different positions, and this application does not specifically limit this.

[0080] The first storage area can be used to store first weight data, which is determined based on the weight parameters of the first linear layer of the first model block of the AI ​​model. Similar to the description of the above embodiments, the weight data can be obtained by encoding based on the weight parameters of the AI ​​model. This application does not limit the encoding method. According to some embodiments of this application, the first weight data may include all weight data corresponding to the weight parameters of the first linear layer. According to other embodiments of this application, the first weight data may include at least a portion of the weight data obtained after the weight parameters of the first linear layer are decomposed. For example, the weight data corresponding to the weight parameters of the first linear layer may include first weight data and second weight data, the first storage area can be used to store the first weight data, and the second storage area can be used to store the second weight data. The decomposition of the weight data corresponding to the weight parameters of the first linear layer can be a split based on operational logic, that is, the superposition of the calculations achieved by the weight data obtained after splitting can be equivalent to the calculations achieved by the weight data corresponding to the weight parameters of the first linear layer before splitting.

[0081] The first weight data can be used for a second calculation in the digital domain; that is, the first weight data can be read from the first storage area and then the calculation can be performed in the digital circuit. Taking the in-memory computing system 100 as an example, the storage area 111 can store the first weight data, and at least one processing circuit 120 can read the first weight data from the storage area 111 and perform a second calculation in the digital domain based on the read first weight data. As an example, controlling the reading of the first weight data from the storage area 111 can be performed by the control circuit 121 in at least one processing circuit 120, and performing the second calculation in the digital domain based on the read first weight data can be performed by the calculation circuit 122 in at least one processing circuit 120.

[0082] The second linear layer can be any linear layer in the first model block other than the first linear layer. The second storage area can be used to store the weight data of the second linear layer, that is, the weight data corresponding to the weight parameters of the second linear layer. The weight data of the second linear layer can be used for the first calculation in the simulation domain, that is, in-memory calculations can be performed based on the weight data of the second linear layer within the second storage area. As an example, taking the in-memory computing system 100 as an example, storage area 112 can store the weight data of the second linear layer, and at least one processing circuit 120 can control storage area 112 to perform the first calculation in the simulation domain based on the weight data of the second linear layer.

[0083] According to some embodiments of this application, the first storage area and the second storage area may be located in the same storage circuit, or the first storage area and the second storage area may be located in different storage circuits.

[0084] Optionally, the control method 700 may also include steps S760 and S770.

[0085] S760: Store the weight data of the linear layer of the second model block in the second storage area.

[0086] S770: Controls the second storage area to perform the fourth calculation of the simulation domain based on the weight data of the linear layer of the second model block.

[0087] The second model block can be any model block in the AI ​​model other than the first model block. The second storage area can be used to store the weight data of the linear layers of the second model block, that is, the weight data corresponding to the weight parameters of the linear layers of the second model block. The weight data of the linear layers of the second model block can be used for the fourth calculation in the simulation domain; that is, the weight data of the linear layers of the second model block can also be used for in-memory calculation within the second storage area. As an example, taking the in-memory computing system 100, storage area 112 can store the weight data of the linear layers of the second model block, and at least one processing circuit 120 can control storage area 112 to perform the fourth calculation in the simulation domain based on the weight data of the linear layers of the second model block. Optionally, the result of the fourth calculation of the last linear layer of one of the model blocks can be used as the input of the next model block. In this way, the linear calculations of more model blocks of the AI ​​model are accelerated in the simulation domain, further improving the operating efficiency of the AI ​​model.

[0088] According to some embodiments of this application, in step S720, the first weight data stored in the first storage area may include all weight data corresponding to the weight parameters of the first linear layer. Correspondingly, the first linear layer of the first model block of the AI ​​model can be calculated entirely in the digital domain. In step S750, the result of the aforementioned first calculation can be used for the second calculation. As an example, taking the in-memory computing system 100 as an example, after the storage area 112 performs the first calculation in the analog domain based on the weight data of the second linear layer, at least one processing circuit 120 can read the result of the first calculation from the storage area 112, read the first weight data from the storage area 111, and perform the second calculation in the digital domain based on the read result of the first calculation and the read first weight data.

[0089] According to other embodiments of this application, in step S720, the first weight data stored in the first storage area may include at least a portion of the weight data obtained after the weight parameters of the first linear layer are decomposed. As an example, the weight data of the first linear layer may include first weight data and second weight data. The first weight data can be stored in the first storage area, and the second weight data can be stored in the second storage area. Correspondingly, the first linear layer of the first model block of the AI ​​model can be performed in both the analog and digital domains. As an example, taking the in-memory computing system 100 as an example, after the storage area 112 performs a first calculation in the analog domain based on the weight data of the second linear layer, at least one processing circuit 120 can input the result of the first calculation into the storage area 112 and control the storage area 112 to perform a third calculation in the analog domain based on the result of the first calculation and the second weight data; at least one processing circuit 120 can read the result of the third calculation from the storage area 112, read the first weight data from the storage area 111, and perform a second calculation in the digital domain based on the read result of the third calculation and the read first weight data.

[0090] The superposition of the third and second calculations is equivalent to the calculation performed using the weight data corresponding to the weight parameters of the first linear layer before splitting. For example, under ideal, perturbation-free conditions, after the input data undergoes a third calculation based on the second weight data in the second storage region, the result of the third calculation is used as input, and the output is obtained by performing a second calculation based on the first weight data. This is equivalent to the output obtained by calculating using the input data based on the weight data corresponding to the weight parameters of the first linear layer before splitting.

[0091] This application embodiment can decompose the weight data of the first linear layer into two or more arrays. According to some embodiments of this application, the first weight data includes a first array, the second weight data includes a second array, and the weight data corresponding to the weight parameters of the first linear layer includes a third array. The first array and the second array are determined based on the third array. The array can include one-dimensional arrays and multi-dimensional arrays. Multi-dimensional arrays include, for example, two-dimensional arrays. The arrangement of data in the array can represent the corresponding vector or matrix. For example, a one-dimensional array can represent a vector, and an M-row N-column two-dimensional array can represent a matrix of size M×N, where M and N are positive integers.

[0092] Optionally, the first and second arrays mentioned above can be obtained by decomposing the third array based on singular value decomposition (SVD).

[0093] Optionally, taking a two-dimensional array as an example, the first and second dimensions of the first array can be equal to the second dimension of the third array; the first dimension of the second array can be equal to the first dimension of the third array, and the second dimension of the second array can be equal to the second dimension of the third array. Taking a two-dimensional array as an example, the first dimension can be the number of rows, and the second dimension can be the number of columns. Therefore, the computation of the AI ​​model in the digital domain is based on a square matrix, which is beneficial for caching and can be adapted to square matrix optimization algorithms. Furthermore, the fact that the dimension of the second array is equal to the dimension of the third array ensures that the deployment of the AI ​​model does not require changes to the hardware architecture due to the computation of the last model block in the digital domain. This allows for compatibility with the hardware architecture of other model blocks in the in-memory computing system, providing high accuracy and noise resistance for the AI ​​model while maintaining hardware architecture compatibility. In addition, the in-memory computing system facilitates adjusting the computation of the linear layers of the AI ​​model from both the analog and digital domains to computation only in the analog domain, and also facilitates adjusting the computation of the linear layers of the AI ​​model from computation only in the analog domain to computation in both the analog and digital domains, thereby improving the flexibility of the in-memory computing system.

[0094] Optionally, the second array can be equal to the product of the first part and the second part, wherein the first dimension and the second dimension of the first part are equal to the first dimension of the third array, the first dimension of the second part are equal to the first dimension of the third array, and the second dimension of the second part are equal to the second dimension of the third array. The first part and the second part may include a first subarray and a second subarray. In some embodiments of this application, the first part and the second part can be deployed as a whole within a storage array region.

[0095] Therefore, the second array used by the linear layer of the AI ​​model for computation in the analog domain can be split into the multiplication operation of two matrices or vectors in the analog domain, so as to be compatible with the hardware architecture of other model blocks in the in-memory computing system, thereby improving the flexibility of the in-memory computing system.

[0096] As an example, the first array may include a first matrix, the second array may include a second matrix, and the third array may include a third matrix of size M×N, where M and N are positive integers, and the third matrix satisfies:

[0097] W=UΣV T

[0098] Where W is the third matrix, U is an M-order orthogonal matrix, V is an N-order orthogonal matrix, Σ is an M×N matrix, the diagonal elements of Σ are the singular values ​​of the third matrix and all other elements are 0, and the first and second matrices satisfy the following:

[0099] A=UΣ

[0100] B = V T

[0101] Here, A is the second matrix, and B is the first matrix. Singular value decomposition allows the first matrix B, which is deployed in the digital domain for computation, to retain more information about the original matrix M compared to the second matrix A, which is stored in the analog domain for in-memory computation. This reduces the impact of disturbances in the analog memory circuitry on the reliability of the AI ​​model's computation results.

[0102] Figure 8 A schematic diagram of a linear layer splitting according to an exemplary embodiment of this application is shown. Figure 8 As shown, taking the Mamba series model as an example, Figure 8 The image shows the last model block in the forward propagation path of this AI model, where the output projection layer is the last linear layer in the propagation path of that model block. In other words, Figure 8 The output projection layer shown can be the first linear layer of the first model block in the aforementioned embodiments. For example... Figure 8 As shown, the weight data corresponding to the weight parameters of the output projection layer can include a matrix #1 of size M×N. Optionally, the output projection layer can be decomposed into the product of a matrix #2 of size M×M, a matrix #3 of size M×N, and a matrix #4 of size N×N. For example, the aforementioned singular value decomposition method can be used for decomposition. The calculations of matrices #2 and #3 can be performed in the analog domain. For example, the elements of the equivalent matrix obtained by multiplying matrices #2 and #3, i.e., a matrix of size M×N, can be stored as weight data in a storage circuit for calculation in the analog domain. The elements of matrix #4 can be stored as weight data in the same or another storage circuit for calculation in the digital domain. Thus, matrices #2 and #3 can be used to maintain compatibility with existing hardware architectures, while matrix #4 can be used to provide high accuracy and noise resistance. Matrix #2 and Matrix #3 can also be deployed in other forms in the analog storage circuit, such as as a matrix of size M×M for implementing matrix #2 calculation and another matrix of size M×N for implementing matrix #3 calculation. This application does not make any specific limitation in this regard.

[0103] According to some embodiments of this application, the storage method of the weight data corresponding to the weight parameters of the first linear layer can be determined based on the size of the first dimension and the second dimension of the weight data corresponding to the weight parameters of the first linear layer. For example, the first dimension and the second dimension of the weight data corresponding to the weight parameters of the first linear layer may include the first dimension and the second dimension of the matrix formed by the weight data of the first linear layer. For example, if the weight data of the first linear layer forms a two-dimensional matrix, then the first dimension may be the number of rows of the two-dimensional matrix, and the second dimension may be the number of columns of the two-dimensional matrix; or, the first dimension may be the number of columns of the two-dimensional matrix, and the second dimension may be the number of rows of the two-dimensional matrix.

[0104] Optionally, when the first dimension of the weight data corresponding to the weight parameters of the first linear layer is less than the second dimension, the weight data corresponding to the weight parameters of the first linear layer is stored in the first storage area. In this case, the computational cost of calculation based on the first array in the digital domain is greater than the computational cost of calculation based on the third array in the digital domain. Therefore, deploying the third array entirely in the digital domain can further improve the reliability of the model without increasing the computational cost.

[0105] Optionally, when the first dimension of the weight data corresponding to the weight parameters of the first linear layer is greater than the second dimension, the first weight data and the second weight data corresponding to the weight parameters of the first linear layer are stored in the first storage area and the second storage area, respectively. In this case, the computational cost of calculation based on the first array in the digital domain is less than the computational cost of calculation based on the third array in the digital domain. Therefore, decomposing the third array and deploying it separately can reduce the computational cost while further improving the reliability of the model.

[0106] Optionally, when the first dimension of the weight data corresponding to the weight parameters of the first linear layer is equal to the second dimension, any implementation method can be selected where the first dimension is greater than or less than the second dimension. Further details will not be provided here.

[0107] Based on the above embodiments, the amount of computation in the digital domain can be reduced, thereby reducing computing energy consumption.

[0108] In the above method embodiments, the order of the process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0109] In a memory-computing architecture, storage and computing (or operations) are physically integrated. This physical integration includes, for example, integrating storage and computing components in close proximity through processes such as packaging; integrating processing circuits with processing capabilities into the memory to achieve in-memory processing integration; or implementing computing through storage devices or storing data in computing devices to achieve tight integration of storage and computing.

[0110] For ease of understanding, Figure 9 A schematic diagram of another in-memory computing system according to an exemplary embodiment of this application is shown. For example... Figure 9As shown, the in-memory computing system 900 may include a storage circuit 910 and a control device 920. The storage circuit 910 is used to store data, such as weight data (also known as weights) of a model; the control device 920 is used to control the operation of the storage circuit 910, such as controlling one or more operations like writing, reading (or retrieving) data, calculating, or retrieving calculation results. The storage system can support near-memory computing or in-memory computing. For example, in near-memory computing, the control device 920 can write weight data into the storage circuit 910 and read the weight data stored in the storage circuit 910 during calculation, and perform calculations based on the read weight data. Similarly, in in-memory computing, the control device 920 can write weight data into the storage circuit 910, control the storage circuit 910 to perform in-memory calculations based on the stored weight data, and the control device 920 can also control the reading of calculation results; optionally, the control device 920 can also process the calculation results. The control device 920 may integrate one or more processing circuits. For example, the control device 920 may include a first processing circuit for controlling the operation of the storage circuit (the first processing circuit may be referred to as the control circuit); or the control device 920 may include a second processing circuit for performing calculations based on the read weight data; or the control device 920 may include a third processing circuit for processing the results of the in-memory calculations of the storage circuit 910.

[0111] The operating states of the storage circuit 910 may include a programming state. In the programming state, weight data is written into the storage circuit 910. During in-memory computation, the operating states of the storage circuit 910 also include a computation state, in which the storage circuit 910 receives an input signal Sin and converts the input signal Sin into an output signal Sout based on the weight data. The storage circuit 910 can store multiple weight data, which can be equivalent to at least one vector (or matrix). The storage circuit 910 can store weight data in units of storage cells, which can also be called storage units or storage structures. For example, the storage circuit 910 includes a storage cell array (hereinafter referred to as a storage array), which includes multiple storage cells arranged in an array. Storage cells may include semiconductor devices, and the storage of weight data is achieved by utilizing the conductance capability of the semiconductor devices, such as conductivity or transconductance. For example, storage cells may include resistive storage devices or transistor storage devices. For example, weight data can be stored by controlling the conductivity of the resistive storage device, or by controlling the transconductance of the transistor storage device. Alternatively, the storage unit can use the energy stored in the energy storage element to store the weight data, for example, by using the charge stored in the capacitor to store the weight data; the energy storage element can be connected to a semiconductor device, and the stored energy can act on the semiconductor device to enable the semiconductor device to generate a corresponding conduction capability.

[0112] In the storage circuit 910, the output terminals of multiple storage cells can be connected collinearly, allowing the output signals of the multiple storage cells to be accumulated and output, thus achieving an equivalent multiplication calculation. The multiple storage cells with collinearly connected output terminals can be called a calculation group. Within the calculation group, the multiple weight data stored in the multiple storage cells can be equivalent to a first data vector (or a first data matrix). The input data corresponding to the input signal received by the calculation group can be equivalent to a second data vector (or a second data matrix). The output of the calculation group can be equivalent to the product of the first data vector (or the first data matrix) and the second data vector (or the second data matrix). In programming mode, weight data is written into the storage cells, which is equivalent to writing the first data vector (or the first data matrix) into the storage circuit 910. In calculation mode, the storage circuit 910 receives input signals, and the conduction capability of the storage cells can change the input signal to obtain an output signal. By accumulating and outputting the output signals of the multiple storage cells in the calculation group, an equivalent multiplication calculation can be achieved. The storage array may include a one-dimensional array, a two-dimensional array, or a three-dimensional array, etc. The computing group may include multiple storage units located in the same row or column of the storage array, or multiple storage units located in multiple rows or columns, etc., and the multiple storage units may be output in a collinear manner.

[0113] In some possible embodiments, the storage system 900 may further include a readout circuit 930. The readout circuit 930 can convert and output the output signal Sout. This conversion may include one or more conversions such as signal type conversion, signal magnitude conversion, etc., for example, one or more conversions such as current-to-voltage conversion, analog-to-digital conversion, amplification, etc. The control device 920 can also control the operating timing of the readout circuit 930. In some embodiments, by controlling the operating timing, the readout circuit 930 can be multiplexed from multiple storage arrays or multiple outputs of a storage array, thereby reducing the hardware overhead of peripheral circuitry.

[0114] As an example, Figure 10 A schematic diagram of yet another storage system according to an exemplary embodiment of this application is shown. For example... Figure 10 As shown, the in-memory computing system 1000 includes a storage circuit 1010 and a control circuit 1020. The storage circuit 1010 may include one or more storage arrays, such as storage arrays A1, A2, etc. A storage array may include multiple storage cells. For example, storage array A1 is described as an example; other storage arrays may be included if the storage circuit 1010 includes them, and these other storage arrays are similar. Figure 10 As shown, the storage array A1 includes multiple storage cells S 11 -S mn Where m is the number of rows in storage array A1, and n is the number of columns in storage array A1. Storage cell S ijIt can store weight data W ij Where i∈[1,m], j∈[1,n]. The control circuit 1020 can control the memory array A1 to be in a programming state, so as to send signals to the memory cells S. ij Write weight data W ij The control circuit 1020 can control the storage array A1 to be in a read state, so as to put the storage cell S... ij Stored weight data W ij Read out or read out the result of calculations performed on multiple memory units. In in-memory calculations, memory unit S... ij Its conduction capability can be based on the stored weight data W ij The control circuit 1020 can control the storage array A1 to be in a computing state, and the control circuit 1020 can send signals to the storage cell S through the input terminal IN of the storage array A1. ij Provide input signal V i The input signal V i Acting on storage unit S ij Storage unit S ij The stored weight data W ij This makes the storage unit S ij It has a corresponding conduction capability, under which current flows to or into the output terminal OUT. Multiple memory cells (e.g., S...) 1j -S mj The output terminals of the memory can be collinear. According to Kirchhoff's laws, the current generated by multiple memory cells accumulates to obtain the output signal I. j Satisfy the following formula:

[0115]

[0116] As can be seen, a storage array can be used to perform multiplication and accumulation calculations, and the result can be output at the output terminal. The readout circuit 1030 can sense and output the result, which can be understood as reading the calculation result. The multiple storage cells with collinear outputs can be called a calculation group.

[0117] The in-memory computing system 1000 may further include a readout circuit 1030 for reading data stored in the storage circuit 1010 and / or for converting and outputting the results of the above calculations. For example, the readout circuit 1030 may include a first conversion circuit 1031 and a second conversion circuit 1032. The first conversion circuit 1031 performs a first conversion on the output signal of the storage circuit 1010 (e.g., readout current), for example, converting the current signal into a voltage signal; the second conversion circuit 1032 performs a second conversion on the output of the first conversion, for example, converting the analog signal into a digital signal. The first conversion circuit 1031 may also be called a sampling circuit. For example, the sampling circuit may be used to sample the current signal output by the storage circuit into a voltage signal. The second conversion circuit 1032 may also be called a decision circuit. For example, the decision circuit may be used to convert the waveform parameters of the analog signal (e.g., voltage signal) into digital signals, for example, converting the amplitude, pulse width, or area of ​​the analog signal into digital signals through decision-making. This application does not limit the accuracy of the conversion or decision-making; for example, it may include 1-bit or multi-bit conversion accuracy. The control circuit 1020 can control the operation of the readout circuit 1030.

[0118] In some possible embodiments, different memory arrays may reuse the first conversion circuit 1031 and / or the second conversion circuit 1032; or, in the same memory array, different output terminals may reuse the sub-circuits in the first conversion circuit 1031 and / or the second conversion circuit 1032 to reduce the hardware cost of the readout circuit and reduce its footprint on the chip.

[0119] This application also provides a control device, including units or means for performing each step of any of the above control methods.

[0120] This application also provides a control device, which can be found in the embodiments of this application. Figure 11 . Figure 11 A schematic diagram of a control device according to an exemplary embodiment of this application is shown. Figure 11 As shown, the control device 1100 includes at least one processor 1110 and an interface circuit 1120. The at least one processor 1110 is used to execute the control method provided in any of the above embodiments, and the interface circuit 1120 is used to signal connect with the storage circuit.

[0121] A processor is a circuit with signal processing capabilities. For example, a processor can be a circuit with instruction read and execute capabilities. In other possible embodiments, the processor can implement its functions through the logical relationships of hardware circuits, which can be fixed or reconfigurable. For example, the processor is a hardware circuit implemented as an Application Specific Integrated Circuit (ASIC) or a Programmable Logic Device (PLD), such as a Field Programmable Gate Array (FPGA). In a reconfigurable hardware circuit, the process of the processor loading a configuration document to configure the hardware circuit can be understood as the process of the processor loading instructions to achieve its functions. This application does not limit the type of processor, but includes, for example, a central processing unit (CPU), a microcontroller unit (MCU), a graphics processing unit (GPU), or a digital signal processor (DSP). Alternatively, the processor can include hardware circuits designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), a tensor processing unit (TPU), or a deep learning processing unit (DPU).

[0122] This application embodiment also provides a storage circuit, including a first storage area and a second storage area. The first storage area is used to store the first weight data of the first linear layer of the first model block of the AI ​​model. The forward propagation path of the AI ​​model includes multiple model blocks, the first model block is the last model block in the forward propagation path of the model, and the forward propagation path of the first model block includes multiple linear layers, the first linear layer is the last linear layer in the forward propagation path of the first model block. The second storage area is used to store the weight data of the second linear layer of the first model block.

[0123] This application also provides a computer program product, which includes instructions that, when executed by a processor, cause any of the control methods described in the above embodiments to be executed.

[0124] This application also provides a computer-readable medium storing instructions that, when executed by a processor, cause any of the control methods described in the above embodiments to be executed.

[0125] This application embodiment also provides an in-memory computing system, which can be any of the in-memory computing systems provided in the above embodiments, such as in-memory computing system 100. The in-memory computing system may include: a first storage area for storing first weight data of a first linear layer of a first model block of an AI model, wherein the forward propagation path of the AI ​​model includes multiple model blocks, the first model block being the last model block in the forward propagation path of the model, and the forward propagation path of the first model block including multiple linear layers, the first linear layer being the last linear layer in the forward propagation path of the first model block; a second storage area for storing weight data of a second linear layer of the first model block; and at least one processing circuit for controlling the second storage area to perform a first calculation in the analog domain based on the weight data of the second linear layer; and for controlling the first weight data to be read from the first storage area, and performing a second calculation in the digital domain based on the read first weight data.

[0126] This application also provides an electronic device, which can be found in [reference 1]. Figure 12 . Figure 12 A schematic diagram of an electronic device according to an exemplary embodiment of this application is shown. Figure 12 As shown, the electronic device 1200 may include any of the above-mentioned in-memory computing systems 1210 for processing data from the electronic device. The electronic device may also include an input / output device 1220 for receiving user input or outputting processing results. This application does not limit the input and output types; for example, input may include voice input, text input, image input, or video input. Output may include text output, voice output, image output, or video output. The electronic device may also include a processor 1230, which can process data provided to the in-memory computing system 1210 or process the output data of the in-memory computing system 1210. The output of the input / output device 1220 may be based on the output of the processor 1230 or the output of the in-memory computing system 1210.

[0127] This application does not limit the type of electronic device. For example, according to some embodiments, the electronic device may include wearable devices. Wearable devices include, but are not limited to: head-mounted devices (e.g., helmets or hats), devices worn on the ears (e.g., headphones), devices worn on the wrist (e.g., watches), and devices worn on other parts of the body (e.g., electronic necklaces, medical monitoring devices, or glasses). According to some embodiments, the electronic device may include portable terminals. For example, the electronic device may include, but is not limited to, mobile phones, general-purpose computing devices (e.g., laptops or tablets), personal digital assistants, etc. According to some embodiments, the electronic device may include other types of edge devices, such as personal computers, in-vehicle computers or in-vehicle computing platforms, or smart home electronic products. According to some embodiments, the electronic device may also include devices such as servers.

[0128] In the above embodiments, the descriptions of different embodiments each have their own emphasis. Parts not described in detail or recorded in a certain embodiment can be referred to in the relevant descriptions of other embodiments. Furthermore, the different embodiments described above can be freely combined as needed. Moreover, as technology evolves, the elements described in this application can be replaced by equivalent elements appearing after this application.

Claims

1. An in-memory computing system, characterized in that, include: The first storage area is used to store the first weight data of the first linear layer of the first model block of the artificial intelligence model. The forward propagation path of the artificial intelligence model includes multiple model blocks. The first model block is the last model block in the forward propagation path of the artificial intelligence model. The forward propagation path of the first model block includes multiple linear layers. The first linear layer is the last linear layer in the forward propagation path of the first model block. The second storage area is used to store the weight data of the second linear layer of the first model block; At least one processing circuit is used to control the second storage area to perform a first calculation of the analog domain based on the weight data of the second linear layer; It is also used to control the reading of the first weight data from the first storage area, and to perform a second calculation in the digital field based on the read first weight data; Wherein, the result of the first calculation is used as the input for the second calculation; or The result of the first calculation is used as the input to the third calculation of the simulation domain, and the result of the third calculation is used as the input to the second calculation. The weight data corresponding to the weight parameters of the first linear layer includes the first weight data and the second weight data. The second storage area is also used to store the second weight data. The at least one processing circuit is also used to control the second storage area to perform the third calculation based on the second weight data.

2. The in-memory computing system according to claim 1, characterized in that, The first weight data includes a first array, the second weight data includes a second array, and the weight data corresponding to the weight parameters of the first linear layer includes a third array. The first array and the second array are determined based on the third array.

3. The in-memory computing system according to claim 2, characterized in that, The first array and the second array are obtained by decomposing the third array using singular value decomposition.

4. The in-memory computing system according to claim 2 or 3, characterized in that, The first dimension and the second dimension of the first array are equal to the second dimension of the third array, the first dimension of the second array are equal to the first dimension of the third array, and the second dimension of the second array are equal to the second dimension of the third array.

5. The in-memory computing system according to claim 4, characterized in that, The second array is equal to the product of the first part and the second part, the first dimension and the second dimension of the first part are equal to the first dimension of the third array, the first dimension of the second part are equal to the first dimension of the third array, and the second dimension of the second part are equal to the second dimension of the third array.

6. The storage system according to any one of claims 1 to 3, characterized in that, When the first dimension of the weight data corresponding to the weight parameters of the first linear layer is less than the second dimension, the weight data corresponding to the weight parameters of the first linear layer is stored in the first storage area. or When the first dimension of the weight data corresponding to the weight parameters of the first linear layer is greater than the second dimension, the first weight data and the second weight data corresponding to the weight parameters of the first linear layer are stored in the first storage area and the second storage area, respectively. or When the first dimension of the weight data corresponding to the weight parameters of the first linear layer is equal to the second dimension, the first weight data and the second weight data corresponding to the weight parameters of the first linear layer are stored in the first storage area and the second storage area respectively, or the weight data corresponding to the weight parameters of the first linear layer are stored in the first storage area.

7. The storage system according to any one of claims 1 to 3, characterized in that, The first storage area and the second storage area are located in the same storage circuit, or the first storage area and the second storage area are located in different storage circuits.

8. The storage system according to any one of claims 1 to 3, characterized in that, The second storage area is also used to store the weight data of the linear layer of the second model block among the plurality of model blocks; The at least one processing circuit is further configured to control the second storage area to perform a fourth calculation of the simulation domain based on the weight data of the linear layer of the second model block.

9. An electronic device, characterized in that, Includes the storage system as described in any one of claims 1 to 8.

10. A control method, characterized in that, include: Obtain the weight data of the artificial intelligence model; The first weight data of the first linear layer of the first model block of the artificial intelligence model is stored in the first storage area of ​​the in-memory computing system. The forward propagation path of the artificial intelligence model includes multiple model blocks. The first model block is the last model block in the forward propagation path of the artificial intelligence model. The forward propagation path of the first model block includes multiple linear layers. The first linear layer is the last linear layer in the forward propagation path of the first model block. The weight data of the second linear layer of the first model block is stored in the second storage area of ​​the in-memory computing system; The second storage region is controlled to perform a first calculation of the simulation domain based on the weight data of the second linear layer; Read the first weight data from the first storage area, and perform a second calculation in the digital field based on the read first weight data; The control method further includes: reading the result of the first calculation; the second calculation in the digital domain based on the read first weight data includes: performing a second calculation in the digital domain based on the result of the first calculation and the read first weight data; or The weight data corresponding to the weight parameters of the first linear layer includes the first weight data and the second weight data. The control method further includes: storing the second weight data in the second storage area; inputting the result of the first calculation into the second storage area; controlling the second storage area to perform a third calculation in the analog domain based on the result of the first calculation and the second weight data; reading out the result of the third calculation; the second calculation in the digital domain based on the read first weight data includes: performing a second calculation in the digital domain based on the result of the read third calculation and the read first weight data.

11. The control method according to claim 10, characterized in that, The first weight data includes a first array, the second weight data includes a second array, and the weight data corresponding to the weight parameters of the first linear layer includes a third array. The first array and the second array are determined based on the third array.

12. The control method according to claim 11, characterized in that, The first array and the second array are obtained by decomposing the third array using singular value decomposition.

13. The control method according to claim 12, characterized in that, The first dimension and the second dimension of the first array are equal to the second dimension of the third array, the first dimension of the second array are equal to the first dimension of the third array, and the second dimension of the second array are equal to the second dimension of the third array.

14. The control method according to claim 13, characterized in that, The second array is equal to the product of the first part and the second part, the first dimension and the second dimension of the first part are equal to the first dimension of the third array, the first dimension of the second part are equal to the first dimension of the third array, and the second dimension of the second part are equal to the second dimension of the third array.

15. The control method according to any one of claims 10 to 14, characterized in that, When the first dimension of the weight data corresponding to the weight parameters of the first linear layer is less than the second dimension, the weight data corresponding to the weight parameters of the first linear layer is stored in the first storage area. or When the first dimension of the weight data corresponding to the weight parameters of the first linear layer is greater than the second dimension, the first weight data and the second weight data corresponding to the weight parameters of the first linear layer are stored in the first storage area and the second storage area, respectively. or When the first dimension of the weight data corresponding to the weight parameters of the first linear layer is equal to the second dimension, the first weight data and the second weight data corresponding to the weight parameters of the first linear layer are stored in the first storage area and the second storage area respectively, or the weight data corresponding to the weight parameters of the first linear layer are stored in the first storage area.

16. The control method according to any one of claims 10 to 14, characterized in that, The control method further includes: The weight data of the linear layer of the second model block among the plurality of model blocks is stored in the second storage area; The second storage area is controlled to perform a fourth calculation of the simulation domain based on the weight data of the linear layer of the second model block.

17. A control device, characterized in that, include: A processing circuit for executing the control method as described in any one of claims 10 to 16; An interface circuit is used for signal connection with at least one storage circuit, the at least one storage circuit including a first storage area and a second storage area.

Images