Tensor-based continuous learning method and device

Abstract

The present application discloses a continuous learning method and device based on tensors, which includes: obtaining input data; the input data includes one or more of video, image, text or voice; inputting the input data into a first neural network to obtain a data processing result; wherein the first neural network is obtained through m task training, and after the training of the i-th task is completed, the neural network contains A tensor cores, the A tensor cores are divided into B tensor layers, and each tensor layer in the B tensor layers contains data of each tensor core in the A tensor cores on the same dimension; in the training of the i+1-th task, the first neural network adds C tensor cores and / or D tensor layers, and the parameters in the C tensor cores and / or D tensor layers are updated. Through the present application, the anti-forgetting ability of the model can be effectively improved, and the scale of the model increases slightly, thereby effectively saving storage and communication overhead.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence (AI) technology in big data, and in particular to a tensor-based continuous learning method and device. Background Art

[0002] Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a branch of computer science that seeks to understand the essence of intelligence and develop new intelligent machines that can respond in a manner similar to human intelligence. AI also involves studying the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0003] With the rapid development of internet technologies such as big data, artificial intelligence, and the Internet of Things, various industries are gradually moving towards digitalization and intelligentization to help improve service efficiency and quality. Among these, interactive methods such as digital humans and virtual humans are gradually emerging in various fields such as finance, e-commerce, healthcare, education, and multimedia.

[0004] Traditional machine learning (ML) requires that training samples be independent and identically distributed (IID). To achieve this, the training data is often randomly shuffled. However, if the model is trained using traditional methods without any processing on continuous data with varying distributions, catastrophic forgetting will occur because the new data will interfere with the model. The model will adjust the parameters it learned from the old data to adapt to the new task, thus forgetting the knowledge learned from the old data.

[0005] Therefore, continuous learning came into being. Existing continuous learning methods mainly achieve the purpose of continuous learning by limiting the update of existing model parameters or adding new sub-models during the training of new tasks.

[0006] However, existing continuous learning methods have poor anti-forgetting capabilities, or after multiple continuous learning cycles, the model becomes large in size, which increases storage and communication overhead. Summary of the Invention

[0007] The embodiments of the present application provide a tensor-based continuous learning method and device, which can effectively improve the anti-forgetting ability of the model, and the scale of the model increases slightly, thereby effectively saving storage and communication overhead.

[0008] In a first aspect, the present application provides a tensor-based continuous learning method, the method comprising: obtaining input data; wherein the input data includes one or more of video, image, text or voice; inputting the input data into a first neural network to obtain a data processing result; wherein the first neural network is obtained by training m tasks, and after the training of the i-th task is completed, the neural network contains A tensor cores, the A tensor cores are divided into B tensor layers, and each tensor layer in the B tensor layers contains data of each tensor core in the A tensor cores on the same dimension; after the training of the i+1-th task is completed, the first neural network adds C tensor cores and / or D tensor layers, and in the training of the i+1-th task, the parameters in the C tensor cores and / or the D tensor layers are updated, m is a positive integer, and i is a positive integer less than or equal to m-1.

[0009] From the technical effect point of view, in the continuous learning process (that is, the process of a neural network that has learned an old task learning a new task), by updating the parameters in the newly added tensor cores and / or tensor layers, the parameters in the original tensor cores can be kept unchanged, that is, the neural network is updated in the dimensions of the tensor cores and / or tensor layers. Compared with the existing technology of directly updating the entire tensor in the neural network when training a new task (the tensor core is obtained by tensor decomposition of the tensor), it can effectively retain the knowledge learned from the previous task, and also use the newly added tensor cores and / or tensor layers to save the knowledge learned from the new task, thereby making the neural network have a higher precision anti-forgetting ability.

[0010] In a feasible implementation, after the training of the i+1th task is completed, the parameters of some or all of the tensor cores in the A tensor cores remain unchanged, or the parameters of some or all of the tensor layers in the B tensor layers remain unchanged.

[0011] From a technical perspective, for two adjacent tasks, during the training of the second task, by keeping some or all of the parameters in the existing tensor cores / tensor layers in the neural network unchanged, the neural network can retain the knowledge learned from the previous task during the continuous learning process without the need for major adjustments to the model structure and scale.

[0012] In a feasible embodiment, the method further includes: after the training of the i-th task is completed, performing tensor combination on the A tensor cores to obtain one or more tensors; and before executing the training of the i+1-th task, performing tensor decomposition on the one or more tensors to obtain the A tensor cores.

[0013] From the technical effect point of view, the neural network used for continuous learning can be a neural network that only contains tensors but not tensor cores during the design. However, during the training process of the task, the tensor is first decomposed into tensors, and then the parameters in the tensor cores / or tensor layers obtained by the decomposition are updated more accurately. Similarly, the neural network can also be a neural network that contains tensor cores / tensor layers during the design. There is no need to perform tensor decomposition and tensor combination during the training process of the task. Different methods can be used to design neural networks, and the method in this application can be applicable to all of them, which has good versatility.

[0014] In a feasible implementation, the training process of the i+1th task includes: training the i-th backup tensor network using the i+1th batch of data sets to obtain the trained i-th backup tensor network; and training the first neural network using the i+1th batch of data sets; wherein the loss function of the first neural network includes the degree of difference between the output of the first neural network and the output of the trained j-th backup tensor network, or the loss function of the first neural network includes the degree of difference between the model parameters of the first neural network and the model parameters of the trained j-th backup tensor network, j=1 to i-1, j is a positive integer less than or equal to i-1.

[0015] From a technical perspective, during the training of a new task for a neural network, the backup tensor network trained for the previous task will be used to constrain the update of the neural network. This method, combined with the tensor core / tensor layer update method of the neural network in the aforementioned embodiment, can make the neural network have better anti-forgetting ability after training for the new task.

[0016] In a feasible embodiment, the first neural network is one of a plurality of neural networks, and the plurality of neural networks are respectively located on different user devices, and the first neural network is located on the first user device. The method further includes: after the training of each task is completed, the first user device sends the model parameters of the first neural network to the server, so that the server updates the second neural network on the server according to the model parameters of each neural network in the plurality of neural networks; wherein the model parameters of the first neural network include the tensor cores contained in the first neural network.

[0017] From the perspective of technical effects, the continuous learning method of the present application can be applied to the federated learning architecture. By updating the neural network parameters using the continuous learning method in the aforementioned embodiment on different user devices, the neural network can maintain good anti-forgetting capabilities without significantly changing its scale. At the same time, only the tensor core of the neural network on each user device is sent to the server for model aggregation, which can effectively save communication overhead in the federated learning process.

[0018] In a feasible implementation, the task includes image recognition, target detection, image segmentation or speech semantic recognition.

[0019] From the perspective of technical effects, the continuous learning method in this application is not restricted by task categories and can be applied to various types of application scenarios with strong versatility.

[0020] In a second aspect, an embodiment of the present application provides a tensor-based continuous learning method, the method comprising: obtaining input data; wherein the input data includes one or more of video, image, text or voice; inputting the input data into a first neural network to obtain a data processing result; wherein the first neural network is obtained by training m tasks, and after the training of the i-th task is completed, the neural network contains A tensor cores, the A tensor cores contain B tensor layers, and each tensor layer in the B tensor layers contains data of the same dimension for each tensor core in the A tensor cores; after the training of the i+1-th task is completed, the parameters of some or all of the tensor cores in the A tensor cores remain unchanged, or the parameters of some or all of the tensor layers in the B tensor layers remain unchanged, m is a positive integer, and i is a positive integer less than or equal to m-1.

[0021] In a feasible embodiment, after the training of the i+1th task is completed, the first neural network adds C tensor cores and / or D tensor layers, and during the training of the i+1th task, the parameters in the C tensor cores and / or the D tensor layers are updated.

[0022] From the perspective of technical effects, the technical effects of the above embodiments can be found in the description of the corresponding embodiments in the above-mentioned first aspect, and will not be repeated here.

[0023] In a third aspect, an embodiment of the present application provides a tensor-based continuous learning method, the method comprising: receiving multiple model parameters respectively sent by multiple user devices, wherein the multiple user devices include a first user device, and the model parameters sent by the first user device include tensor cores contained in a first neural network; updating a second neural network based on the multiple model parameters, and processing the data to be processed based on the updated second neural network to obtain a data processing result; wherein the data to be processed includes one or more of pictures, videos, voices or texts; wherein the first neural network The network is obtained by training for m tasks. After the training of the i-th task, the neural network includes A tensor cores, and the A tensor cores are divided into B tensor layers. Each tensor layer in the B tensor layers contains data of each tensor core in the A tensor cores on the same dimension. After the training of the i+1-th task, the first neural network adds C tensor cores and / or D tensor layers, and in the training of the i+1-th task, the parameters of the C tensor cores and / or the D tensor layers are updated, m is a positive integer, and i is a positive integer less than or equal to m-1.

[0024] In a feasible implementation, after the training of the i+1th task is completed, the parameters of some or all of the tensor cores in the A tensor cores remain unchanged, or the parameters of some or all of the tensor layers in the B tensor layers remain unchanged.

[0025] From the perspective of technical effects, the technical effects of the above embodiments can be found in the description of the corresponding embodiments in the above-mentioned first aspect, and will not be repeated here.

[0026] In a fourth aspect, an embodiment of the present application provides a tensor-based continuous learning method, characterized in that the method includes: receiving multiple model parameters sent by multiple user devices respectively, wherein the multiple user devices include a first user device, and the model parameters sent by the first user device include tensor cores included in a first neural network; updating a second neural network based on the multiple model parameters, and processing the data to be processed based on the updated second neural network to obtain a data processing result; wherein the data to be processed includes one or more of pictures, videos, voice or text; wherein the first neural network is obtained by training m tasks, and after the training of the i-th task is completed, the neural network contains A tensor cores, the A tensor cores are divided into B tensor layers, and each tensor layer in the B tensor layers contains data of the same dimension for each tensor core in the A tensor cores; after the training of the i+1-th task is completed, the parameters of some or all tensor cores in the A tensor cores remain unchanged, or the parameters of some or all tensor layers in the B tensor layers remain unchanged, m is a positive integer, and i is a positive integer less than or equal to m-1.

[0027] In a feasible embodiment, after the training of the i+1th task is completed, the first neural network adds C tensor cores and / or D tensor layers, and during the training of the i+1th task, the parameters in the C tensor cores and / or the D tensor layers are updated.

[0028] From the perspective of technical effects, the technical effects of the above embodiments can be found in the description of the corresponding embodiments in the above-mentioned first aspect, and will not be repeated here.

[0029] In a fifth aspect, an embodiment of the present application provides a tensor-based continuous learning device, comprising: an acquisition unit for acquiring input data; wherein the input data includes one or more of video, image, text or voice; a processing unit for processing the input data based on a first neural network to obtain a data processing result; wherein the first neural network is obtained through m task training, and after the training of the i-th task is completed, the neural network contains A tensor cores, and the A tensor cores are divided into B tensor layers, and each tensor layer in the B tensor layers contains data of the same dimension for each tensor core in the A tensor cores; after the training of the i+1-th task is completed, the first neural network adds C tensor cores and / or D tensor layers, and in the training of the i+1-th task, the parameters in the C tensor cores and / or the D tensor layers are updated, m is a positive integer, and i is a positive integer less than or equal to m-1.

[0030] In a feasible implementation, after the training of the i+1th task is completed, the parameters of some or all of the tensor cores in the A tensor cores remain unchanged, or the parameters of some or all of the tensor layers in the B tensor layers remain unchanged.

[0031] In a feasible embodiment, the processing unit is also used to: after the training of the i-th task is completed, perform tensor combination on the A tensor cores to obtain one or more tensors; and before executing the training of the i+1-th task, perform tensor decomposition on the one or more tensors to obtain the A tensor cores.

[0032] In a feasible embodiment, during the training process of the i+1th task, the processing unit is specifically used to: use the i+1th batch of data sets to train the i-th backup tensor network to obtain the trained i-th backup tensor network; and use the i+1th batch of data sets to train the first neural network; wherein the loss function of the first neural network includes the degree of difference between the output of the first neural network and the output of the trained j-th backup tensor network, or the loss function of the first neural network includes the degree of difference between the model parameters of the first neural network and the model parameters of the trained j-th backup tensor network, j=1 to i-1, j is a positive integer less than or equal to i-1.

[0033] In a feasible embodiment, the first neural network is one of a plurality of neural networks, and the plurality of neural networks are respectively located on different user devices, and the first neural network is located on the first user device. The apparatus further includes: a sending unit, configured to, after the training of each task is completed, cause the first user device to send the model parameters of the first neural network to the server, so that the server updates the second neural network on the server according to the model parameters of each neural network in the plurality of neural networks; wherein the model parameters of the first neural network include the tensor cores contained in the first neural network.

[0034] In a feasible implementation, the task includes image recognition, target detection, image segmentation or speech semantic recognition.

[0035] In a sixth aspect, an embodiment of the present application provides a tensor-based continuous learning device, the device comprising: an acquisition unit for acquiring input data; wherein the input data includes one or more of video, image, text or voice; a processing unit for inputting the input data into a first neural network to obtain a data processing result; wherein the first neural network is obtained through m task training, and after the training of the i-th task is completed, the neural network contains A tensor cores, the A tensor cores contain B tensor layers, and each tensor layer in the B tensor layers contains data of the same dimension for each tensor core in the A tensor cores; after the training of the i+1-th task is completed, the parameters of some or all of the tensor cores in the A tensor cores remain unchanged, or the parameters of some tensor layers in the B tensor layers remain unchanged, m is a positive integer, and i is a positive integer less than or equal to m-1.

[0036] In a feasible embodiment, after the training of the i+1th task is completed, the first neural network adds C tensor cores and / or D tensor layers, and during the training of the i+1th task, the parameters in the C tensor cores and / or the D tensor layers are updated.

[0037] In a seventh aspect, an embodiment of the present application provides a tensor-based continuous learning device, the device comprising: a receiving unit for receiving multiple model parameters respectively sent by multiple user devices, wherein the multiple user devices include a first user device, and the model parameters sent by the first user device include tensor cores contained in a first neural network; a processing unit for updating a second neural network based on the multiple model parameters, and processing the data to be processed based on the updated second neural network to obtain a data processing result; wherein the data to be processed includes one or more of pictures, videos, voices or texts; wherein, The first neural network is obtained by training m tasks. After the training of the i-th task is completed, the neural network contains A tensor cores, and the A tensor cores are divided into B tensor layers. Each tensor layer in the B tensor layers contains data of the same dimension for each tensor core in the A tensor cores; after the training of the i+1-th task is completed, the first neural network adds C tensor cores and / or D tensor layers, and in the training of the i+1-th task, the parameters of the C tensor cores and / or the D tensor layers are updated, m is a positive integer, and i is a positive integer less than or equal to m-1.

[0038] In a feasible implementation, after the training of the i+1th task is completed, the parameters of some or all of the tensor cores in the A tensor cores remain unchanged, or the parameters of some or all of the tensor layers in the B tensor layers remain unchanged.

[0039] In an eighth aspect, an embodiment of the present application provides a tensor-based continuous learning device, the device comprising: a receiving unit for receiving multiple model parameters respectively sent by multiple user devices, wherein the multiple user devices include a first user device, and the model parameters sent by the first user device include tensor cores included in a first neural network; a processing unit for updating a second neural network based on the multiple model parameters, and processing the data to be processed based on the updated second neural network to obtain a data processing result; wherein the data to be processed includes one or more of pictures, videos, voice or text; wherein the first neural network is obtained by training m tasks, and after the training of the i-th task is completed, the neural network includes A tensor cores, the A tensor cores are divided into B tensor layers, and each tensor layer in the B tensor layers contains data of the same dimension for each tensor core in the A tensor cores; after the training of the i+1-th task is completed, the parameters in some or all of the tensor cores in the A tensor cores remain unchanged, or the parameters in some tensor layers in the B tensor layers remain unchanged, m is a positive integer, and i is a positive integer less than or equal to m-1.

[0040] In a feasible embodiment, after the training of the i+1th task is completed, the first neural network adds C tensor cores and / or D tensor layers, and during the training of the i+1th task, the parameters in the C tensor cores and / or the D tensor layers are updated.

[0041] In the ninth aspect, an embodiment of the present application provides an electronic device, which includes at least one processor, a memory and an interface circuit, wherein the memory, the interface circuit and the at least one processor are interconnected through lines, and instructions are stored in the at least one memory; when the instructions are executed by the processor, any method described in the first to fourth aspects above is implemented.

[0042] In a tenth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed, the method described in any one of the first to fourth aspects above is implemented.

[0043] In an eleventh aspect, an embodiment of the present application provides a computer program, which includes instructions. When the computer program is executed, the method described in any one of the first to fourth aspects above is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The following is an introduction to the drawings used in the embodiments of this application.

[0045] Figure 1a-Figure 1c Schematic diagrams of several system architectures that can execute the tensor-based continuous learning method of the present application, provided in embodiments of the present application;

[0046] Figure 2 Another system architecture diagram provided for an embodiment of the present application;

[0047] Figure 3 A schematic diagram of the structure of a convolutional neural network provided in an embodiment of the present application;

[0048] Figure 4 A schematic diagram of the structure of another convolutional neural network provided in an embodiment of the present application;

[0049] Figure 5 A schematic diagram of a chip hardware structure provided in an embodiment of the present application;

[0050] Figure 6 A flowchart of a tensor-based continuous learning method provided in an embodiment of the present application;

[0051] Figure 7 A schematic diagram of the relationship between a tensor core and a tensor layer provided in an embodiment of the present application;

[0052] Figure 8a-8c A schematic diagram of a method for changing a tensor structure provided in an embodiment of the present application;

[0053] Figure 9a-9b A schematic diagram of a method for updating a tensor structure provided in an embodiment of the present application;

[0054] Figure 10 A structure of a first neural network provided in an embodiment of the present application;

[0055] Figure 11 A schematic diagram of an alternating training process provided in an embodiment of the present application;

[0056] Figure 12 A schematic diagram of a continuous learning process based on a federated learning architecture provided in an embodiment of the present application;

[0057] Figure 13 A flowchart of another continuous learning method provided in an embodiment of the present application;

[0058] Figure 14 A flowchart of another continuous learning method provided in an embodiment of the present application;

[0059] Figure 15 A flowchart of another continuous learning method provided in an embodiment of the present application;

[0060] Figure 16 A schematic diagram of the structure of a continuous learning device provided in an embodiment of the present application;

[0061] Figure 17 A schematic diagram of another continuous learning device structure provided in an embodiment of the present application;

[0062] Figure 18 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0063] The following describes the embodiments of the present application in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " represents or, for example, A / B can represent A or B; "and / or" in the text is merely a description of the association relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" refers to two or more than two.

[0064] The terms "first", "second", "third" and "fourth" in the specification and claims of this application and the accompanying drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices. Reference to "embodiments" herein means that the specific features, structures or characteristics described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0065] First, the professional terms in this application are introduced and described accordingly

[0066] (1) Tensor: A multidimensional array. A scalar can be considered a 0-dimensional tensor, a vector a 1-dimensional tensor, and a matrix a 2-dimensional tensor. In practice, data is usually referred to as a tensor only when its dimension is greater than three-dimensional.

[0067] (2) Tensor decomposition: It is essentially a higher-order generalization of matrix decomposition. It is usually used for dimensionality reduction, missing data filling (or "sparse data filling"), and implicit relationship mining. Common tensor decomposition methods include CP decomposition, Tucker decomposition, t-SVD decomposition, etc.

[0068] (3) Tensor Core: The factor matrix of the original tensor obtained by tensor decomposition is the tensor core.

[0069] (4) Tensor layer: a data set of different tensor cores on the same dimension.

[0070] (5) Federated Learning: During the machine learning process, each participant can use the data of other parties to jointly model the data. The parties do not need to share data resources, that is, the data does not leave the local area, and the data is jointly trained to build a shared machine learning model.

[0071] (6) Continual Learning / Life-Long Learning: When learning a new task, the experience of the previous task can be used to quickly learn the new task while maintaining the memory of the old task. This is a type of deep learning with anti-forgetting ability.

[0072] The following examples introduce two types of application scenarios to which the continuous learning method in this application is applicable. It should be understood that they do not constitute a limitation on the scope of application scenarios of the method in this application.

[0073] (1) Computer Vision (CV): The continuous learning method in this application can be used to train models for various tasks in the field of CV (e.g., object detection, image segmentation, image classification, etc.), and then use the trained models for inference. The trained models can learn from the training data of new tasks and also retain knowledge from previous tasks.

[0074] (2) Natural Language Processing (NLP): The continuous learning method in this application can be used to train models for various tasks in the field of natural language processing (e.g., speech semantic recognition, virtual human video generation, etc.), and then use the trained models for inference. The trained models can learn from the training data of new tasks and retain knowledge from previous tasks.

[0075] See Figure 1a-Figure 1c , Figure 1a-Figure 1c Schematic diagram of several system architectures provided in the embodiments of the present application that can execute the tensor-based continuous learning method of the present application.

[0076] in, Figure 1a The system shown includes a user device 110 and a data processing device 120 (server). The user device 110 includes a smart terminal such as a mobile phone, personal computer, vehicle-mounted terminal, or information processing center. The user device 110 is the initiator of data processing, and users usually initiate requests through the user device 110.

[0077] The data processing device 120 can be a device or server with data processing capabilities, such as a cloud server, a network server, an application server, and a management server. The data processing device 120 receives query statements / voice / text and other requests from the user device 110 through the interactive interface, and then performs data processing such as machine learning, deep learning, search, reasoning, and decision-making through the memory for storing data and the processor for data processing to execute the continuous learning method in this application, and then uses the neural network trained by the continuous learning method to perform reasoning, and finally transmits the data processing results obtained by reasoning to the user device 110 through the network. The memory can be a general term, including local storage and a database for storing historical data. The database can be on the data processing device or on other network servers.

[0078] Figure 1b The user device 110 in the system shown directly acts as a data processing device, directly receiving input from the user and directly processing it by the hardware of the user device 110 itself. The specific process is the same as Figure 1a Similarly, please refer to the above description and will not be repeated here.

[0079] Figure 1c The system shown includes at least one local device (such as local device 301 and local device 302), an execution device 210 and a data storage system 250. The local device is equivalent to Figure 1a and Figure 1b In the user device 110, the execution device 210 is equivalent to the data processing device 120, and the data storage system 250 can be integrated on the execution device 210, or it can be set on the cloud or other network servers.

[0080] See Figure 2 , Figure 2 Another system architecture diagram provided in this application embodiment. Figure 2 As shown, the data acquisition device 260 is used to collect voice, text, image, video and other data and store them in the database 230. The training device 220 trains the neural network 201 (i.e., the first neural network in this application) based on the image and text data maintained in the database 230. Figure 6 The illustrated method embodiment will describe in detail how training device 220 continuously trains neural network 201. Trained neural network 201 can process one or more input data types, such as text, speech, images, and video, to generate data processing results corresponding to target tasks (e.g., object detection, image recognition, speech semantic recognition, etc.). The input data is based on user requests sent by client device 240.

[0081] Figure 2It is also a functional module diagram in the execution process of the continuous learning method. Figure 1a-Figure 1c In the system (ie, the actual application scenario diagram), the client device 240 can be Figure 1a-Figure 1c In the user device 110 or local device, the execution device 210 and the data storage system 250 can be integrated into the user device 110 or the local device when the data processing capability of the user device 110 is relatively strong. In some embodiments, the execution device 210 and the data storage system 250 can also be integrated into the user device 110 or the local device. Figure 1a The database 230, the training device 220 and the data acquisition device 260 can be integrated into the data processing device 120. Figure 1a The data processing device 120 in the system may be configured as follows, or configured as another server on the cloud or the network, which is not limited in this application.

[0082] The data acquisition device 260 may be a terminal device or an input / output interface of a server or cloud, and may be an interactive layer (interface) for obtaining query statements and returning reply statements.

[0083] The following is a brief introduction to the training and reasoning principles of the deep learning model in this application.

[0084] The architecture of a deep learning model can be a deep neural network. The work of each layer in a deep neural network can be expressed mathematically as To describe: From a physical perspective, the work of each layer in a deep neural network can be understood as completing the transformation from input space to output space (i.e., from the row space to the column space of a matrix) through operations on five input spaces (a set of input vectors). These five operations include: 1. Dimensionality increase / decrease; 2. Zoom in / out; 3. Rotation; 4. Translation; 5. “Bending”. Operations 1, 2, and 3 are performed by W Completed, operation 4 is completed by +b, and operation 5 is implemented by a(). The word "space" is used here because the object being classified is not a single thing, but a class of things, and space refers to the collection of all individuals of this class of things. Among them, W is a weight vector, and each value in the vector represents the weight value of a neuron in this layer of the neural network. This vector W determines the spatial transformation from the input space to the output space described above, that is, the weight W of each layer controls how to transform the space. The purpose of training a deep neural network is to eventually obtain the weight matrix of all layers of the trained neural network (the weight matrix formed by many layers of vectors W). Therefore, the training process of a neural network is essentially to learn how to control spatial transformation, and more specifically to learn the weight matrix.

[0085] Because we want the output of a deep neural network to be as close as possible to the value we really want to predict, we can compare the current network's predicted value with the target value we really want, and then update the weight vector of each layer of the neural network based on the difference between the two (of course, there is usually an initialization process before the first update, which is to pre-configure the parameters for each layer in the deep neural network). For example, if the network's predicted value is too high, the weight vector is adjusted to make it predict a lower value, and this adjustment is continued until the neural network can predict the target value we really want. Therefore, it is necessary to pre-define "how to compare the difference between the predicted value and the target value." This is the loss function or objective function, which are important equations used to measure the difference between the predicted value and the target value. Taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference. Then, the training of the deep neural network becomes a process of minimizing this loss as much as possible.

[0086] exist Figure 2 In the example, the neural network 201 obtained by the training device 220 can be applied to different systems or devices. The execution device 210 is configured with an I / O interface 212 for data exchange with external devices. A "user" can input data, i.e., a user request, into the I / O interface 212 through a client device 240, including user voice or user-entered text information, or image or video information for input.

[0087] The execution device 210 can call data, code, etc. in the data storage system 250 , and can also store data, instructions, etc. in the data storage system 250 .

[0088] The calculation module 211 uses the neural network 201 to process the input data (ie, the user request), thereby generating a data processing result corresponding to the target task.

[0089] Finally, the I / O interface 212 returns the data processing result to the client device 240 and presents it to the user on the client device 240 .

[0090] More deeply, the training device 220 can obtain a neural network 201 suitable for new tasks based on different task training according to different scenario requirements, while maintaining compatible processing capabilities for old tasks to provide users with better results.

[0091] exist Figure 2In the case shown in , the user can manually specify the data to be input into the execution device 210, for example, by operating in the interface provided by the I / O interface 212. In another case, the client device 240 can automatically input data into the I / O interface 212 and obtain results. If the automatic input of data by the client device 240 requires user authorization, the user can set the corresponding permissions in the client device 240. The user can view the results output by the execution device 210 on the client device 240, and the specific presentation form can be specific methods such as display, sound, action, etc. The client device 240 can also serve as a data acquisition terminal to store the collected video, image, voice and text data in the database 230 for use in the training process.

[0092] It is worth noting that Figure 2 This is only a schematic diagram of a system architecture provided by an embodiment of the present invention. Figure 2 The positional relationship between the devices, components, modules, etc. shown in the figure does not constitute any limitation. For example, Figure 2 In the embodiment, the data storage system 250 is an external memory relative to the execution device 210. In other cases, the data storage system 250 can also be placed in the execution device 210.

[0093] See Figure 3 , Figure 3 The structural diagram of a convolutional neural network provided in an embodiment of the present application is used as an example to characterize the relevant internal structure of the neural network in the present application, but is not limiting.

[0094] A convolutional neural network (CNN) is a deep neural network with a convolutional structure and a deep learning architecture. Deep learning architectures use machine learning algorithms to perform multiple levels of learning at different levels of abstraction. As a deep learning architecture, a CNN is a feed-forward artificial neural network in which individual neurons respond to overlapping regions in an input image.

[0095] like Figure 3 As shown, a convolutional neural network (CNN) 100 may include an input layer 110 , a convolutional layer / pooling layer 120 , wherein the pooling layer is optional, and a neural network layer 130 .

[0096] Convolutional layer:

[0097] like Figure 3As shown, the convolutional / pooling layers 120 may include layers 121-126. In one implementation, layer 121 is a convolutional layer, layer 122 is a pooling layer, layer 123 is a convolutional layer, layer 124 is a pooling layer, layer 125 is a convolutional layer, and layer 126 is a pooling layer. In another implementation, layers 121 and 122 are convolutional layers, layer 123 is a pooling layer, layers 124 and 125 are convolutional layers, and layer 126 is a pooling layer. That is, the output of a convolutional layer can serve as the input to a subsequent pooling layer, or as the input to another convolutional layer to continue the convolution operation.

[0098] Taking the convolution layer 121 as an example, the convolution layer 121 can include many convolution operators, which are also called kernels. Their role in this application is equivalent to a filter that extracts specific information from the input speech or semantic information. The convolution operator can essentially be a weight matrix, which is usually predefined.

[0099] The weight values ​​in these weight matrices need to be obtained through a lot of training in practical applications. The weight matrices formed by the weight values ​​obtained through training can extract information from the input video / image / speech / text, thereby helping the convolutional neural network 100 to make correct predictions.

[0100] When the convolutional neural network 100 has multiple convolutional layers, the initial convolutional layer (for example, 121) often extracts more general features, which can also be called low-level features. As the depth of the convolutional neural network 100 increases, the features extracted by the later convolutional layers (for example, 125) become more and more complex, such as high-level image and semantic features. Features with higher semantics are more suitable for the problem to be solved.

[0101] Pooling layer:

[0102] Since it is often necessary to reduce the number of training parameters, it is often necessary to periodically introduce a pooling layer after the convolution layer, such as Figure 3 The layers 121-126 in the example of Figure 120 can be a convolutional layer followed by a pooling layer, or multiple convolutional layers followed by one or more pooling layers. In natural language data processing, the sole purpose of the pooling layer is to reduce the spatial size of the data.

[0103] Neural Network Layer 130:

[0104] After being processed by the convolution layer / pooling layer 120, the convolution neural network 100 is not sufficient to output the required output information. As mentioned above, the convolution layer / pooling layer 120 only extracts features and reduces the parameters brought by the input data. However, in order to generate the final output information (the required class information or other related information), the convolution neural network 100 needs to use the neural network layer 130 to generate one or a group of outputs of the required number of classes. Therefore, the neural network layer 130 may include multiple hidden layers (such as Figure 3 131, 132 to 13n) and the output layer 140 shown, the parameters contained in the multi-layer hidden layer can be pre-trained according to relevant training data of a specific task type, for example, the task type may include speech or semantic recognition, classification or generation, etc.

[0105] After the multiple hidden layers in the neural network layer 130, that is, the last layer of the entire convolutional neural network 100 is the output layer 140, which has a loss function similar to the classification cross entropy, specifically for calculating the prediction error. Once the forward propagation of the entire convolutional neural network 100 (such as Figure 3 The propagation from 110 to 140 is forward propagation), and the reverse propagation (such as Figure 3 The propagation from 140 to 110 is back propagation) and then starts to update the weight values ​​and biases of the aforementioned layers to reduce the loss of the convolutional neural network 100 and the error between the result output by the convolutional neural network 100 through the output layer and the ideal result.

[0106] It should be noted that if Figure 3 The convolutional neural network 100 shown is only an example of a convolutional neural network. In specific applications, the convolutional neural network can also exist in the form of other network models, such as Figure 4 The multiple convolutional layers / pooling layers shown are operated in parallel, and the features extracted from each layer are input to the neural network layer 130 for processing.

[0107] In this scheme, there are Figure 3 and Figure 4 The first neural network of the structure shown can be trained for tasks using the continuous learning method in this application. While retaining the knowledge learned from the old task, it learns the knowledge in the new task, so that the first neural network has good anti-forgetting ability and can be applied to different task reasoning, such as image recognition, target detection, speech semantic recognition, etc.

[0108] See Figure 5 , Figure 5 A schematic diagram of a chip hardware structure provided in an embodiment of the present application. Figure 5As shown, the Neural-Networks Processing Unit (NPU) 50 is mounted on the host CPU as a coprocessor, and the host CPU assigns tasks. The core of the NPU is the arithmetic circuit 503. The controller 504 controls the arithmetic circuit 503 to extract data from the memory (weight memory or input memory) and perform calculations.

[0109] In some implementations, the arithmetic circuit 503 includes multiple processing units (PEs). In some implementations, the arithmetic circuit 503 is a two-dimensional systolic array. The arithmetic circuit 503 can also be a one-dimensional systolic array or other electronic circuitry capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 503 is a general-purpose matrix processor.

[0110] For example, assume there are input matrix A, weight matrix B, and output matrix C. The arithmetic circuit retrieves the corresponding data of matrix B from weight memory 502 and caches it on each PE in the arithmetic circuit. The arithmetic circuit retrieves the data of matrix A from input memory 501 and performs a matrix operation on matrix B. The partial or final matrix result is stored in accumulator 508.

[0111] The vector calculation unit 507 can further process the output of the operation circuit, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc. For example, the vector calculation unit 507 can be used for network calculations of non-convolutional / non-FC layers in a neural network, such as pooling, batch normalization, local response normalization, etc.

[0112] In some implementations, the vector calculation unit 507 can store the processed output vector to the unified memory 506. For example, the vector calculation unit 507 can apply a nonlinear function to the output of the operation circuit 503, such as a vector of accumulated values, to generate an activation value. In some implementations, the vector calculation unit 507 generates a normalized value, a merged value, or both. In some implementations, the processed output vector can be used as an activation input to the operation circuit 503, for example, for use in a subsequent layer in a neural network.

[0113] The unified memory 506 is used to store input data and output data.

[0114] The memory unit access controller 505 (Direct Memory Access Controller, DMAC) moves the input data in the external memory to the input memory 501 and / or the unified memory 506, stores the weight data in the external memory into the weight memory 502, and stores the data in the unified memory 506 into the external memory.

[0115] The bus interface unit (BIU) 510 is used to implement interaction between the main CPU, DMAC and instruction fetch memory 509 through the bus.

[0116] An instruction fetch buffer 509 connected to the controller 504 is used to store instructions used by the controller 504 .

[0117] The controller 504 is used to call the instructions cached in the instruction fetch memory 509 to control the working process of the computing accelerator.

[0118] Generally, the unified memory 506, the input memory 501, the weight memory 502 and the instruction fetch memory 509 are all on-chip memories, and the external memory is a memory outside the NPU, which can be a double data rate synchronous dynamic random access memory (DDR SDRAM), a high bandwidth memory (HBM) or other readable and writable memory.

[0119] See Figure 6 , Figure 6 A flow chart of a tensor-based continuous learning method provided in an embodiment of the present application. Figure 6 As shown, the method includes step S610 and step S620.

[0120] Step S610: Acquire input data; wherein the input data includes one or more of video, image, text or voice.

[0121] Specifically, the input data obtained in the above steps includes but is not limited to video, image, text or voice, etc., which can be processed using neural networks, and this application does not list them all.

[0122] Step S620: Input the input data into the first neural network to obtain a data processing result. The first neural network is obtained by training m tasks. After the training of the i-th task is completed, the neural network contains A tensor cores, and the A tensor cores are divided into B tensor layers. Each tensor layer in the B tensor layers contains data of the same dimension for each tensor core in the A tensor cores. After the training of the i+1-th task is completed, the first neural network adds C tensor cores and / or D tensor layers, and in the training of the i+1-th task, the parameters of the C tensor cores and / or the D tensor layers are updated, where m is a positive integer and i is a positive integer less than or equal to m-1.

[0123] Wherein, the above A, B, C and D are positive integers.

[0124] Specifically, the input data input into the first neural network can be one or more of video, image, text, voice or other data that can be processed by a neural network.

[0125] Specifically, each tensor core may include multiple levels, each level including at least one dimension. Each tensor layer contains the data of each tensor core on the same dimension.

[0126] For example, if the tensor core is a 7*3*4 matrix, it contains three levels, the first level contains 7 dimensions, the second level contains 3 dimensions, and the third level contains 4 dimensions.

[0127] The following will be combined Figure 7 The example describes in detail the specific relationship between tensor cores and tensor layers. Figure 7 The figure can be used as a specific example of the relationship between the tensor core and the tensor layer after tensor decomposition.

[0128] exist Figure 7 In the example, after tensor decomposition of the tensor, three tensor cores are obtained: tensor core A, tensor core B, and tensor core C. Tensor core A can be divided into two dimensions of data at a certain level, namely A1 and A2. Tensor core B can be divided into two dimensions of data at the same level: B1 and B2. Tensor core C can be divided into two dimensions of data at the same level: C1 and C2. For the above three tensor cores, the data on the same dimension constitute a tensor layer. Figure 7 As shown, the three tensor cores can be divided into two tensor layers: tensor layer 1 and tensor layer 2. Tensor layer 1 contains data A1, B1 and C1, and tensor layer 2 contains data A2, B2 and C2.

[0129] For example, tensor cores A, B, and C are all 5*4*2 matrices, with 5 dimensions at the first level, 3 dimensions at the second level, and 2 dimensions at the third level. At this point, at the third level, these three tensor cores can be divided into two tensor layers, each containing the data for one dimension of each tensor core at the third level, i.e., a 5*4*1 matrix.

[0130] It should be understood that Figure 7 This application is only a specific example used to describe the relationship between tensor cores and tensor layers, and does not constitute a limitation on the number of tensor cores and the number of tensor layers.

[0131] For any two adjacent tasks among the m tasks (taking the i-th and i+1-th tasks as an example), during the training of the i+1-th task, tensor layers and / or tensor cores are added to the first neural network, and the parameters of the added tensor layers and / or tensor cores are updated to learn new knowledge from the training of the i+1-th task. That is, the newly added tensor cores / tensor layers are used to preserve the knowledge learned from the i+1-th task, ensuring the continuous learning ability of the first neural network.

[0132] Specifically, during the training process of the (i+1)th task, the changes to the tensor structure in the first neural network include three ways: (1) adding at least one tensor core; (2) adding at least one tensor layer; (3) adding at least one tensor core and at least one tensor layer.

[0133] Furthermore, the aforementioned adding C tensor cores and / or D tensor layers during the training process of the i+1th task specifically includes three methods: (1) adding C tensor cores during the training process of the i+1th task; (2) adding D tensor layers during the training process of the i+1th task; (3) adding C tensor cores and D tensor layers during the training process of the i+1th task, where C and D are positive integers.

[0134] For example, the three ways of changing the tensor structure in the first neural network can be specifically referred to in Figure 8a-8c Three examples are shown.

[0135] Figure 8a Corresponding to the first method mentioned above. After the training of the i-th task is completed, the first neural network contains three tensor cores and two tensor layers. During the training of the i+1-th task, a tensor layer (A3, B3+C3) is added, and the parameters in the added tensor layer are updated, that is, Figure 8a Shown in the shaded area.

[0136] Figure 8bCorresponding to the second method mentioned above. After the training of the i-th task is completed, the first neural network contains three tensor cores and two tensor layers. During the training of the i+1-th task, a tensor core (D1+D2) is added, and the parameters of the added tensor core are updated, that is, Figure 8b Shown in the shaded area.

[0137] Figure 8c Corresponding to the third method mentioned above. After the training of the i-th task is completed, the first neural network contains three tensor cores and two tensor layers. During the training of the i+1-th task, a tensor core (D1+D2+D3) and a tensor layer (A3, B3, C3+D3) are added, and the parameters of the added tensor core and tensor layer are updated, that is, Figure 8c Shown in the shaded area.

[0138] Optionally, when training the i+1th task by adding tensor cores (i.e., the second method above), the loss functions in the i-th task and i+1-th task training processes are as follows:

[0139] The loss function of the i-th task is:

[0140]

[0141] Among them, L1 is the value of the loss function; Y is the label; X is the input data; f is the output of the first neural network; Rank is the rank; A, B and C are tensor cores; ⊙ represents the tensor inner product.

[0142] The loss function of the i+1th task is:

[0143]

[0144] Among them, L2 is the value of the loss function; Y is the label; X is the input data; f is the output of the first neural network; Rank is the rank; A, B, C and F are tensor cores; ⊙ represents the tensor inner product.

[0145] The weight parameters of the tensor network are:

[0146] W=A⊙B⊙C...⊙F

[0147] The continuous learning method in this application includes two implementation methods:

[0148] (1) Add new tensor cores and / or tensor layers during the training of the new task and update the parameters of the added tensor cores and / or tensor layers

[0149] Specifically, the three methods in the aforementioned embodiments can be used to add new tensor cores and / or tensor layers during the training process of the i+1th task, and update the parameters in the added tensor cores and / or tensor layers, which will not be repeated here.

[0150] Furthermore, in this way, during the new task learning process, for non-newly added tensor cores / tensor layers, the parameters of all or some tensor cores remain unchanged, or the parameters of all or some tensor layers remain unchanged.

[0151] That is, after the training of the i+1th task is completed, the parameters of some or all of the tensor cores in the A tensor cores remain unchanged, or the parameters of some or all of the tensor layers in the B tensor layers remain unchanged.

[0152] Specifically, if the parameters in some tensor cores / tensor layers remain unchanged, they can be updated in two ways: (1) using tensor cores as the update object to update the parameters in some tensor cores; (2) using tensor layers as the update object to update the parameters in some tensor layers.

[0153] (2) Do not add new tensor cores and / or tensor layers during the training of a new task, but update the parameters of some existing tensor cores / tensor cores.

[0154] Specifically, during the training process of a new task, the tensor layer or tensor core can be used as the update object, and the parameters in some tensor cores or some tensor layers can be updated.

[0155] For example, Figure 9a-9b As shown in Figure 2, two update methods are described, using tensor cores and tensor layers as update objects respectively.

[0156] Figure 9a Describes how to use tensor layers as update objects. Figure 9a As shown in the figure, after the training of the i-th task is completed, the first neural network contains three tensor cores and two tensor layers. During the training of the i+1-th task, the tensor layer 2 (i.e. Figure 9a The parameters in the tensor layer 1 remain unchanged, and the tensor layer 2 is used to save the knowledge learned from the i+1th task.

[0157] Figure 9b Describes how to update tensor cores. Figure 9b As shown in the figure, after the training of the i-th task is completed, the first neural network contains three tensor cores and two tensor layers. During the training of the i+1-th task, the tensor core A (i.e. Figure 9bIn the middle shaded area, the updated tensor core A includes the parameters in A3+A4), the parameters in tensor cores B and C remain unchanged, and tensor core A is used to save the knowledge learned from the i+1th task.

[0158] Optionally, the first neural network targeted by the continuous learning method in this application can be designed in two ways:

[0159] (1) The first neural network designed contains tensors that can be decomposed into tensors

[0160] Specifically, before training each task, the tensors in the first neural network are decomposed into multiple tensor cores / tensor layers. Training is then performed using the training data to update the parameters in some or all of the tensor cores, or in some or all of the tensor layers. After training for each task is completed, the tensor cores are combined. The specific tensor decomposition and combination processes are not expanded in this application.

[0161] Further, based on the aforementioned embodiment, after the training of the i-th task is completed, the A tensor cores are tensor-combined to obtain one or more tensors; and before executing the training of the i+1-th task, the one or more tensors are tensor-decomposed to obtain the A tensor cores.

[0162] Optionally, the above-mentioned tensor decomposition method includes: CP decomposition, Tucker decomposition, etc., which is not limited in this application.

[0163] (2) The first neural network designed contains at least one tensor core, but does not contain tensors that can be decomposed.

[0164] Specifically, when designing the first neural network, tensors are replaced by tensor core / tensor layer structures. Therefore, during the training process of the task, there is no need to perform tensor decomposition and tensor combination operations.

[0165] For example, see Figure 10 The structure of a first neural network is shown. In the first neural network represented by this structure, during the training process of each task, since the network does not contain tensors that can be decomposed, there is no need to perform tensor decomposition and tensor combination operations.

[0166] like Figure 10As shown in the figure, the structure of the first neural network includes two deep neural networks (DNNs) and a tensor network. The tensor network contains n tensor layers. The DNN at the input end is used to transform the data dimension to meet the data processing requirements of the tensor network; the DNN at the output end is used to aggregate the data processed by each tensor core.

[0167] Optionally, the deep neural network at the output end can also be replaced by other network structures, such as GateNet, etc., which is not limited in this application.

[0168] Optionally, an alternating update approach can be used to train for new tasks. During alternating training, the first neural network serves as the primary model, i.e., the model used for inference. During each training session for a new task, a backup tensor network is trained using the training data for the new task, and the parameters of the trained backup tensor network are stored.

[0169] Furthermore, the training process of the i+1th task includes: using the i+1th batch of data sets to train the i-th backup tensor network to obtain the trained i-th backup tensor network; and using the i+1th batch of data sets to train the first neural network; wherein the loss function of the first neural network includes the degree of difference between the output of the first neural network and the output of the trained j-th backup tensor network, or the loss function of the first neural network includes the degree of difference between the model parameters of the first neural network and the model parameters of the trained j-th backup tensor network, j=1 to i-1, j is a positive integer less than or equal to i-1.

[0170] Specifically, see Figure 11 During the training of the i+1th task, the i-th backup tensor network is first trained using the training data of the i+1th task. At this point, the i-1 backup tensor networks have already been trained after learning the first i tasks. When the main model (i.e., the first neural network) is trained using the training data of the i+1th task, the trained i-1 backup tensor networks are used to constrain the update of the parameters in the main model.

[0171] Furthermore, two loss functions can be used to train the main model: (1) the loss function contains the difference between the output of the main model and the output of the trained j-th backup tensor network, j = 1 to i-1; (2) the loss function contains the difference between the model parameters in the main model and the model parameters of the trained j-th backup tensor network.

[0172] Optionally, during the training of the i+1th task, the specific form of the first loss function can be as follows:

[0173]

[0174] Where L is the value of the loss function; θ is the main model; l is the difference between the main model output and the label; f(θ) is the main model output; rank is the rank; A, B, C and F are tensor cores; f1 and f i-1 is the output of the 1st backup tensor network and the i-1th backup tensor network; D is the training data of the i+1th task.

[0175] Optionally, during the training of the i+1th task, the specific form of the second loss function can be as follows:

[0176]

[0177] The physical meaning of each parameter can be found in the above description and will not be repeated here.

[0178] Optionally, the above continuous learning method can be applied to a federated learning architecture.

[0179] Specifically, the first neural network is one of a plurality of neural networks, the plurality of neural networks are located on different user devices, and the first neural network is located on the first user device. That is, the training of the first neural network is a process on the user device.

[0180] Furthermore, after the training of each task is completed, the first user device sends the model parameters of the first neural network to the server, so that the server updates the second neural network on the server according to the model parameters of each neural network in the multiple neural networks; wherein the model parameters of the first neural network include the tensor cores contained in the first neural network.

[0181] Specifically, if Figure 12 As shown, in the federated learning architecture, there are a server and s user devices. The continuous learning process on each user device can be the same as that described in the above embodiment and will not be repeated here. After each user device completes the training of the new task, the updated model parameters (i.e., model parameter 1, model parameter 2, ..., model parameter s) are sent to the server respectively, and the server uses all the received model parameters to update the neural network on the server (i.e., the second neural network in the above embodiment).

[0182] Optionally, the loss function used when aggregating the model on the server is as follows:

[0183]

[0184] Wherein, L is the total loss; i is the model parameter on the i-th user device; λ is the constraint strength parameter; other variables can be found in the description of the above formula and are not repeated here.

[0185] Optionally, the tasks include image recognition, target detection, image segmentation or speech semantic recognition.

[0186] Specifically, the continuous learning method in this application is used to learn tasks including but not limited to image recognition, target detection, image segmentation or speech semantic recognition.

[0187] See Figure 13 , Figure 13 A flow chart of another continuous learning method provided in an embodiment of the present application. The method includes steps S1310 and S1320.

[0188] Step S1310: Acquire input data; wherein the input data includes one or more of video, image, text or voice.

[0189] Step S1320: Input the input data into the first neural network to obtain a data processing result.

[0190] In which, the first neural network is obtained through training of m tasks. After the training of the i-th task is completed, the neural network contains A tensor cores, the A tensor cores contain B tensor layers, and each tensor layer in the B tensor layers contains data of the same dimension of each tensor core in the A tensor cores; after the training of the i+1-th task is completed, the parameters of some or all of the tensor cores in the A tensor cores remain unchanged, or the parameters of some tensor layers in the B tensor layers remain unchanged, m is a positive integer, and i is a positive integer less than or equal to m-1.

[0191] Optionally, after the training of the i+1th task is completed, the first neural network adds C tensor cores and / or D tensor layers, and during the training of the i+1th task, the parameters in the C tensor cores and / or the D tensor layers are updated.

[0192] Specifically, Figure 13 The specific implementation process of the embodiment shown can be referred to the aforementioned Figure 6 The description in the embodiments will not be repeated here.

[0193] See Figure 14 , Figure 14 A flow chart of another continuous learning method provided in an embodiment of the present application. The method includes steps S1410 and S1420.

[0194] Step S1410: Receive multiple model parameters respectively sent by multiple user devices, wherein the multiple user devices include a first user device, and the model parameters sent by the first user device include tensor cores included in the first neural network.

[0195] Step S1420: updating the second neural network based on the multiple model parameters, and processing the data to be processed based on the updated second neural network to obtain a data processing result; wherein the data to be processed includes one or more of pictures, videos, voice or text.

[0196] In which, the first neural network is obtained through training of m tasks. After the training of the i-th task is completed, the neural network contains A tensor cores, and the A tensor cores are divided into B tensor layers. Each tensor layer in the B tensor layers contains data of each tensor core in the A tensor cores on the same dimension; after the training of the i+1-th task is completed, the first neural network adds C tensor cores and / or D tensor layers, and in the training of the i+1-th task, the parameters of the C tensor cores and / or the D tensor layers are updated, m is a positive integer, and i is a positive integer less than or equal to m-1.

[0197] Optionally, after the training of the i+1th task is completed, the parameters of some or all of the A tensor cores remain unchanged, or the parameters of some or all of the B tensor layers remain unchanged.

[0198] Specifically, Figure 14 The specific implementation process of the embodiment shown can be referred to the aforementioned Figure 6 The description in the embodiments will not be repeated here.

[0199] See Figure 15 , Figure 15 A flow chart of another continuous learning method provided in an embodiment of the present application. The method includes steps S1510 and S1520.

[0200] Step S1510: Receive multiple model parameters respectively sent by multiple user devices, wherein the multiple user devices include a first user device, and the model parameters sent by the first user device include tensor cores included in the first neural network.

[0201] Step S1520: updating the second neural network based on the multiple model parameters, and processing the data to be processed based on the updated second neural network to obtain a data processing result; wherein the data to be processed includes one or more of pictures, videos, voice or text.

[0202] In which, the first neural network is obtained by training m tasks. After the training of the i-th task is completed, the neural network contains A tensor cores, the A tensor cores are divided into B tensor layers, and each tensor layer in the B tensor layers contains data of the same dimension for each tensor core in the A tensor cores; after the training of the i+1-th task is completed, the parameters of some or all of the tensor cores in the A tensor cores remain unchanged, or the parameters of some tensor layers in the B tensor layers remain unchanged, m is a positive integer, and i is a positive integer less than or equal to m-1.

[0203] Specifically, Figure 15 The specific implementation process of the embodiment shown can be referred to the aforementioned Figure 6 The description in the embodiments will not be repeated here.

[0204] See Figure 16 , Figure 16 This is a schematic diagram of the structure of a continuous learning device provided in an embodiment of the present application. Figure 16 As shown, the device includes an acquisition unit 1610 and a processing unit 1620.

[0205] The continuous learning device can be used to perform the above Figure 6 The method in the embodiment is as follows:

[0206] The acquisition unit 1610 is used to acquire input data; wherein the input data includes one or more of video, image, text or voice; the processing unit 1620 is used to process the input data based on the first neural network to obtain a data processing result; wherein the first neural network is obtained by training m tasks, and after the training of the i-th task is completed, the neural network contains A tensor cores, and the A tensor cores are divided into B tensor layers, and each tensor layer in the B tensor layers contains data of each tensor core in the A tensor cores on the same dimension; after the training of the i+1-th task is completed, the first neural network adds C tensor cores and / or D tensor layers, and in the training of the i+1-th task, the parameters of the C tensor cores and / or the D tensor layers are updated, m is a positive integer, and i is a positive integer less than or equal to m-1.

[0207] In a feasible implementation, after the training of the i+1th task is completed, the parameters of some or all of the tensor cores in the A tensor cores remain unchanged, or the parameters of some or all of the tensor layers in the B tensor layers remain unchanged.

[0208] In a feasible embodiment, the processing unit is also used to: after the training of the i-th task is completed, perform tensor combination on the A tensor cores to obtain one or more tensors; and before executing the training of the i+1-th task, perform tensor decomposition on the one or more tensors to obtain the A tensor cores.

[0209] In a feasible embodiment, during the training process of the i+1th task, the processing unit is specifically used to: use the i+1th batch of data sets to train the i-th backup tensor network to obtain the trained i-th backup tensor network; and use the i+1th batch of data sets to train the first neural network; wherein the loss function of the first neural network includes the degree of difference between the output of the first neural network and the output of the trained j-th backup tensor network, or the loss function of the first neural network includes the degree of difference between the model parameters of the first neural network and the model parameters of the trained j-th backup tensor network, j=1 to i-1, j is a positive integer less than or equal to i-1.

[0210] In a feasible embodiment, the first neural network is one of a plurality of neural networks, and the plurality of neural networks are respectively located on different user devices, and the first neural network is located on the first user device. The apparatus further includes: a sending unit, configured to, after the training of each task is completed, cause the first user device to send the model parameters of the first neural network to the server, so that the server updates the second neural network on the server according to the model parameters of each neural network in the plurality of neural networks; wherein the model parameters of the first neural network include the tensor cores contained in the first neural network.

[0211] In a feasible implementation, the task includes image recognition, target detection, image segmentation or speech semantic recognition.

[0212] The continuous learning device can also be used to perform the above Figure 13 The method in the embodiment is as follows:

[0213] The acquisition unit 1610 is used to acquire input data; wherein the input data includes one or more of video, image, text or voice; the processing unit 1620 is used to input the input data into the first neural network to obtain a data processing result; wherein, the first neural network is obtained by training m tasks, and after the training of the i-th task is completed, the neural network contains A tensor cores, the A tensor cores contain B tensor layers, and each tensor layer in the B tensor layers contains data of the same dimension for each tensor core in the A tensor cores; after the training of the i+1-th task is completed, the parameters of some or all of the tensor cores in the A tensor cores remain unchanged, or the parameters of some tensor layers in the B tensor layers remain unchanged, m is a positive integer, and i is a positive integer less than or equal to m-1.

[0214] In a feasible embodiment, after the training of the i+1th task is completed, the first neural network adds C tensor cores and / or D tensor layers, and during the training of the i+1th task, the parameters in the C tensor cores and / or the D tensor layers are updated.

[0215] Specifically, Figure 16 The specific operation process of the device shown can be found in the corresponding method embodiment mentioned above, which will not be repeated here.

[0216] See Figure 17 , Figure 17 This is a schematic diagram of the structure of a continuous learning device provided in an embodiment of the present application. Figure 16 As shown, the device includes a receiving unit 1710 and a processing unit 1720.

[0217] The continuous learning device can be used to perform the above Figure 14 The method in the embodiment is as follows:

[0218] The receiving unit 1710 is configured to receive multiple model parameters respectively sent by multiple user devices, wherein the multiple user devices include a first user device, and the model parameters sent by the first user device include tensor cores included in a first neural network; the processing unit 1720 is configured to update a second neural network based on the multiple model parameters, and process the data to be processed based on the updated second neural network to obtain a data processing result; wherein the data to be processed includes one or more of images, videos, voice, or text; wherein the first neural network is obtained by training m tasks, and after the training of the i-th task is completed, the neural network includes A tensor cores, the A tensor cores are divided into B tensor layers, and each tensor layer in the B tensor layers contains data of each tensor core in the A tensor cores on the same dimension; after the training of the i+1-th task is completed, the first neural network adds C tensor cores and / or D tensor layers, and in the training of the i+1-th task, the parameters in the C tensor cores and / or the D tensor layers are updated, where m is a positive integer and i is a positive integer less than or equal to m-1.

[0219] In a feasible implementation, after the training of the i+1th task is completed, the parameters of some or all of the tensor cores in the A tensor cores remain unchanged, or the parameters of some or all of the tensor layers in the B tensor layers remain unchanged.

[0220] The continuous learning device can also be used to perform the above Figure 15 The method in the embodiment is as follows:

[0221] The receiving unit 1710 is configured to receive multiple model parameters respectively sent by multiple user devices, wherein the multiple user devices include a first user device, and the model parameters sent by the first user device include tensor cores included in a first neural network; the processing unit 1720 is configured to update a second neural network based on the multiple model parameters, and process the data to be processed based on the updated second neural network to obtain a data processing result; wherein the data to be processed includes one or more of images, videos, voice, or text; wherein the first neural network is obtained by training m tasks, and after the training of the i-th task is completed, the neural network includes A tensor cores, the A tensor cores are divided into B tensor layers, and each tensor layer in the B tensor layers contains data of the same dimension for each tensor core in the A tensor cores; after the training of the i+1-th task is completed, the parameters of some or all of the tensor cores in the A tensor cores remain unchanged, or the parameters of some tensor layers in the B tensor layers remain unchanged, where m is a positive integer and i is a positive integer less than or equal to m-1.

[0222] In a feasible embodiment, after the training of the i+1th task is completed, the first neural network adds C tensor cores and / or D tensor layers, and during the training of the i+1th task, the parameters in the C tensor cores and / or the D tensor layers are updated.

[0223] Specifically, Figure 17 The specific operation process of the device shown can be found in the corresponding method embodiment mentioned above, which will not be repeated here.

[0224] See Figure 18 , Figure 18 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 18 As shown, the device includes a processor 1801 , a memory 1802 , an interface circuit 1803 and a bus 1804 .

[0225] When the electronic device is used as a user device:

[0226] Interface circuit 1803 is configured to obtain input data. Processor 1801 is configured to process the input data based on the first neural network to obtain a data processing result. Memory 1802 is configured to store the data processing result. Processor 1801, memory 1802, and interface circuit 1803 are interconnected via bus 1804.

[0227] When the electronic device acts as a server:

[0228] Interface circuit 1803 is configured to receive multiple model parameters transmitted by multiple user devices. Processor 1801 is configured to update a second neural network based on the multiple model parameters and process the data to be processed based on the updated second neural network to obtain a data processing result. Memory 1802 is configured to store the data processing result. Processor 1801, memory 1802, and interface circuit 1803 are interconnected via bus 1804.

[0229] It should be understood that the specific operation process of the processor and memory on the electronic device in the embodiment of the present application can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0230] An embodiment of the present application provides a chip system, which includes at least one processor, a memory and an interface circuit. The memory, the interface circuit and the at least one processor are interconnected by lines, and instructions are stored in the at least one memory. When the instructions are executed by the processor, some or all of the steps of any one of the above method embodiments are implemented.

[0231] An embodiment of the present application provides a computer storage medium storing a computer program. When the computer program is executed, part or all of the steps of any one of the above method embodiments are implemented.

[0232] An embodiment of the present application provides a computer program, which includes instructions. When the computer program is executed by a processor, some or all of the steps of any one of the above method embodiments are implemented.

[0233] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. It should be noted that for the aforementioned method embodiments, in order to simplify the description, they are all expressed as a series of action combinations, but those skilled in the art should know that this application is not limited to the order of the actions described, because according to this application, some steps may be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for this application.

[0234] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the above-mentioned units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0235] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0236] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A tensor-based continuous learning method, characterized in that: The method comprises: Acquiring input data; wherein the input data includes one or more of video, image, text or voice; Inputting the input data into a first neural network to obtain a data processing result; In which, the first neural network is obtained through training of m tasks. After the training of the i-th task is completed, the neural network contains A tensor cores, and the A tensor cores are divided into B tensor layers. Each tensor layer in the B tensor layers contains data of each tensor core in the A tensor cores on the same dimension; after the training of the i+1-th task is completed, the first neural network adds C tensor cores and / or D tensor layers, and in the training of the i+1-th task, the parameters of the C tensor cores and / or the D tensor layers are updated, m is a positive integer, and i is a positive integer less than or equal to m-1.

2. The method according to claim 1, characterized in that After the training of the (i+1)th task is completed, parameters of some or all of the tensor cores in the A tensor cores remain unchanged, or parameters of some or all of the tensor layers in the B tensor layers remain unchanged.

3. The method according to claim 1 or 2, characterized in that The method further comprises: After the training of the i-th task is completed, performing tensor combination on the A tensor cores to obtain one or more tensors; and Before executing the training of the (i+1)th task, the one or more tensors are subjected to tensor decomposition to obtain the A tensor cores.

4. The method according to claim 1, wherein The training process of the i+1th task includes: Using the i+1th batch of data sets to train the i-th backup tensor network to obtain a trained i-th backup tensor network; and The first neural network is trained using the (i+1)th batch of data sets; wherein the loss function of the first neural network includes the degree of difference between the output of the first neural network and the output of the trained j-th backup tensor network, or the loss function of the first neural network includes the degree of difference between the model parameters of the first neural network and the model parameters of the trained j-th backup tensor network, j=1 to i-1, and j is a positive integer less than or equal to i-1.

5. The method according to claim 1, wherein The first neural network is one of a plurality of neural networks, the plurality of neural networks are respectively located on different user devices, the first neural network is located on a first user device, and the method further includes: After the training of each task is completed, the first user device sends the model parameters of the first neural network to the server, so that the server updates the second neural network on the server according to the model parameters of each neural network in the multiple neural networks; wherein the model parameters of the first neural network include the tensor cores contained in the first neural network.

6. The method according to claim 1, wherein The tasks include image recognition, object detection, image segmentation or speech semantic recognition.

7. A tensor-based continuous learning method, characterized in that The method comprises: Acquiring input data; wherein the input data includes one or more of video, image, text or voice; Inputting the input data into a first neural network to obtain a data processing result; In which, the first neural network is obtained by training m tasks. After the training of the i-th task is completed, the neural network contains A tensor cores, the A tensor cores contain B tensor layers, and each tensor layer in the B tensor layers contains data of the same dimension of each tensor core in the A tensor cores; after the training of the i+1-th task is completed, the parameters of some or all of the tensor cores in the A tensor cores remain unchanged, or the parameters of some tensor layers in the B tensor layers remain unchanged, m is a positive integer, and i is a positive integer less than or equal to m-1.

8. The method according to claim 7, characterized in that After the training of the i+1th task is completed, the first neural network adds C tensor cores and / or D tensor layers, and during the training of the i+1th task, the parameters in the C tensor cores and / or the D tensor layers are updated.

9. A tensor-based continuous learning method, characterized in that The method comprises: Receiving a plurality of model parameters respectively sent by a plurality of user devices, wherein the plurality of user devices include a first user device, and the model parameters sent by the first user device include tensor cores included in a first neural network; Updating the second neural network based on the multiple model parameters, and processing the data to be processed based on the updated second neural network to obtain a data processing result; wherein the data to be processed includes one or more of pictures, videos, voice or text; In which, the first neural network is obtained through training of m tasks. After the training of the i-th task is completed, the neural network contains A tensor cores, and the A tensor cores are divided into B tensor layers. Each tensor layer in the B tensor layers contains data of each tensor core in the A tensor cores on the same dimension; after the training of the i+1-th task is completed, the first neural network adds C tensor cores and / or D tensor layers, and in the training of the i+1-th task, the parameters of the C tensor cores and / or the D tensor layers are updated, m is a positive integer, and i is a positive integer less than or equal to m-1.

10. The method according to claim 9, characterized in that After the training of the (i+1)th task is completed, parameters of some or all of the tensor cores in the A tensor cores remain unchanged, or parameters of some or all of the tensor layers in the B tensor layers remain unchanged.

11. A tensor-based continuous learning method, characterized in that The method comprises: Receiving a plurality of model parameters respectively sent by a plurality of user devices, wherein the plurality of user devices include a first user device, and the model parameters sent by the first user device include tensor cores included in a first neural network; Updating the second neural network based on the multiple model parameters, and processing the data to be processed based on the updated second neural network to obtain a data processing result; wherein the data to be processed includes one or more of pictures, videos, voice or text; In which, the first neural network is obtained by training m tasks. After the training of the i-th task is completed, the neural network contains A tensor cores, the A tensor cores are divided into B tensor layers, and each tensor layer in the B tensor layers contains data of the same dimension for each tensor core in the A tensor cores; after the training of the i+1-th task is completed, the parameters of some or all of the tensor cores in the A tensor cores remain unchanged, or the parameters of some tensor layers in the B tensor layers remain unchanged, m is a positive integer, and i is a positive integer less than or equal to m-1.

12. The method according to claim 11, characterized in that After the training of the i+1th task is completed, the first neural network adds C tensor cores and / or D tensor layers, and during the training of the i+1th task, the parameters in the C tensor cores and / or the D tensor layers are updated.

13. A tensor-based continuous learning device, characterized in that: The device comprises: An acquisition unit, configured to acquire input data; wherein the input data includes one or more of video, image, text, or voice; a processing unit, configured to process the input data based on the first neural network to obtain a data processing result; In which, the first neural network is obtained through training of m tasks. After the training of the i-th task is completed, the neural network contains A tensor cores, and the A tensor cores are divided into B tensor layers. Each tensor layer in the B tensor layers contains data of each tensor core in the A tensor cores on the same dimension; after the training of the i+1-th task is completed, the first neural network adds C tensor cores and / or D tensor layers, and in the training of the i+1-th task, the parameters of the C tensor cores and / or the D tensor layers are updated, m is a positive integer, and i is a positive integer less than or equal to m-1.

14. The device according to claim 13, characterized in that After the training of the (i+1)th task is completed, parameters of some or all of the tensor cores in the A tensor cores remain unchanged, or parameters of some or all of the tensor layers in the B tensor layers remain unchanged.

15. The device according to claim 13 or 14, characterized in that The processing unit is further configured to: After the training of the i-th task is completed, the A tensor cores are combined to obtain one or more tensors; as well as Before executing the training of the (i+1)th task, the one or more tensors are subjected to tensor decomposition to obtain the A tensor cores.

16. The device according to claim 13, characterized in that During the training process of the i+1th task, the processing unit is specifically configured to: Using the i+1th batch of data sets to train the i-th backup tensor network to obtain a trained i-th backup tensor network; and The first neural network is trained using the (i+1)th batch of data sets; wherein the loss function of the first neural network includes the degree of difference between the output of the first neural network and the output of the trained j-th backup tensor network, or the loss function of the first neural network includes the degree of difference between the model parameters of the first neural network and the model parameters of the trained j-th backup tensor network, j=1 to i-1, and j is a positive integer less than or equal to i-1.

17. The device according to claim 13, characterized in that The first neural network is one of a plurality of neural networks, the plurality of neural networks are respectively located on different user devices, the first neural network is located on a first user device, and the apparatus further includes: a sending unit, configured to send, by the first user equipment, the model parameters of the first neural network to the server after the training of each task is completed, so that the server updates the second neural network on the server according to the model parameters of each neural network in the multiple neural networks; The model parameters of the first neural network include the tensor cores included in the first neural network.

18. The device according to claim 13, characterized in that The tasks include image recognition, object detection, image segmentation or speech semantic recognition.

19. A tensor-based continuous learning device, characterized in that The device comprises: An acquisition unit, configured to acquire input data; wherein the input data includes one or more of video, image, text, or voice; a processing unit, configured to input the input data into a first neural network to obtain a data processing result; In which, the first neural network is obtained by training m tasks. After the training of the i-th task is completed, the neural network contains A tensor cores, the A tensor cores contain B tensor layers, and each tensor layer in the B tensor layers contains data of the same dimension of each tensor core in the A tensor cores; after the training of the i+1-th task is completed, the parameters of some or all of the tensor cores in the A tensor cores remain unchanged, or the parameters of some tensor layers in the B tensor layers remain unchanged, m is a positive integer, and i is a positive integer less than or equal to m-1.

20. The device according to claim 19, characterized in that After the training of the i+1th task is completed, the first neural network adds C tensor cores and / or D tensor layers, and during the training of the i+1th task, the parameters in the C tensor cores and / or the D tensor layers are updated.

21. A continuous learning device based on tensors, characterized in that The device comprises: a receiving unit, configured to receive a plurality of model parameters respectively sent by a plurality of user devices, wherein the plurality of user devices include a first user device, and the model parameters sent by the first user device include tensor cores included in a first neural network; a processing unit, configured to update the second neural network based on the plurality of model parameters, and process the data to be processed based on the updated second neural network to obtain a data processing result; wherein the data to be processed includes one or more of images, videos, voice, or text; In which, the first neural network is obtained through training of m tasks. After the training of the i-th task is completed, the neural network contains A tensor cores, and the A tensor cores are divided into B tensor layers. Each tensor layer in the B tensor layers contains data of each tensor core in the A tensor cores on the same dimension; after the training of the i+1-th task is completed, the first neural network adds C tensor cores and / or D tensor layers, and in the training of the i+1-th task, the parameters of the C tensor cores and / or the D tensor layers are updated, m is a positive integer, and i is a positive integer less than or equal to m-1.

22. The device according to claim 21, characterized in that After the training of the (i+1)th task is completed, parameters of some or all of the tensor cores in the A tensor cores remain unchanged, or parameters of some or all of the tensor layers in the B tensor layers remain unchanged.

23. A tensor-based continuous learning device, characterized in that The device comprises: a receiving unit, configured to receive a plurality of model parameters respectively sent by a plurality of user devices, wherein the plurality of user devices include a first user device, and the model parameters sent by the first user device include tensor cores included in a first neural network; a processing unit, configured to update the second neural network based on the plurality of model parameters, and process the data to be processed based on the updated second neural network to obtain a data processing result; wherein the data to be processed includes one or more of images, videos, voice, or text; In which, the first neural network is obtained by training m tasks. After the training of the i-th task is completed, the neural network contains A tensor cores, the A tensor cores are divided into B tensor layers, and each tensor layer in the B tensor layers contains data of the same dimension for each tensor core in the A tensor cores; after the training of the i+1-th task is completed, the parameters of some or all of the tensor cores in the A tensor cores remain unchanged, or the parameters of some tensor layers in the B tensor layers remain unchanged, m is a positive integer, and i is a positive integer less than or equal to m-1.

24. The device according to claim 23, characterized in that After the training of the i+1th task is completed, the first neural network adds C tensor cores and / or D tensor layers, and during the training of the i+1th task, the parameters in the C tensor cores and / or the D tensor layers are updated.

25. An electronic device, characterized in that: The electronic device includes at least one processor, a memory and an interface circuit, the memory, the interface circuit and the at least one processor are interconnected via lines, and instructions are stored in the at least one memory; when the instructions are executed by the processor, the method described in any one of claims 1 to 12 is implemented.

26. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed, the method according to any one of claims 1 to 12 is implemented.

Images