Data processing method and related equipment

Abstract

The invention relates to the field of artificial intelligence, and discloses a data processing method comprising the steps of acquiring to-be-processed data and a target neural network model, whereinthe target neural network model comprises a first transformer layer, the first transformer layer comprises a first residual error branch and a second residual error branch, the first residual error branch comprises a first attention head, and the second residual error branch comprises a target feedforward layer FFN; performing target task related processing on the to-be-processed data according toa target neural network model to obtain a data processing result, the target neural network model being used for performing target operation on the output of the first attention head and the first weight value to obtain the output of the first residual branch, and / or using the target neural network model for performing target operation on the output of the target FFN and the second weight value to obtain the output of the second residual branch. According to the embodiment of the invention, for different tasks, the weight value for controlling the output of the residual branch is set, so thatthe computing resource requirement of the terminal equipment for operating the target neural network model is reduced.

Description

technical field [0001] The present application relates to the field of artificial intelligence, and in particular, to a data processing method and related equipment. Background technique [0002] Artificial intelligence (AI) is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a branch of computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that responds in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that the machines have the functions of perception, reasoning and decision-making. [0003] With the continuous development of artificial intellig...

AI Technical Summary

Problems solved by technology

[0006] In the existing implementation, you can pre-train a skeleton network (such as BERT), share the parameters of the skeleton network for all tasks, and then use it as the underlying network, and add a task-specific neural network on the basis of it. Training, such methods usually need to add more parameters, which cannot meet the needs of low computing resources of terminal devices

Images