Model processing method and device and device for model processing
A model processing and model technology, applied in the computer field, can solve the problems that the effect of the business model cannot be improved, and the business model cannot be optimized by the pre-training model, so as to achieve the effect of improving business performance
Pending Publication Date: 2022-03-08
BEIJING SOGOU TECHNOLOGY DEVELOPMENT CO LTD
View PDF0 Cites 1 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0003] However, in some scenarios, the pre-trained model cannot be used to optimize the business model, so that the effect of the business model cannot be improved
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0027] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some of the embodiments of the present invention, but not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.
[0028] method embodiment
[0029] refer to figure 1 , shows a flow chart of the steps of an embodiment of a model processing method of the present invention, the method may specifically include the following steps:
[0030] Step 101, acquiring business data.
[0031] Step 102: Input the business data into the pre-training model and the initial business model respectively, process the business data through the self-attention mechanism, and obtain the ...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The embodiment of the invention provides a model processing method and device and a device for model processing. The method comprises the following steps: acquiring business data; respectively inputting the business data into a pre-training model and an initial business model, and processing the business data through a self-attention mechanism to obtain full-connection layer output of the pre-training model and full-connection layer output of the initial business model; matching the modeling unit of the pre-training model with the modeling unit of the initial business model, and determining a target character in the modeling unit of the pre-training model; and according to the full connection layer output of the initial business model and the full connection layer output corresponding to the target character, performing knowledge distillation on the pre-training model and the initial business model to obtain a target business model. According to the embodiment of the invention, information loss in the knowledge distillation process can be avoided, and the model performance of the business model is improved.
Description
technical field [0001] The invention relates to the field of computer technology, in particular to a model processing method, device and device for model processing. Background technique [0002] At present, the development of pre-training models is advancing by leaps and bounds. Pre-training can obtain task-independent pre-trained models from large-scale data through self-supervised learning. Moreover, the pre-trained model can transfer the knowledge learned from large-scale data to specific businesses. In other words, if the business model is optimized using the pre-trained model, the effect of the business model can be improved. [0003] However, in some scenarios, the business model cannot be optimized using the pre-trained model, so that the effect of the business model cannot be improved. Contents of the invention [0004] Embodiments of the present invention provide a model processing method, device and device for model processing, which can optimize an initial b...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Applications(China)
IPC IPC(8): G06F30/27G06K9/62
CPCG06F30/27G06F18/214
Inventor 凡子威
Owner BEIJING SOGOU TECHNOLOGY DEVELOPMENT CO LTD