Analysis model construction method and device, analysis model application method and device, equipment and storage medium

Abstract

The invention relates to the field of artificial intelligence and financial science and technology, and discloses an analysis model construction and application method and device, equipment and a storage medium, and the method comprises the steps: obtaining a trained BERT model; obtaining a distillation model corresponding to the trained Bert model; obtaining a first feature extraction result output by the trained Bert model based on the preset news text, and obtaining a second feature extraction result output by the distillation model based on the preset news text; obtaining absolute value loss between the first feature extraction result and the second feature extraction result, stopping training the distillation model when the absolute value loss meets a stopping condition, and storing the trained distillation model; and when a preset condition is met, constructing an analysis model through the trained distillation model and the second full connection layer. The method is beneficial to expanding the application range of the analysis model.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence and financial technology, and in particular to the construction, application methods, devices, equipment and storage media of analysis models. Background Art

[0002] In the current news text analysis business, the main method is to manually analyze the current news text and obtain the analysis results. Manual analysis is incapable of dealing with the massive amount of current news text, because manual analysis is not only time-consuming and labor-intensive, but also easily affected by subjective judgment and emotional factors, resulting in inconsistency and lag in analysis results.

[0003] Therefore, manual analysis increases the analysis time and is not conducive to improving analysis efficiency, while neural network models cannot directly replace manual analysis because existing neural network models require a large amount of computing resources and memory space to support the reasoning process. A large amount of computing resources and memory space far exceeds the carrying capacity of mobile devices. Therefore, how to build a lightweight analysis model is an urgent problem to be solved. Summary of the invention

[0004] The present invention provides an analysis model construction method, device, computer equipment and storage medium to solve the technical problem of how to construct a lightweight analysis model.

[0005] In a first aspect, a method for constructing an analysis model is provided, comprising:

[0006] Obtaining a preset news text and a preset analysis result corresponding to the preset news text;

[0007] Input the preset news text into the first fully connected layer connected to the Bert model, obtain a first prediction result output by the first fully connected layer, train the BERT model through the first prediction result and the preset analysis result, and obtain the trained BERT model;

[0008] Obtain a distillation model corresponding to the trained Bert model;

[0009] Obtain a first feature extraction result output by the trained Bert model based on the preset news text, and obtain a second feature extraction result output by the distillation model based on the preset news text;

[0010] Obtaining an absolute value loss between the first feature extraction result and the second feature extraction result, and when the absolute value loss satisfies a stop condition, stopping training the distillation model, and saving the trained distillation model;

[0011] When the preset conditions are met, an analysis model is constructed through the trained distillation model and the second fully connected layer.

[0012] Furthermore, the step of inputting the preset news text into a first fully connected layer connected to a Bert model, obtaining a first prediction result output by the first fully connected layer, training the BERT model through the first prediction result and the preset analysis result, and obtaining the trained BERT model includes:

[0013] Input the preset news text into the first fully connected layer connected to the Bert model, obtain a first prediction result output by the first fully connected layer, and obtain a first loss value between the first prediction result and the preset analysis result through a cross entropy loss function;

[0014] The parameters in the BERT model are trained by using a back propagation algorithm and the first loss value. When the first loss value is less than a first preset value, the training of the parameters in the BERT model is stopped to obtain the trained BERT model.

[0015] Furthermore, the step of obtaining a distillation model corresponding to the trained Bert model includes:

[0016] Obtain the encoding layer and the attention blocks of the first three layers of the trained Bert model;

[0017] A distillation model is constructed through the encoding layer and the attention block.

[0018] Furthermore, the step of obtaining a first feature extraction result output by the trained Bert model based on the preset news text and obtaining a second feature extraction result output by the distillation model based on the preset news text includes:

[0019] Inputting the preset news text into the trained Bert model, and inputting the preset news text into the distillation model;

[0020] The first feature extraction result output by the trained Bert model based on the current preset news text is obtained, and the second feature extraction result output by the distillation model based on the preset news text is obtained.

[0021] Further, the obtaining of the absolute value loss between the first feature extraction result and the second feature extraction result, and when the absolute value loss satisfies a stopping condition, stopping the training of the distillation model, and saving the trained distillation model, comprises:

[0022] Obtaining the absolute value loss between the first feature extraction result and the second feature extraction result by using a preset absolute value loss function, and training the distillation model with the goal of minimizing the absolute value loss;

[0023] When the reduction range of the absolute value loss is less than a preset range, the training of the distillation model is stopped, and the trained distillation model is saved.

[0024] Furthermore, when the preset conditions are met, an analysis model is constructed through the trained distillation model and the second fully connected layer, including:

[0025] Decomposing the preset news text into a plurality of semantic units through the trained distillation model, extracting features from the semantic units to obtain first embedding vectors corresponding to the semantic units, inputting the first embedding vectors corresponding to the semantic units into an average pooling layer, and performing average pooling processing on the first embedding vectors corresponding to the semantic units using the average pooling layer to generate second embedding vectors;

[0026] The second embedding vector is input into a hidden layer, and the second embedding vector and the third embedding vector corresponding to the noise data are concatenated using the hidden layer to generate a fourth embedding vector, and the fourth embedding vector is input into a second fully connected layer connected to the trained distillation model to obtain a second prediction result output by the second fully connected layer, and a second loss value between the second prediction result and a preset analysis result is obtained through a cross entropy loss function, and when the second loss value is less than a second preset value, an analysis model is constructed through the trained distillation model and the second fully connected layer.

[0027] In a second aspect, a method for applying an analysis model is provided, wherein the analysis model is obtained according to the above-mentioned construction method, and the method for applying the analysis model comprises:

[0028] The current news text is obtained, the current news text is input into the analysis model, and the current analysis result output by the analysis model based on the current news text is obtained.

[0029] In a third aspect, an analysis model building device is provided, comprising:

[0030] A first acquisition module, used to acquire a preset news text and a preset analysis result corresponding to the preset news text;

[0031] A second acquisition module is used to input the preset news text into a first fully connected layer connected to a Bert model, obtain a first prediction result output by the first fully connected layer, train the BERT model through the first prediction result and the preset analysis result, and obtain the trained BERT model;

[0032] A third acquisition module is used to obtain a distillation model corresponding to the trained Bert model;

[0033] A fourth acquisition module is used to obtain a first feature extraction result output by the trained Bert model based on the preset news text, and to obtain a second feature extraction result output by the distillation model based on the preset news text;

[0034] A saving module, used for obtaining an absolute value loss between the first feature extraction result and the second feature extraction result, and when the absolute value loss satisfies a stopping condition, stopping training the distillation model and saving the trained distillation model;

[0035] A construction module is used to construct an analysis model through the trained distillation model and the second fully connected layer when preset conditions are met.

[0036] In a fourth aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned analysis model construction method or the above-mentioned analysis model application method when executing the computer program.

[0037] In a fifth aspect, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps of the above-mentioned analysis model construction method or the above-mentioned analysis model application method are implemented.

[0038] The present application provides a method, apparatus, computer equipment and storage medium for constructing an analysis model, which obtains a preset news text and a preset analysis result corresponding to the preset news text; inputs the preset news text into a first fully connected layer connected to a Bert model, obtains a first prediction result output by the first fully connected layer, trains the BERT model through the first prediction result and the preset analysis result, and obtains the trained BERT model; obtains a distillation model corresponding to the trained Bert model; obtains a first feature extraction result output by the trained Bert model based on the preset news text, and obtains a second feature extraction result output by the distillation model based on the preset news text; obtains the first feature extraction result and the second feature extraction result output by the distillation model The absolute value loss between the feature extraction results. When the absolute value loss meets the stopping condition, stop training the distillation model and save the trained distillation model. When the preset conditions are met, the analysis model is constructed through the trained distillation model and the second fully connected layer. The beneficial effects are in two aspects. On the one hand, when the preset conditions are met, the analysis model is constructed through the trained distillation model and the second fully connected layer. Since no manual analysis is required, the analysis time is reduced, which is conducive to improving the analysis efficiency. On the other hand, the model structure of the analysis model is simplified through the trained distillation model, which solves the problem of how to build a lightweight analysis model, so that the analysis model can run in an environment with limited resources, which is conducive to expanding the application scope of the analysis model. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0040] Figure 1 is a schematic diagram of an application environment of a method for building an analysis model in one embodiment of the present invention;

[0041] Figure 2 A schematic diagram of a flow chart of a method for constructing an analysis model provided by an embodiment of the present invention;

[0042] Figure 3 yes Figure 1 A schematic flow chart of a specific implementation of step S23;

[0043] Figure 4 yes Figure 1 A schematic flow chart of a specific implementation of step S25;

[0044] Figure 5 yes Figure 1 A schematic flow chart of a specific implementation of step S26;

[0045] Figure 6 is a structural schematic diagram of an analysis model building device in one embodiment of the present invention;

[0046] Figure 7 is a schematic diagram of a structure of a computer device in one embodiment of the present invention;

[0047] Figure 8 It is another structural schematic diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] See also Figure 1 , Figure 1 is a schematic diagram of an application environment of an analysis model construction method according to an embodiment of the present invention. The analysis model construction method provided by the embodiment of the present invention can be applied in Figure 1 In an application environment, a client communicates with a server through a network.

[0050] The server obtains a preset news text and a preset analysis result corresponding to the preset news text through the client;

[0051] Input the preset news text into the first fully connected layer connected to the Bert model, obtain a first prediction result output by the first fully connected layer, train the BERT model through the first prediction result and the preset analysis result, and obtain the trained BERT model;

[0052] Obtain a distillation model corresponding to the trained Bert model;

[0053] Obtain a first feature extraction result output by the trained Bert model based on the preset news text, and obtain a second feature extraction result output by the distillation model based on the preset news text;

[0054] Obtaining an absolute value loss between the first feature extraction result and the second feature extraction result, and when the absolute value loss satisfies a stop condition, stopping training the distillation model, and saving the trained distillation model;

[0055] When the preset conditions are met, an analysis model is constructed through the trained distillation model and the second fully connected layer.

[0056] In the scheme implemented by the above-mentioned analysis model construction method, device, equipment and medium, the beneficial effects are in two aspects. On the one hand, when the preset conditions are met, the analysis model is constructed by the trained distillation model and the second fully connected layer. Since no manual analysis is required, the analysis time is reduced, which is beneficial to improving the analysis efficiency. On the other hand, the model structure of the analysis model is streamlined through the trained distillation model, which solves the problem of how to build a lightweight analysis model, so that the analysis model can run in an environment with limited resources, which is beneficial to expanding the application scope of the analysis model.

[0057] The client may include but is not limited to various personal computers, laptops, smart phones, tablet computers, and portable wearable devices.

[0058] The server can be implemented with an independent task database or a task database cluster composed of multiple task databases. The present invention is described in detail below through specific embodiments.

[0059] See also Figure 2 , Figure 2 A flowchart of a method for constructing an analysis model provided by an embodiment of the present invention includes the following steps:

[0060] S21, obtaining a preset news text and a preset analysis result corresponding to the preset news text;

[0061] The preset news text refers to the news text that has been analyzed.

[0062] For example, obtaining a preset news text and a preset analysis result corresponding to the preset news text includes:

[0063] A database is connected, and a preset news text and a preset analysis result corresponding to the preset news text are obtained through the database.

[0064] S22, inputting the preset news text into the first fully connected layer connected to the Bert model, obtaining a first prediction result output by the first fully connected layer, training the BERT model through the first prediction result and the preset analysis result, and obtaining the trained BERT model;

[0065] The BERT (Bidirectional Encoder Representations from Transformers) model is a natural language processing model based on deep learning.

[0066] Among them, the first fully connected layer receives the embedding vector output by the BERT model as input, and further transforms and combines the embedding vector output by the BERT model through the weight and bias parameters inside the first fully connected layer to generate a first prediction result for the preset news text.

[0067] The step of inputting the preset news text into the first fully connected layer connected to the Bert model, obtaining a first prediction result output by the first fully connected layer, training the BERT model through the first prediction result and the preset analysis result, and obtaining the trained BERT model includes:

[0068] Input the preset news text into the first fully connected layer connected to the Bert model, obtain a first prediction result output by the first fully connected layer, and obtain a first loss value between the first prediction result and the preset analysis result through a cross entropy loss function;

[0069] The parameters in the BERT model are trained by using a back propagation algorithm and the first loss value. When the first loss value is less than a first preset value, the training of the parameters in the BERT model is stopped to obtain the trained BERT model.

[0070] S23, obtaining a distillation model corresponding to the trained Bert model;

[0071] Among them, the distillation model can reduce the demand for computing resources and reasoning time.

[0072] S24, obtaining a first feature extraction result output by the trained Bert model based on the preset news text, and obtaining a second feature extraction result output by the distillation model based on the preset news text;

[0073] The step of obtaining a first feature extraction result output by the trained Bert model based on the preset news text and obtaining a second feature extraction result output by the distillation model based on the preset news text includes:

[0074] Inputting the preset news text into the trained Bert model, and inputting the preset news text into the distillation model;

[0075] The first feature extraction result output by the trained Bert model based on the current preset news text is obtained, and the second feature extraction result output by the distillation model based on the preset news text is obtained.

[0076] S25, obtaining the absolute value loss between the first feature extraction result and the second feature extraction result, and when the absolute value loss meets the stop condition, stopping the training of the distillation model and saving the trained distillation model;

[0077] Among them, Absolute Loss is a loss function in machine learning and statistics, which is used to measure the difference between the first feature extraction result and the second feature extraction result of the model.

[0078] The greater the absolute value loss, the greater the difference between the first feature extraction result and the second feature extraction result;

[0079] The smaller the absolute value loss is, the smaller the difference between the first feature extraction result and the second feature extraction result is.

[0080] S26, when the preset conditions are met, construct an analysis model through the trained distillation model and the second fully connected layer. The distillation model uses knowledge distillation technology to transfer the feature representation in the complex Bert model to the analysis model in an efficient and compressed form.

[0081] This process not only removes redundant and unnecessary information in the Bert model, but also retains the core predictive ability and generalization performance of the Bert model. Therefore, the analysis model is more streamlined in model structure, and the required computing resources and memory space will not exceed the carrying capacity of mobile devices, so the analysis model can be applied on mobile devices.

[0082] In the embodiment of the present invention, the beneficial effects lie in two aspects. On the one hand, when the preset conditions are met, the analysis model is constructed by the trained distillation model and the second fully connected layer. Since no manual analysis is required, the analysis time is reduced, which is beneficial to improving the analysis efficiency. On the other hand, the model structure of the analysis model is streamlined by the trained distillation model, so that the analysis model can run in an environment with limited resources, which is beneficial to expanding the application scope of the analysis model.

[0083] Exemplarily, an analysis model application method is provided, wherein the analysis model is obtained according to the above construction method. The analysis model application method can be applied to a mobile device, and the analysis model application method includes:

[0084] The current news text is obtained, the current news text is input into the analysis model, and the current analysis result output by the analysis model based on the current news text is obtained.

[0085] The current news text refers to the news text to be analyzed.

[0086] Among them, the data format of the preset news text and the current news text is text format.

[0087] The current news text includes one or a combination of financial news text, science and technology news text, and environmental news text.

[0088] Financial news texts are news about financial markets, industrial development, economic policies, and corporate dynamics. Financial news texts can deliver economic information to the public in a timely manner and influence people's economic decisions and consumption behaviors.

[0089] Science and technology news refers to news about technological innovation, new product releases, scientific research results, and science and technology policies. With the rapid development of science and technology, the influence of science and technology news on the public is increasing, becoming an important force in promoting social progress.

[0090] Environmental news texts are news about environmental protection, climate change, and the use of natural resources. As environmental issues become increasingly prominent, environmental news texts are receiving more and more attention and attention from the public. For the sake of explanation, the current news text is a financial news text, as an example, as follows:

[0091] For example, the current news text is: The chairman of Company A announced that he will leave the company in a certain month;

[0092] The current analysis result is: affecting the stock price of Company A.

[0093] For example, the current news text is: The CEO of Company B publicly announced that he would retire early;

[0094] The current analysis result is: affecting the stock price of Company B.

[0095] For example, the current news text is: The founder of Company C publicly announced that he would retire early;

[0096] The current analysis result is: impact on the stock price of Company C.

[0097] For example, the current news text is that the president of Company D offered to resign because the performance did not meet expectations;

[0098] The current analysis result is: impact on the stock price of Company D.

[0099] For the sake of explanation, the current news text is taken as a science and technology news text as an example, as follows:

[0100] For example, the current news text is: Company E released an electronic product equipped with the latest processor and camera technology;

[0101] The current analysis result is: affecting the stock price of Company E. For example, the current news text is: Company F has successfully developed image recognition technology based on deep learning algorithms, which can more accurately identify objects and scenes, and provide strong support for smart security, autonomous driving and other fields;

[0102] The current analysis result is: affecting the stock price of Company F.

[0103] For the sake of explanation, the current news text is taken as an environmental news text as an example, as follows:

[0104] For example, the current news text is: Company G has launched an intelligent energy management system based on solar and wind energy to help enterprises achieve energy conservation, emission reduction and sustainable development;

[0105] The current analysis result is: affecting the stock price of G Company.

[0106] For example, the current news text is: Company H announced the launch of new environmental protection projects, such as afforestation, wetland protection, and ecological restoration, aiming to improve the ecological environment and enhance biodiversity;

[0107] The current analysis result is: affecting the stock price of H Company.

[0108] Exemplarily, after obtaining the current news text, inputting the current news text into the analysis model, and obtaining the current analysis result output by the analysis model based on the current news text, the analysis model construction method includes:

[0109] A storage area corresponding to the analysis model is obtained, and the current news text and the current analysis result are stored in the storage area.

[0110] In an embodiment of the present invention, the current news text is input into the analysis model, and a current analysis result output by the analysis model based on the current news text is obtained. Since there is no need for manual analysis of the current news text, the analysis time of the current news text is reduced, which is conducive to improving the analysis efficiency of the current news text.

[0111] See also Figure 3 , Figure 3 yes Figure 1 A specific implementation flow diagram of step S23 is described in detail as follows:

[0112] S31, obtaining the encoding layer and the attention blocks of the first three layers of the trained Bert model;

[0113] S32, constructing a distillation model through the encoding layer and the attention block.

[0114] Among them, the Attention Block is a network structure in the field of deep learning, especially in natural language processing applications. The Attention Block can dynamically focus on the key parts of the input data, thereby improving the understanding ability and processing efficiency of the distillation model.

[0115] In the embodiment of the present invention, the distillation model significantly reduces the computational complexity and storage requirements, speeds up the reasoning speed, and enables the analysis model with the distillation model to be more flexibly deployed on resource-constrained platforms.

[0116] See also Figure 4 , Figure 4 yes Figure 1 A specific implementation flow diagram of step S25 is described in detail as follows:

[0117] S41, obtaining the absolute value loss between the first feature extraction result and the second feature extraction result through a preset absolute value loss function, and training the distillation model with the goal of minimizing the absolute value loss;

[0118] S42: When the reduction range of the absolute value loss is less than a preset range, stop training the distillation model and save the trained distillation model.

[0119] In an embodiment of the present invention, the trained distillation model is saved, and the trained distillation model is reloaded and used when needed, which is beneficial to the use efficiency of the trained distillation model.

[0120] See also Figure 5 , Figure 5 yes Figure 1 A specific implementation flow diagram of step S26 is described in detail as follows:

[0121] S51, decomposing the preset news text into multiple semantic units through the trained distillation model, performing feature extraction on the semantic units to obtain first embedding vectors corresponding to the semantic units, inputting the first embedding vectors corresponding to the respective semantic units into an average pooling layer, and performing average pooling processing on the first embedding vectors corresponding to the respective semantic units using the average pooling layer to generate second embedding vectors;

[0122] S52, input the second embedding vector into a hidden layer, use the hidden layer to concatenate the second embedding vector and the third embedding vector corresponding to the noise data to generate a fourth embedding vector, input the fourth embedding vector into the second fully connected layer connected to the trained distillation model, obtain the second prediction result output by the second fully connected layer, obtain the second loss value between the second prediction result and the preset analysis result through the cross entropy loss function, and when the second loss value is less than the second preset value, construct an analysis model through the trained distillation model and the second fully connected layer.

[0123] The second embedding vector is input into the hidden layer, and the hidden layer is used to concatenate the second embedding vector and the third embedding vector corresponding to the noise data to generate a fourth embedding vector. In this way, by introducing a certain degree of noise data into the training data, the second fully connected layer can learn a more generalized feature representation, which is conducive to enhancing the generalization performance.

[0124] The second fully connected layer receives the fourth embedded vector as input, and further transforms and combines the fourth embedded vector through weights and bias parameters inside the second fully connected layer to generate a second prediction result for the fourth embedded vector.

[0125] In the embodiment of the present invention, the analysis model is constructed, and since manual analysis is not required, the analysis time is reduced, which is conducive to improving the analysis efficiency.

[0126] See also Figure 6 , Figure 6 is a schematic diagram of a structure of an analysis model building device in one embodiment of the present invention, such as Figure 6 As shown, the analysis model building device includes a first acquisition module 101, a second acquisition module 102, a third acquisition module 103, a fourth acquisition module 104, a storage module 105, and a building module 106. The detailed description of each functional module is as follows:

[0127] The first acquisition module 101 is used to acquire a preset news text and a preset analysis result corresponding to the preset news text;

[0128] The second acquisition module 102 is used to input the preset news text into the first fully connected layer connected to the Bert model, obtain the first prediction result output by the first fully connected layer, train the BERT model through the first prediction result and the preset analysis result, and obtain the trained BERT model;

[0129] A third acquisition module 103 is used to acquire a distillation model corresponding to the trained Bert model;

[0130] A fourth acquisition module 104 is used to acquire a first feature extraction result output by the trained Bert model based on the preset news text, and to acquire a second feature extraction result output by the distillation model based on the preset news text;

[0131] A saving module 105 is used to obtain an absolute value loss between the first feature extraction result and the second feature extraction result, and when the absolute value loss meets a stop condition, stop training the distillation model and save the trained distillation model;

[0132] The construction module 106 is used to construct an analysis model through the trained distillation model and the second fully connected layer when a preset condition is met.

[0133] In one embodiment, the second acquisition module 102 includes:

[0134] A first acquisition subunit is used to input the preset news text into a first fully connected layer connected to the Bert model, obtain a first prediction result output by the first fully connected layer, and obtain a first loss value between the first prediction result and a preset analysis result through a cross entropy loss function;

[0135] The second acquisition subunit is used to train the parameters in the BERT model through the back propagation algorithm and the first loss value. When the first loss value is less than a first preset value, the training of the parameters in the BERT model is stopped to obtain the trained BERT model.

[0136] In one embodiment, the third acquisition module 103 includes:

[0137] A third acquisition subunit is used to obtain the encoding layer and the attention blocks of the first three layers of the trained Bert model;

[0138] The first construction subunit is used to construct a distillation model through the encoding layer and the attention block.

[0139] In one embodiment, the fourth acquisition module 104 includes:

[0140] An input subunit, used to input the preset news text into the trained Bert model, and input the preset news text into the distillation model;

[0141] The output subunit is used to obtain the first feature extraction result output by the trained Bert model based on the current preset news text, and to obtain the second feature extraction result output by the distillation model based on the preset news text.

[0142] In one embodiment, the storage module 105 includes:

[0143] A training subunit, configured to obtain an absolute value loss between the first feature extraction result and the second feature extraction result by using a preset absolute value loss function, and train the distillation model with the goal of minimizing the absolute value loss;

[0144] The saving subunit is used to stop training the distillation model and save the trained distillation model when the reduction range of the absolute value loss is less than a preset range.

[0145] In one embodiment, the building module 106 includes:

[0146] A generating subunit is used to decompose the preset news text into multiple semantic units through the trained distillation model, perform feature extraction on the semantic units, obtain first embedding vectors corresponding to the semantic units, input the first embedding vectors corresponding to each of the semantic units into an average pooling layer, and use the average pooling layer to perform average pooling processing on the first embedding vectors corresponding to each of the semantic units to generate a second embedding vector;

[0147] The second construction subunit is used to input the second embedding vector into a hidden layer, use the hidden layer to concatenate the second embedding vector and the third embedding vector corresponding to the noise data to generate a fourth embedding vector, input the fourth embedding vector into a second fully connected layer connected to the trained distillation model, obtain a second prediction result output by the second fully connected layer, obtain a second loss value between the second prediction result and a preset analysis result through a cross entropy loss function, and when the second loss value is less than a second preset value, construct an analysis model through the trained distillation model and the second fully connected layer.

[0148] In one embodiment, the analysis model building device further includes:

[0149] The fifth acquisition module is used to acquire the current news text, input the current news text into the analysis model, and acquire the current analysis result output by the analysis model based on the current news text.

[0150] In the embodiment of the present invention, the beneficial effects lie in two aspects. On the one hand, when the preset conditions are met, the analysis model is constructed by the trained distillation model and the second fully connected layer. Since no manual analysis is required, the analysis time is reduced, which is beneficial to improving the analysis efficiency. On the other hand, the model structure of the analysis model is streamlined through the trained distillation model, which solves the problem of how to construct a lightweight analysis model, so that the analysis model can run in an environment with limited resources, which is beneficial to expanding the application scope of the analysis model.

[0151] For the specific limitations on the analysis model building device, please refer to the limitations on the analysis model building method above, which will not be repeated here.

[0152] Each module in the above-mentioned analysis model construction device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0153] See also Figure 7 , Figure 7 is a schematic diagram of a computer device in one embodiment of the present invention. In one embodiment, a computer device is provided. The computer device may be a server. The internal structure diagram thereof may be as shown in FIG. Figure 7 The computer device includes a processor, a memory, a network interface and a database connected through a system bus.

[0154] The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client through a network connection. When the computer program is executed by the processor, the functions or steps of an analysis model construction method can be implemented on the server side.

[0155] See also Figure 8 , Figure 8 is another structural diagram of a computer device in one embodiment of the present invention. In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as shown in FIG. Figure 8 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external task database through a network connection. When the computer program is executed by the processor, the functions or steps of an analysis model construction method can be implemented on the client.

[0156] In one embodiment, a computer device is provided, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor.

[0157] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant descriptions of the server and the client in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0158] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a graphics processing unit (GPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0159] The above description and accompanying drawings fully illustrate the embodiments of the present disclosure so that those skilled in the art can practice them. Other embodiments may include structural, logical, electrical, process and other changes. The embodiments represent possible changes only. Unless explicitly required, separate components and functions are optional, and the order of operation may vary. Parts and subsamples of some embodiments may be included in or replace parts and subsamples of other embodiments. Moreover, the words used in this application are only used to describe the embodiments and are not used to limit the claims. As used in the description of the embodiments and claims, unless the context clearly indicates, the singular forms of "a", "an" and "the" are intended to include plural forms as well. Similarly, the term "and / or" as used in this application refers to any and all possible combinations of listings containing one or more associated ones. In addition, when used in the present application, the term "comprise" and its variants "comprises" and / or comprising refer to the presence of a stated sub-sample, whole, step, operation, element, and / or component, but do not exclude the presence or addition of one or more other sub-samples, wholes, steps, operations, elements, components and / or groups of these. In the absence of further restrictions, the elements defined by the sentence "comprising a ..." do not exclude the presence of other identical elements in the process, method or device comprising the elements. In this article, each embodiment may focus on the differences from other embodiments, and the same and similar parts between the various embodiments may refer to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can refer to the description of the method part.

[0160] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. Technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present disclosure. Technicians can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0161] In the embodiments disclosed herein, the disclosed methods and products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units can be only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some sub-samples can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to implement this embodiment. In addition, each functional unit in the embodiment of the present disclosure may be integrated in a processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit.

[0162] The flowchart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to the embodiment of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. In the description corresponding to the flowchart and the block diagram in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in a different order from the order disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified functions or actions, or may be implemented by a combination of dedicated hardware and computer instructions.

Claims

1. A method for constructing an analysis model, characterized in that: include: Obtaining a preset news text and a preset analysis result corresponding to the preset news text; Input the preset news text into the first fully connected layer connected to the Bert model, obtain a first prediction result output by the first fully connected layer, train the BERT model through the first prediction result and the preset analysis result, and obtain the trained BERT model; Obtain a distillation model corresponding to the trained Bert model; Obtain a first feature extraction result output by the trained Bert model based on the preset news text, and obtain a second feature extraction result output by the distillation model based on the preset news text; Obtaining an absolute value loss between the first feature extraction result and the second feature extraction result, and when the absolute value loss satisfies a stop condition, stopping training the distillation model, and saving the trained distillation model; When the preset conditions are met, an analysis model is constructed through the trained distillation model and the second fully connected layer.

2. The analysis model construction method according to claim 1, characterized in that: The step of inputting the preset news text into the first fully connected layer connected to the Bert model, obtaining a first prediction result output by the first fully connected layer, training the BERT model through the first prediction result and the preset analysis result, and obtaining the trained BERT model comprises: Input the preset news text into the first fully connected layer connected to the Bert model, obtain a first prediction result output by the first fully connected layer, and obtain a first loss value between the first prediction result and the preset analysis result through a cross entropy loss function; The parameters in the BERT model are trained by using a back propagation algorithm and the first loss value. When the first loss value is less than a first preset value, the training of the parameters in the BERT model is stopped to obtain the trained BERT model.

3. The analysis model construction method according to claim 1, characterized in that: The step of obtaining a distillation model corresponding to the trained Bert model includes: Obtain the encoding layer and the attention blocks of the first three layers of the trained Bert model; A distillation model is constructed through the encoding layer and the attention block.

4. The analysis model construction method according to claim 1, characterized in that: The step of obtaining a first feature extraction result output by the trained Bert model based on the preset news text, and obtaining a second feature extraction result output by the distillation model based on the preset news text, comprises: Inputting the preset news text into the trained Bert model, and inputting the preset news text into the distillation model; The first feature extraction result output by the trained Bert model based on the current preset news text is obtained, and the second feature extraction result output by the distillation model based on the preset news text is obtained.

5. The method for constructing an analysis model according to claim 1, characterized in that: The obtaining of the absolute value loss between the first feature extraction result and the second feature extraction result, and when the absolute value loss satisfies a stopping condition, stopping the training of the distillation model, and saving the trained distillation model, comprises: Obtaining the absolute value loss between the first feature extraction result and the second feature extraction result by using a preset absolute value loss function, and training the distillation model with the goal of minimizing the absolute value loss; When the reduction range of the absolute value loss is less than a preset range, the training of the distillation model is stopped, and the trained distillation model is saved.

6. The method for constructing an analysis model according to claim 1, characterized in that: When the preset conditions are met, an analysis model is constructed through the trained distillation model and the second fully connected layer, including: Decomposing the preset news text into a plurality of semantic units through the trained distillation model, extracting features from the semantic units to obtain first embedding vectors corresponding to the semantic units, inputting the first embedding vectors corresponding to the semantic units into an average pooling layer, and performing average pooling processing on the first embedding vectors corresponding to the semantic units using the average pooling layer to generate second embedding vectors; The second embedding vector is input into a hidden layer, and the second embedding vector and the third embedding vector corresponding to the noise data are concatenated using the hidden layer to generate a fourth embedding vector, and the fourth embedding vector is input into a second fully connected layer connected to the trained distillation model to obtain a second prediction result output by the second fully connected layer, and a second loss value between the second prediction result and a preset analysis result is obtained through a cross entropy loss function, and when the second loss value is less than a second preset value, an analysis model is constructed through the trained distillation model and the second fully connected layer.

7. A method for applying an analysis model, characterized in that: The analysis model is obtained according to the construction method according to any one of claims 1 to 6, and the analysis model application method comprises: The current news text is obtained, the current news text is input into the analysis model, and the current analysis result output by the analysis model based on the current news text is obtained.

8. An analysis model building device, characterized in that: include: A first acquisition module, used to acquire a preset news text and a preset analysis result corresponding to the preset news text; A second acquisition module is used to input the preset news text into a first fully connected layer connected to a Bert model, obtain a first prediction result output by the first fully connected layer, train the BERT model through the first prediction result and the preset analysis result, and obtain the trained BERT model; A third acquisition module is used to obtain a distillation model corresponding to the trained Bert model; A fourth acquisition module is used to obtain a first feature extraction result output by the trained Bert model based on the preset news text, and to obtain a second feature extraction result output by the distillation model based on the preset news text; A saving module, used for obtaining an absolute value loss between the first feature extraction result and the second feature extraction result, and when the absolute value loss satisfies a stopping condition, stopping training the distillation model and saving the trained distillation model; A construction module is used to construct an analysis model through the trained distillation model and the second fully connected layer when preset conditions are met.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the analysis model construction method according to any one of claims 1 to 6 or the analysis model application method according to claim 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the analysis model construction method according to any one of claims 1 to 6 or the analysis model application method according to claim 7 are implemented.

Images