A model training method, device and equipment based on feature selection

Abstract

Embodiments of the present specification disclose a model training method and device based on feature selection and equipment. The method comprises the following steps: obtaining shared features Z of M tasks; for the kth task, determining a weight vector of features for the task according to the shared features Z, and determining a first prediction result of the kth task according to the shared features and the weight vector; for the ith feature, replacing the ith row in the shared features Z with a preset value to generate modified shared features, and determining a second prediction result of the kth task according to the modified shared features; determining a causal effect factor of the ith feature for the kth task according to the difference between the first prediction result and the second prediction result; determining the difference between the causal effect factor and the weight vector, generating a loss value according to the difference, and training the multi-task model, so as to selectively learn features with causal relationship for each task in the training process.

Description

[0001] This application is a divisional application of the invention patent application filed on October 14, 2022, with application number 202211275449.X, entitled "A Model Training Method, Apparatus and Device Based on Feature Selection". Technical Field

[0002] This specification relates to the field of Internet technology, and in particular to a model training method, apparatus, and device based on feature selection. Background Technology

[0003] In multi-task models, because all tasks share all features without distinguishing their usefulness to each task, some tasks may be affected by features that are useful to other tasks but not to them, leading to negative transfer and a decline in model performance. Furthermore, when using conventional methods to calculate the correlation between tasks and features, potential confounding between features and task objectives may result in spurious correlations, which can also lead to negative transfer.

[0004] Therefore, a model training scheme that can accurately express the causal relationship between features and tasks is needed. Summary of the Invention

[0005] This specification provides a model training method, apparatus, device, and storage medium based on feature selection to address the following technical problem: the need for a model training scheme that can accurately express the causal relationship between features and tasks.

[0006] To solve the above-mentioned technical problems, one or more embodiments of this specification are implemented as follows:

[0007] In a first aspect, embodiments of this specification provide a model training method based on feature selection, applied to a multi-task model containing N features and M tasks, where N and M are natural numbers greater than 1. The method includes: obtaining shared features Z of the M tasks, wherein the size of Z is N*M, and each row corresponds to one feature; for the k-th task, determining the weight vector of the feature for the task based on the shared features Z. Based on the shared feature Z and the weight vector Determine the first prediction result for the k-th task. Where 1≤k≤M; for the i-th feature, replace the i-th row of the shared feature Z with a preset value to generate the modified shared feature. According to the modified shared features Determine the second prediction result for the k-th task. Where 1≤i≤N; the causal effect factor of the i-th feature on the k-th task is determined based on the difference between the first prediction result and the second prediction result. Determine the causal effect factors. and the weight vector The difference is used to generate a loss value based on the difference, and the multi-task model is trained.

[0008] In a second aspect, embodiments of this specification provide a model training apparatus based on feature selection, applied to a multi-task model containing N features and M tasks, where N and M are natural numbers greater than 1. The apparatus includes: an acquisition module for acquiring shared features Z of the M tasks, wherein the size of Z is N*M, and each row corresponds to one feature; and a first prediction module for determining, for the k-th task, the weight vector of the feature relative to the task based on the shared features Z. Based on the shared feature Z and the weight vector Determine the first prediction result for the k-th task. Where 1≤k≤M; the second prediction module, for the i-th feature, replaces the i-th row of the shared feature Z with a preset value to generate the modified shared feature. According to the modified shared features Determine the second prediction result for the k-th task. Where 1≤i≤N; the causal effect factor module determines the causal effect factor of the i-th feature for the k-th task based on the difference between the first prediction result and the second prediction result. The fusion training module determines the causal effect factors. and the weight vector The difference is used to generate a loss value based on the difference, and the multi-task model is trained.

[0009] In a third aspect, embodiments of this specification provide an electronic device, including:

[0010] At least one processor; and,

[0011] A memory communicatively connected to the at least one processor; wherein,

[0012] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method as described in the first aspect.

[0013] In a fourth aspect, embodiments of this specification provide a non-volatile computer storage medium storing computer-executable instructions, which, when read by a computer from the storage medium, cause one or more processors to perform the method described in the first aspect.

[0014] The above-described at least one technical solution adopted in one or more embodiments of this specification can achieve the following beneficial effects: By acquiring the shared features Z of the M tasks; for the k-th task, determining the weight vector of the feature for the task based on the shared features Z, and determining the first prediction result of the k-th task based on the shared features and the weight vector; for the i-th feature, replacing the i-th row in the shared features Z with a preset value to generate a modified shared feature, and determining the second prediction result of the k-th task based on the modified shared feature; determining the causal effect factor of the i-th feature for the k-th task based on the difference between the first prediction result and the second prediction result; determining the difference between the causal effect factor and the weight vector, and generating a loss value based on the difference to train the multi-task model. This achieves the selective learning of more stable causal features for each task during training, rather than spurious correlation features, thereby improving the accuracy of the model. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 A schematic diagram illustrating the correlation between a feature and a task, provided as an embodiment of this specification;

[0017] Figure 2 This is a flowchart illustrating a feature selection-based model training method provided in an embodiment of this specification.

[0018] Figure 3 A schematic diagram of the architecture of a model provided in an embodiment of this specification;

[0019] Figure 4 A schematic diagram of a feature selection-based model training device provided in the embodiments of this specification;

[0020] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this specification. Detailed Implementation

[0021] This specification provides a model training method, apparatus, device, and storage medium based on feature selection.

[0022] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0023] In current multi-task learning, it is often necessary to determine the correlation between features and tasks. However, this situation can also lead to the model seemingly selecting useful features for each task because there is a potential common factor among multiple tasks. For example... Figure 1 As shown, Figure 1 This diagram illustrates the correlation between features and tasks. X1 and Y2 exhibit a strong correlation because they may share a common underlying factor U.

[0024] For example, there is a strong correlation between ice cream sales and the number of drowning deaths in summer. This is because there is a correlation path in summer: "ice cream sales <—temperature—> number of drowning deaths", where temperature is a common underlying factor. In this case, a conventional multi-task model can give the feature of ice cream sales (i.e., X1) a very large weight in predicting the number of drowning deaths (i.e., Y2). However, in winter, as the relationship between ice cream sales and temperature becomes weaker or even almost non-existent, the feature of ice cream sales obviously cannot accurately predict the number of drowning deaths. This manifests as a negative transfer in the model's recognition accuracy for task Y2.

[0025] Based on this, the embodiments of this specification provide a model training scheme based on feature selection to accurately express the causal relationship between features and tasks, thereby improving the accuracy of the model.

[0026] In the first aspect, such as Figure 2 As shown, Figure 2 This is a flowchart illustrating a feature selection-based model training method provided in an embodiment of this specification. It is applied to a multi-task model containing N features and M tasks, where N and M are natural numbers greater than 1. The method includes the following steps:

[0027] S201, Obtain the shared feature Z of the M tasks.

[0028] For the input training samples, conventional feature extraction methods can be used to obtain N embedded features for the multi-task model. These N embedded features are then transformed into feature representations of the same dimension (denoted as D) through dimensionality transformation. The feature representations of the N embedded features can then be merged to obtain a merged feature X, where X is N*D in size.

[0029] Then, the merged feature X can be processed using the corresponding activation function to obtain the shared feature Z of the M tasks, where the size of Z is N*M and each row corresponds to one feature.

[0030] For example, X can be processed in the following way to obtain shared features. ,in, These are shared parameters in a multi-task model, which can be any multi-task model, for example, using a shared-bottom network skeleton. This is a commonly used activation function. Under this activation function, values ​​less than 0 will be reset to 0, while values ​​greater than 0 will remain unchanged.

[0031] S203, for the k-th task, determine the weight vector of the feature for the task based on the shared feature Z. Based on the shared feature Z and the weight vector Determine the first prediction result for the k-th task. Where 1≤k≤M

[0032] In this process, for each task, the input is the same shared feature Z, and the weight vector of the feature with respect to the task is learned in the process. , Let N be a vector of length N.

[0033] For example, ,in, It is a learnable weight parameter of length N, and This represents the importance of the N input features for each task. Each element in the matrix is ​​renormalized to the interval (0,1) based on the softmax activation function. Furthermore, during this process, the model can also learn weight vectors representing the importance of features for each task through self-attention mechanisms, SENet, or MSSM. .

[0034] Furthermore, it can be determined based on the shared feature Z and the weight vector. Determine the first prediction result for the k-th task. Specifically, after obtaining the shared features and weight vector, the shared features Z and the weight vector can be... Generate output features ,Right now .

[0035] The output features obtained after weight processing This clearly reflects the importance of each feature to the task much better. Therefore, it can... The output feature is used as input to the k-th task and activated by a preset activation function. Generate the first prediction result : ,in, This refers to the network parameters contained in the k-th task tower.

[0036] S205, for the i-th feature, replace the i-th row of the shared feature Z with a preset value to generate the modified shared feature. According to the modified shared features Determine the second prediction result for the k-th task. .

[0037] At this time, in order to avoid such Figure 1 In cases where features and tasks may share common factors, it is necessary to sever the connection between features and tasks. Specifically, for the i-th feature, the i-th row of the shared feature Z is replaced with a preset value.

[0038] The preset value here can be 0, or it can be that the values ​​in the i-th row of the shared feature Z are synchronously reduced (for example, by multiplying the i-th row by a positive number much less than 1, such as 0.01), so that after synchronous reduction, the feature value corresponding to the i-th row has almost no impact on the calculation of the first prediction result relative to other features. That is, the i-th feature after being replaced by the preset value no longer affects the calculation of the first prediction result.

[0039] Therefore, based on the modified shared features Determine the second prediction result for the k-th task. : In this process, the same activation function should be used to calculate the second prediction result and the first prediction result; for example, both should use the activation function [function name missing]. .

[0040] S207, determine the causal effect factor of the i-th feature on the k-th task based on the difference between the first prediction result and the second prediction result. .

[0041] First, the difference between the first prediction result and the second prediction result can be calculated: Since the first prediction result is calculated when the i-th feature is included, while the second prediction result is calculated when the i-th feature is not included, the difference between the two can already characterize the influence of the i-th feature on the k-th task.

[0042] Furthermore, for the k-th task, after calculating the differences corresponding to the other features, the causal effect factor of the i-th feature on the k-th task can be calculated jointly for all N features. :

[0043] , among them This is a preset temperature parameter used to control the clarity of the entire causal effect factor. Clearly, if... The larger the value, the stronger the causal relationship between the i-th feature and the k-th task; conversely, the smaller the value, the weaker the causal relationship. It is positively correlated with the causal relationship between the i-th feature and the k-th task.

[0044] S209, Determine the causal effect factor and the weight vector The difference is used to generate a loss value based on the difference, and the multi-task model is trained.

[0045] As mentioned earlier, the weight vector Similarly, the importance of each feature contained in the shared feature Z relative to each task is represented by a vector. Therefore, the causal effect factor can be determined. and the weight vector The differences.

[0046] For example, calculating the causal effect factor and the weight vector The distance is determined as the difference. The distance calculation here can include various distance values ​​such as KL divergence, JS divergence, Euclidean distance, or cosine similarity.

[0047] Then, the causal effect factors for each task were calculated separately. and the weight vector The distances are calculated, resulting in N*M distance values; these N*M distance values ​​are summed to obtain the feature impact loss value. The multi-task model is trained based on the loss value influenced by the aforementioned features. Where D represents the distance calculation method. For N The merged vector, and All feature positions have a length of N, and each feature position corresponds to another feature position.

[0048] Therefore, the multi-task model can be trained based on the aforementioned loss value. Specific training includes training on the aforementioned... Training is performed, as well as training is performed on the parameters contained in each mission tower.

[0049] In one implementation, training the multi-task model based on the feature-affected loss value includes: obtaining sample labels. According to the first prediction result With sample labels Determine the sample prediction loss value based on the difference ; Fuse the sample prediction loss values The loss value influenced by the aforementioned features is used to train the multi-task model.

[0050] That is, the overall sample prediction loss value of the multi-task model can be calculated first. ,in, Let be the loss function for the k-th task. The weight of the loss value for the k-th task.

[0051] Therefore, the predicted loss values ​​of the samples can be fused together. The multi-task model is trained using the loss value influenced by the aforementioned features. , among them This is a hyperparameter.

[0052] In another implementation, the regularized loss value of the weight vector can also be determined, and the feature influence loss value and the regularized loss value can be fused to train the multi-task model.

[0053] That is, a regularized loss value is introduced for the weight vector, for example, the weight vector is determined using the first norm. First regularization loss value ,thereby , here That is, to represent the weight vector The first norm, Also a hyperparameter and The size of the value is used to control the influence of the feature on the loss value and the strength of the regularization term.

[0054] like Figure 3 As shown, Figure 3This diagram illustrates the architecture of a model provided in an embodiment of this specification. The diagram exemplifies a multi-task learning model for two tasks (task A and task B). Each task calculates its corresponding weight vector. and causal effect factors and the corresponding task B and Thus, the results of task A can be calculated separately. And Task B Then, the loss value is calculated to train TowerA and TowerB.

[0055] By acquiring shared features Z from the M tasks; for the k-th task, determining the weight vector of the feature relative to the task based on the shared features Z, and determining the first prediction result for the k-th task based on the shared features and the weight vector; for the i-th feature, replacing the i-th row of the shared features Z with preset values ​​to generate a modified shared feature, and determining the second prediction result for the k-th task based on the modified shared feature; determining the causal effect factor of the i-th feature for the k-th task based on the difference between the first prediction result and the second prediction result; determining the difference between the causal effect factor and the weight vector, and generating a loss value based on the difference to train the multi-task model. This allows for the selective learning of more stable causal features for each task during training, rather than spurious correlation features, thereby improving the model's accuracy.

[0056] Based on the same idea, one or more embodiments of this specification also provide apparatus and devices corresponding to the above methods.

[0057] In the second aspect, such as Figure 4 As shown, Figure 4 This is a schematic diagram of a feature selection-based model training device provided in an embodiment of this specification. It is applied to a multi-task model containing N features and M tasks, where N and M are natural numbers greater than 1. The device includes:

[0058] The acquisition module 401 acquires the shared feature Z of the M tasks, wherein the size of Z is N*M, and each row corresponds to one feature;

[0059] The first prediction module 403, for the k-th task, determines the weight vector of the features for the task based on the shared feature Z. Based on the shared feature Z and the weight vector Determine the first prediction result for the k-th task. Where 1≤k≤M;

[0060] The second prediction module 405, for the i-th feature, replaces the i-th row of the shared feature Z with a preset value to generate the modified shared feature. According to the modified shared features Determine the second prediction result for the k-th task. Where 1≤i≤N;

[0061] Causal effect factor module 407 determines the causal effect factor of the i-th feature for the k-th task based on the difference between the first prediction result and the second prediction result. ;

[0062] The fusion training module 409 determines the causal effect factors. and the weight vector The difference is used to generate a loss value based on the difference, and the multi-task model is trained.

[0063] Optionally, the first prediction module 403 multiplies the shared feature Z and the weight vector. Generate output features The output feature is activated using a preset activation function. Generate the first prediction result Correspondingly, the second prediction module 405, based on the modified shared features... Determine the second prediction result for the k-th task. This includes: activating the modified shared feature using the preset activation function. Generate a second prediction result .

[0064] Optionally, the causal effect factor module 407 calculates the causal effect factor. and the weight vector The distance is determined as the difference.

[0065] Optionally, the fusion training module 409 calculates the causal effect factors for each task separately. and the weight vector The distances are calculated, resulting in N*M distance values; these N*M distance values ​​are summed to obtain the feature impact loss value. The multi-task model is trained based on the loss value influenced by the features.

[0066] Optionally, the fusion training module 409 acquires sample labels. According to the first prediction result With sample labels Determine the sample prediction loss value based on the difference ; Fuse the sample prediction loss values The loss value influenced by the aforementioned features is used to train the multi-task model.

[0067] Optionally, the fusion training module 409 determines the regularized loss value of the weight vector and fuses the feature influence loss value and the regularized loss value to train the multi-task model.

[0068] Optionally, the second prediction module replaces the i-th row of the shared feature Z with 0; or, synchronously reduces the value in the i-th row of the shared feature Z.

[0069] Optionally, in the device, the distance includes KL divergence, JS divergence, Euclidean distance, or cosine similarity.

[0070] In the third aspect, such as Figure 5 As shown, Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this specification. The device includes:

[0071] At least one processor; and,

[0072] A memory communicatively connected to the at least one processor; wherein,

[0073] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method as described in the first aspect.

[0074] In a fourth aspect, based on the same idea, embodiments of this specification also provide a non-volatile computer storage medium corresponding to the above method, storing computer-executable instructions, which, when read by a computer from the storage medium, cause one or more processors to execute the method described in the first aspect.

[0075] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0076] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0077] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0078] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0079] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the embodiments of this specification can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0080] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0081] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0082] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0083] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0084] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0085] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0086] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0087] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0088] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and non-volatile computer storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0089] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0090] The above description is merely one or more embodiments of this specification and is not intended to limit this specification. Various modifications and variations can be made to the one or more embodiments of this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of one or more embodiments of this specification should be included within the scope of the claims of this specification.

Claims

1. A model training method based on feature selection, applied to a multi-task model in the Internet domain containing N features and M tasks, where N and M are natural numbers greater than 1, the method comprising: Obtain the shared feature Z of the M tasks, where the size of Z is N*M, and each row corresponds to one feature; For the k-th task, determine the weight vector of each feature for that task based on the shared feature Z. Based on the shared feature Z and the weight vector Determine the first prediction result for the k-th task. Where 1≤k≤M; For the i-th feature, replace the i-th row of the shared feature Z with a preset value to generate the modified shared feature. According to the modified shared features Determine the second prediction result for the k-th task. Where 1≤i≤N; The causal effect factor of the i-th feature on the k-th task is determined based on the difference between the first prediction result and the second prediction result. ; Determine the causal effect factor and the weight vector The difference is used to generate a loss value based on the difference, and the multi-task model is trained.

2. The method as described in claim 1, wherein, Based on the shared feature Z and the weight vector Determine the first prediction result for the k-th task. ,include: Multiply the shared feature Z and the weight vector Generate output features The output feature is activated using a preset activation function. Generate the first prediction result ; Accordingly, based on the modified shared features Determine the second prediction result for the k-th task. This includes: activating the modified shared feature using the preset activation function. Generate a second prediction result .

3. The method as described in claim 1, wherein, Determine the causal effect factor and the weight vector The differences include: Calculate the causal effect factor and the weight vector The distance is determined as the difference.

4. The method of claim 3, wherein, Training the multi-task model based on the loss value generated from the difference includes: Calculate the causal effect factor for each task separately. and the weight vector Given the distance, obtain N*M distance values; The sum of the N*M distance values ​​is used as the feature impact loss value. The multi-task model is trained based on the loss value influenced by the features.

5. The method of claim 4, wherein, Training the multi-task model based on the loss value influenced by the aforementioned features includes: Get sample labels According to the first prediction result With sample labels Determine the sample prediction loss value based on the difference ; Fuse the sample prediction loss value The loss value influenced by the aforementioned features is used to train the multi-task model.

6. The method of claim 4, wherein, Training the multi-task model based on the loss value influenced by the aforementioned features includes: Determine the regularized loss value of the weight vector, and train the multi-task model by fusing the feature influence loss value and the regularized loss value.

7. The method of claim 1, wherein, Replacing the i-th row in the shared feature Z with a preset value includes: Replace the i-th row in the shared feature Z with 0; or... The values ​​in the i-th row of the shared feature Z are synchronously reduced.

8. The method as described in claim 3, wherein the distance includes KL divergence, JS divergence, Euclidean distance, or cosine similarity.

9. A model training device based on feature selection, applied to a multi-task model in the Internet domain containing N features and M tasks, where N and M are natural numbers greater than 1, the device comprising: The acquisition module acquires the shared feature Z of the M tasks, wherein the size of Z is N*M, and each row corresponds to one feature; The first prediction module, for the k-th task, determines the weight vector of each feature for that task based on the shared feature Z. Based on the shared feature Z and the weight vector Determine the first prediction result for the k-th task. Where 1≤k≤M; The second prediction module, for the i-th feature, replaces the i-th row of the shared feature Z with a preset value to generate the modified shared feature. According to the modified shared features Determine the second prediction result for the k-th task. Where 1≤i≤N; The causal effect factor module determines the causal effect factor of the i-th feature for the k-th task based on the difference between the first prediction result and the second prediction result. ; The fusion training module determines the causal effect factors. and the weight vector The difference is used to generate a loss value based on the difference, and the multi-task model is trained.

10. An electronic device, comprising: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, such that the at least one processor can perform the method as described in any one of claims 1 to 8.

Images