Data processing method, device and related equipment
By extracting the target correlation data of the target object and risk estimate processing in multiple business scenarios, integrating the risk estimate probability and weight impact values, it solves the problem that it is difficult to effectively identify risks in multiple business scenarios in the prior art. , achieving higher accuracy in risk identification.
Patent Information
- Application Number
- CN202411189593.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-28
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-08-28
AI Technical Summary
The prior art is difficult to effectively identify the target object in another business scenario through the neural network model of one business scenario, resulting in a decrease in the accuracy of risk identification.
A data processing method is adopted to extract vector features of the target associated data of the target object, obtain embedded vector features, and perform risk estimate processing on these features based on multiple business scenarios, integrating the risk estimate probability of each business scenario and Weight affects the value to determine the risk identification probability of the target object.
It improves the accuracy of risk identification of target objects, can accurately identify risks in multiple business scenarios, and avoids the limitations of a single business scenario.
Smart Images

Figure CN118690182B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to data processing methods, devices and related equipment. Background Art
[0002] At present, when identifying risks for a target object in a certain business scenario, the business data of the target object in the business scenario can be obtained, and the data features of the business data can be extracted through the neural network model corresponding to the business scenario. Then, based on a single classification module for the business scenario in the neural network model, the extracted data features can be used for risk identification to obtain the risk identification probability of the target object.
[0003] However, the inventors have found in practice that when a neural network model of a business scenario (such as business scenario C1) can only be used to identify risks for the target object U1 in one business scenario, it is difficult to use the neural network model corresponding to the business scenario C1 to identify risks for the target object U1 in another business scenario (such as business scenario C2), which will reduce the accuracy of risk identification for the target object. Summary of the invention
[0004] The embodiments of the present application provide a data processing method, apparatus and related equipment, which can integrate the risk estimation probabilities corresponding to various business scenarios to obtain risk identification probabilities, thereby improving the accuracy of risk identification for target objects.
[0005] On the one hand, an embodiment of the present application provides a data processing method, the method comprising:
[0006] Extracting vector features from target-related data of the target object to obtain embedded vector features corresponding to the target-related data;
[0007] Perform feature extraction processing on the embedded vector features to obtain target extraction features corresponding to the embedded vector features;
[0008] Determine N business scenarios associated with the target-related data, perform risk estimation processing on the target extraction features based on each business scenario, and obtain the risk estimation probability corresponding to each business scenario; N is a positive integer;
[0009] Perform weight extraction processing on the embedded vector features to obtain the weight impact value corresponding to each business scenario;
[0010] Based on the risk estimation probability corresponding to each business scenario and the weighted impact value corresponding to each business scenario, the risk identification probability of the target object is determined.
[0011] On the one hand, an embodiment of the present application provides a data processing device, the device comprising:
[0012] The embedded vector extraction module is used to extract vector features from the target-related data of the target object to obtain the embedded vector features corresponding to the target-related data;
[0013] A feature extraction module is used to perform feature extraction processing on the embedded vector feature to obtain a target extraction feature corresponding to the embedded vector feature;
[0014] The risk estimation module is used to determine N business scenarios associated with the target-related data, and perform risk estimation processing on the target extraction features based on each business scenario to obtain the risk estimation probability corresponding to each business scenario; N is a positive integer;
[0015] The weight extraction module is used to perform weight extraction processing on the embedded vector features to obtain the weight impact value corresponding to each business scenario;
[0016] The risk identification module is used to determine the risk identification probability of the target object based on the risk estimation probability corresponding to each business scenario and the weighted impact value corresponding to each business scenario.
[0017] In one aspect, the present application provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method provided in the present application.
[0018] In an embodiment of the present application, vector feature extraction can be performed on the target associated data to obtain embedded vector features, and N business scenarios can be determined, and then when the target extraction feature is determined based on the embedded vector feature, the risk estimation probability corresponding to each business scenario can be determined based on the target extraction feature, and the weight influence value corresponding to each business scenario can be determined based on the embedded vector feature, and then the risk estimation probability corresponding to each business scenario can be integrated to determine the final risk identification probability. In this way, the risk identification capabilities of multiple business scenarios can be integrated. The present application can dynamically determine the weight influence value of the target associated data for each business scenario to determine the distribution of the weight influence value of the current target associated data on each business scenario that best matches, so that the risk identification probability of the business scenario corresponding to the current target associated data can be determined based on the distribution of the determined weight influence value, thereby combining the risk estimation probability of multiple business scenarios to fit the most accurate risk identification probability, and then no longer limited to the risk identification of one business scenario, and can accurately identify the risk of the target object in each business scenario, which helps to improve the accuracy of risk identification of the target object. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0020] Figure 1 It is a structural diagram of a data processing system provided in an embodiment of the present application;
[0021] Figure 2 It is a scenario schematic diagram of a data processing method provided in an embodiment of the present application;
[0022] Figure 3 It is a flowchart of a data processing method provided in an embodiment of the present application;
[0023] Figure 4 It is a schematic diagram of the structure of a target recognition network provided in an embodiment of the present application;
[0024] Figure 5 This is a training process of a target recognition network provided by an embodiment of the present application;
[0025] Figure 6 It is a flowchart of another data processing method provided in an embodiment of the present application;
[0026] Figure 7 is a schematic diagram of a training process of a first service network provided in an embodiment of the present application;
[0027] Figure 8 is a training diagram of a second service network provided in an embodiment of the present application;
[0028] Fig. 9 It is a training diagram of a third service network provided in an embodiment of the present application;
[0029] Fig.10 This is an intentional effect of enhancing the generation of a network provided by an embodiment of the present application;
[0030] Fig.11 It is a schematic diagram of the effect of distillation learning provided in an embodiment of the present application;
[0031] Fig.12 is a structural schematic diagram of a data processing device provided in an embodiment of the present application;
[0032] Fig.13 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0033] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0034] The following is an introduction to some technical terms involved in the embodiments of the present application:
[0035] Large models and their fine-tuning techniques: Large models refer to machine learning models with large parameter scales and complexity, such as the target recognition network (also called target recognition model) in this application. Relying on larger and more complex data sets or training tasks. Large models are usually able to learn more subtle patterns and regularities, thus having stronger generalization and expression capabilities. Large model fine-tuning technology is a deep learning model training method, mainly used to deal with situations where labeled samples are scarce. This technology usually requires a model that has been pre-trained with a large amount of data. During fine-tuning, the pre-trained model will be further trained on a data set for a specific task, and this data set is usually much smaller than the data set in the pre-training stage. The fine-tuning process allows the model to adapt to specific tasks and learn task-related knowledge. The advantage of large model fine-tuning technology is that the pre-trained model has learned a wealth of knowledge, so even if there are only a small number of labeled samples in the fine-tuning stage, it can achieve good results. This technology has been widely used in natural language processing, computer vision and other fields.
[0036] Expert Network: Expert Network is a deep learning model structure, each expert network is responsible for processing a specific part or a specific type of input data. These expert networks usually work in parallel to learn knowledge of different tasks (such as different business scenarios in this application) based on the same feature module, thereby enhancing the generalization ability of the model.
[0037] Few-shot learning and zero-shot learning: Few-shot learning refers to the task of training a machine learning model with only a small number of training sample labels. Zero-shot learning refers to the task of training a machine learning model without any training sample labels.
[0038] Financial risk control model: A financial risk control model is a mathematical model used to predict and quantify financial risks. Risk control service providers use statistical analysis data based on sample data provided by customers to build artificial intelligence models to predict users' fraud / credit risks. In the field of financial risk control, models can be divided into many types, including fraud models, credit models, qualification models, etc.
[0039] See also Figure 1 , Figure 1Schematic diagram of a data processing system provided in an embodiment of the present application. Figure 1 As shown, the data processing system may include terminal devices (such as device 11a, device 12a, device 13a) and server 100a. It can be understood that Figure 1 The number of terminal devices and servers in the example is only for illustration, and any number of terminal devices and servers may be provided according to the implementation requirements. The terminal devices (such as device 11a, device 12a, and device 13a) may communicate with the server through a network (i.e., a medium providing a communication link through a wired or wireless communication link or an optical fiber cable, etc.) to transmit data.
[0040] It is understandable that a client may be running on the terminal device (such as device 12a), and the client may be a program that provides local services to the user (also referred to as the target object, the operation object). The server 100a may be a server corresponding to the client, and the server 100a may run a program for providing resources, service data and other services to the client. It is understandable that the client running on the terminal device may also be referred to as an application client, a business client, and the like. For example, the client running on the terminal device may be a client for applying for a loan (such as a client of a bank, a client of an online financial platform, and the like), and then the user may apply for a loan through the client, and the server may obtain the user's target association data when detecting that the user has submitted a loan application, thereby performing risk identification on the user based on the target association data, so as to identify whether the user has a risk of overdue repayment after the loan, and thereby determine whether to approve the user's loan application based on the result of the risk identification.
[0041] It is understandable that terminal devices (such as device 11a) may include but are not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, aircraft, smart speakers, smart home appliances, etc., and are not limited here. The server 100a can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms, etc., and are not limited here. The embodiments of the present application can be applied to the above-mentioned terminal devices, and can also be applied to the above-mentioned servers, and are not limited here.
[0042] See also Figure 2 , Figure 2 Schematic diagram of a data processing method provided in an embodiment of the present application. Figure 2 As shown, the target associated data X1 of the target object U1 to be risk identified can be obtained (such as Figure 2 As shown in 201a in the figure, the target object U1 may be an individual, an enterprise, an institution, etc. The target associated data X1 may be data associated with the target object U1 for risk identification. For example, the data attributes included in the target associated data X1 may be the contact information of the target object (such as mobile phone number, landline number, etc.), object identity information (such as personal identity), time information, data source information, and consumption amount information, which are not limited here. Furthermore, vector feature extraction may be performed based on the target associated data X1 to obtain embedded vector features (such as Figure 2 As shown in 202a in the figure), the target associated data X1 can be subjected to sequence conversion processing to obtain conversion sequence data. The conversion sequence data may include word segmentation conversion data corresponding to each data attribute. One word segmentation conversion data may be obtained by converting word segmentation attribute data on one data attribute. Thus, the word segmentation attribute data on each data attribute can be represented by a unique word segmentation conversion data, thereby improving the convenience of data input. Then, embedding vector feature extraction is performed on the conversion sequence data to obtain an embedded vector feature (such as Figure 2 202a in the figure). The embedding vector feature extraction may be used to convert the word segmentation conversion data in the conversion sequence data into a vector representation, thereby determining the embedding vector feature based on the vector representation corresponding to each word segmentation conversion data.
[0043] Furthermore, feature extraction can be performed on the embedded vector features to obtain target extracted features (such as Figure 2 203a in the example), and then extract the features based on the target to perform risk estimation processing to obtain the risk estimation probability corresponding to each business scenario. Here, the business scenarios including business scenarios C1, C2, ..., Cn are used as examples for explanation, such as Figure 2 As shown in 204a, the risk estimation probability P01 corresponding to the business scenario C1, the risk estimation probability P02 corresponding to the business scenario C2, the risk estimation probability P03 corresponding to the business scenario C3, and so on can be obtained.
[0044] Furthermore, we can embed the vector features (such as Figure 2 202a in the figure) performs weight extraction processing to obtain the weight influence value corresponding to each business scenario, such as Figure 2 As shown in 205a, the weight influence value W01 corresponding to the business scenario C1, the weight influence value W02 corresponding to the business scenario C2, the weight influence value W03 corresponding to the business scenario C3, and so on can be obtained.
[0045] Furthermore, the risk identification probability P11 (e.g. Figure 2As shown in 206a in FIG. 1 ), the risk estimation probability corresponding to the corresponding business scenario can be weighted by the weight impact value corresponding to each business scenario, and then the weighted values are summed to obtain the risk identification probability. The risk identification probability can be used to indicate the probability that the predicted target object has a risk.
[0046] It should be understood that in the embodiments of the present application, the risk here may refer to the risk of overdue loan business of the target object, or the risk of tax situation of the target object, etc. Further, the target object can be managed based on risk identification. For example, when the target object is determined to have overdue risk based on the risk identification probability, the target object can be blacklisted so that the target object will not be provided with loan business in the future, or the loan amount of the target object can be reduced. For another example, when the target object is determined to have tax risk based on the risk identification probability, the tax index of the target object can be reduced, and the tax management of the target object can be strengthened.
[0047] It should be noted that this application can display a prompt interface, pop-up window or output voice prompt information before collecting relevant data of the user and during the process of collecting relevant data of the user. The prompt interface, pop-up window or voice prompt information is used to prompt the user that its relevant data is currently being collected, so that this application only starts to execute the relevant steps of obtaining relevant data of the user after obtaining the confirmation operation issued by the user to the prompt interface or pop-up window, otherwise (that is, when the confirmation operation issued by the user to the prompt interface or pop-up window is not obtained), the relevant steps of obtaining relevant data of the user are terminated, that is, the relevant data of the user is not obtained. In other words, all user data collected by this application are collected with the consent and authorization of the user, and the collection, use and processing of relevant user data need to comply with the relevant laws, regulations and standards of the relevant regions.
[0048] It is understood that the above scenarios are only examples and do not constitute a limitation on the application scenarios of the technical solutions provided in the embodiments of the present application. The technical solutions of the present application can also be applied to other scenarios. For example, it is known to those skilled in the art that with the evolution of the system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.
[0049] For further information, see Figure 3 , Figure 3 is a flow chart of a data processing method provided in an embodiment of the present application. The method can be executed by an electronic device, for example, the electronic device can be Figure 1 The electronic device 11a in the embodiment of the present invention. The method may at least include the following steps S101 to S105.
[0050] S101. Extract vector features from target-related data of a target object to obtain embedded vector features corresponding to the target-related data.
[0051] The target object may be an object to be risk identified, and the target object may be an individual, enterprise or institution, etc., and may also be referred to as a business object. It should be understood that the risk identification here may be to identify whether the target object has a risk of overdue loans (such as loans taken by the target object from a bank or enterprise, loans for consumer payments, etc.), or may be to identify whether the target object has tax risks, etc.
[0052] The target-related data may be data used to identify the risk of the target object, and the target-related data may be data associated with the target object. For example, when identifying the overdue risk of the target object, the target-related data may include segmentation attribute data on multiple data attributes (i.e., data corresponding to one data attribute). For example, the data attributes of the target-related data may be the contact information (such as mobile phone number, landline number, etc.) of the target object, object identity information (such as personal identity identification), time information, data source information, consumption amount information, consumption frequency information, etc., which are not limited here. It should be understood that the data of some data attributes in the target-related data may be data that undergoes desensitization processing. For example, for the contact information and object identity information, some fields therein may imply the geographic location information of the target object. Therefore, the contact information and object identity information may be desensitized, and only the fields used to provide the geographic location information of the target object are retained. The other fields are desensitized and deleted, so that the desensitized data is determined as the target-related data, thereby improving the security of the data.
[0053] Among them, the embedded vector feature can be a feature obtained by performing vector feature extraction on the target associated data. Among them, the embedded vector feature extraction can be used to convert each word segmentation attribute data in the target associated data into a corresponding vector representation, thereby obtaining an embedded vector feature. In an embodiment of the present application, each word segmentation attribute data can be first converted into a unique word segmentation conversion data, so as to determine the corresponding vector representation based on the word segmentation conversion data converted from each word segmentation attribute data, and then determine the embedded vector feature based on the vector representation corresponding to each word segmentation conversion data.
[0054] Specifically, performing vector feature extraction on target-related data of the target object to obtain embedded vector features corresponding to the target-related data may include the following steps: performing sequence conversion processing on the target-related data to obtain converted sequence data of the target-related data; performing embedded vector feature extraction on the converted sequence data to obtain embedded vector features of the converted sequence data.
[0055] The conversion sequence data may be sequence data obtained by performing sequence conversion processing on the target associated data, and the conversion sequence data may include word segmentation conversion data corresponding to each data attribute, and a word segmentation conversion data may be converted from word segmentation attribute data on a data attribute. Word segmentation conversion data may refer to data converted from word segmentation attribute data in the target associated data, and may also be referred to as a token. For example, the target associated data may include word segmentation attribute data on data attribute S1, data attribute S2, data attribute S3, data attribute S4, and data attribute S5, and then the word segmentation attribute data corresponding to each data attribute may be converted into word segmentation conversion data, such as converting the word segmentation attribute data of data attribute S1 into token1, converting the word segmentation attribute data of data attribute S1 into token1, and so on. The word segmentation conversion data corresponding to data attributes S1-S5 are token1, token2, token3, token4, and token5, respectively. Then, the converted conversion sequence data may be: token1, token2, token3, token4, and token5.
[0056] It should be understood that by performing sequence conversion processing on the target associated data, the word segmentation conversion data corresponding to each data attribute in the conversion sequence data can be unique. It should be understood that for any data attribute, the value range (i.e., mapping space) of the word segmentation conversion data corresponding to the data attribute is different from the value range (i.e., mapping space) corresponding to other data attributes, thereby ensuring that the data on different data attributes are mapped to different word segmentation conversion data to ensure that the word segmentation conversion data corresponding to each data attribute is unique, and then the corresponding feature information on the corresponding data attribute can be represented by the unique word segmentation conversion data without considering the specific sequence position of the word segmentation conversion data in the conversion sequence data. In other words, when determining the target associated data, the embodiment of the present application does not need to consider the positional relationship of each data attribute, for example, it is not necessary to determine the target associated data by fixing the order of each data attribute, so that when risk identification is performed through unique word segmentation conversion data, the convenience of determining the data used for risk identification can be improved. For example, for data attribute 1, the word segmentation attribute data of data attribute 1 in the target associated data can be converted into a positive integer between 0 and 10. For data attribute 2, the word segmentation attribute data of data attribute 2 in the target associated data can be converted into a positive integer between 11 and 20. Then, for any target associated data, the value of the word segmentation conversion data corresponding to the converted data attribute 1 is different from the value of the word segmentation conversion data corresponding to the converted data attribute 12. In this way, the target associated data can be converted into unique word segmentation conversion data (i.e., token).
[0057] It should be understood that in the embodiment of the present application, the data used for risk identification is heterogeneous in different data attributes, that is, the values of different data attributes may exist in multiple types or formats, for example, the multiple types may specifically include numerical type, text type, time series type, image type, etc. In addition, the same value may represent different meanings (i.e., corresponding to different data attributes). For example, in the target associated data X1, the consumption amount in the most recent month is 1000, and the total loan amount in the most recent month is also 1000. Then, for the target associated data X1, the data 1000 can represent two different meanings. Therefore, the embodiment of the present application can map different data attributes to different mapping spaces, so that the word segmentation attribute data in any target associated data can be converted into unique word segmentation conversion data. For example, the mapping space of the data attribute of the consumption amount is 0-10, so that the consumption amount of 1000 can be mapped to the mapping space of 0-10, such as mapping to 1, and the mapping space for the data attribute of the total loan amount is 11-20, so that the total loan amount of 1000 can be mapped to the mapping space of 11-20, such as mapping to 11. Therefore, different word segmentation conversion data can be used to represent the numerical values on different attribute data. Even if the word segmentation conversion data appears in different positions, the value on the corresponding data attribute can be represented by the unique word segmentation conversion data, so that the input target associated data can be converted into an order-insensitive token sequence (i.e., conversion sequence data), completing the serialization of heterogeneous data for risk identification. Through the serialization of heterogeneous data, different types of data can be uniformly processed and used together to maximize the use of the characteristic information on each data attribute.
[0058] It is understandable that in the embodiment of the present application, when the target association data of the target object is obtained, the target association data can be input into the target identification network for risk identification of the target object, so that the target association data can be processed by the target identification network to achieve risk identification of the target object. It should be understood that the target identification network can include a first business network, and the first business network can include an input layer, so that the target association data can be subjected to sequence conversion processing by the input layer.
[0059] Furthermore, when extracting embedded vector features from the conversion sequence data, each word segmentation conversion data in the conversion sequence data may be converted into a corresponding vector representation, thereby determining the embedded vector features based on the vector representation corresponding to each word segmentation conversion data. For example, the conversion sequence data may include: token1, token2, token3, token4, token5, then token1 may be converted into vector V1, token12 may be converted into vector V2, token3 may be converted into vector V3, token4 may be converted into vector V4, and token5 may be converted into vector V5, and then the embedded vector features may be determined based on vectors V1-V5, such as directly combining vectors V1-V5 (such as combining each vector as a row of data in a matrix), and determining the combined matrix as the embedded vector features, or combining vectors V1-V5, and reducing the dimension of the combined matrix to obtain the embedded vector features.
[0060] It should be understood that in the first business network, an embedding layer may be cascaded after the input layer, and then the embedding layer in the first business network may be used to extract embedded vector features of the conversion sequence data to obtain embedded vector features of the conversion sequence data.
[0061] S102: Perform feature extraction processing on the embedded vector feature to obtain a target extraction feature corresponding to the embedded vector feature.
[0062] It is understood that the target extraction feature may refer to a feature obtained by performing feature extraction processing on the embedded vector feature. It should be understood that the target extraction feature may carry some information about the risk identification law, which can be used to estimate the risk probability of the target object.
[0063] It should be understood that in the first business network, a feature extraction network can be cascaded after the embedding layer, and then the feature extraction network in the first business network can perform feature extraction processing on the embedded vector features to obtain the target extraction features corresponding to the embedded vector features. It can be understood that the feature extraction network can be a model of a Transformer structure (converter model). Transformer is a deep learning model based on the self-attention mechanism, and its essence is an encoder-decoder structure. For example, the feature extraction network in the first business network can be a Transformer model with 4 depth and 8 heads, wherein 4 depth means that the depth of the Transformer model is 4, that is, the number of encoders is 4, and 8 heads means that the number of heads of the multi-head attention module in the Transformer model is 4, that is, the number of times the multi-head attention module can repeat calculations in parallel is 4. In this way, the features (i.e., target extraction features) that accurately characterize the business-related data can be extracted through the first business network, thereby improving the accuracy of risk identification of business objects.
[0064] S103, determining N business scenarios associated with the target-related data, performing risk estimation processing on the target extraction features based on each business scenario, and obtaining the risk estimation probability corresponding to each business scenario; N is a positive integer.
[0065] Among them, the business scenario may refer to a business scenario for risk estimation. For example, the business scenario may include business scenarios divided by loan type, such as first-category institutional loan scenarios (such as loans from banks), second-category institutional loan scenarios (such as loans through online platforms), third-category loan scenarios (such as loans when purchasing goods online), etc., or business scenarios divided by the timing of risk identification, such as business scenarios for risk identification at the first risk timing (such as before the loan), business scenarios for risk identification at the second risk timing (such as after the loan), etc. Alternatively, business scenarios divided according to various division methods may be combined to obtain richer business scenarios, such as business scenarios for risk identification before the loan in the first-category institutional loan scenario, business scenarios for risk identification after the loan in the first-category institutional loan scenario, business scenarios for risk identification before the loan in the third-category institutional loan scenario, business scenarios for risk identification after the loan in the third-category institutional loan scenario, etc., which are not limited here.
[0066] It should be understood that the risk estimation probability may refer to the probability that the target object has risk estimated for each business scenario, for example, the probability that the target object has a risk of overdue on the loan it has borrowed.
[0067] It should be understood that in the embodiments of the present application, the corresponding risk estimation probability can be determined for each business scenario respectively. That is to say, for the target-related data of the same target object, risk estimation can be performed for multiple business scenarios. Therefore, when performing risk identification for the target object in any business scenario, the results of risk estimation for multiple business scenarios can be referred to, rather than only performing risk identification in a single business scenario, thereby improving the generalization ability and accuracy of risk identification for the target object.
[0068] It should be understood that in the target identification network, a second business network can be cascaded after the feature extraction network of the first business network, and the second business network can include N expert networks corresponding to N business scenarios, one business scenario corresponds to one business network, and the N expert networks can be connected in parallel after the feature extraction network. Then, the target extraction features can be subjected to risk estimation processing based on the expert network corresponding to each business scenario to obtain the risk estimation probability corresponding to each business scenario. It should be understood that the essence of each expert network can be a classifier, which can be used to classify and obtain the probability that the target object has risk in the corresponding business scenario (i.e., the classification estimation probability).
[0069] It should be understood that determining the N business scenarios associated with the target association data may be based on the business scenarios corresponding to the N expert networks included in the second business network to determine the N business scenarios associated with the target association data.
[0070] S104: Perform weight extraction processing on the embedded vector features to obtain a weight influence value corresponding to each business scenario.
[0071] Among them, the weighted impact value may refer to the impact value of each business scenario when determining the result of risk identification. The larger the weighted impact value corresponding to a business scenario, the greater the impact of the risk estimation probability corresponding to the corresponding business scenario on the result of risk identification of the target object. Conversely, the smaller the weighted impact value corresponding to a target object, the smaller the impact of the risk estimation probability corresponding to the corresponding business scenario on the result of risk identification of the target object. It can be understood that the sum of the weighted impact values corresponding to the N business scenarios can be 1.
[0072] It can be understood that the risk estimation probability corresponding to each of the above-mentioned business scenarios can be determined based on the corresponding expert network. Then the weight influence value of each business scenario can also be understood as the weight influence value for each expert network, thereby representing the influence of the risk estimation probability output by each expert network on the risk identification result.
[0073] It should be understood that in the target recognition network, a third business network can be cascaded after the embedding layer of the first business network. Then, the embedded vector features can be processed by the third business network to extract weights, and the weight influence value corresponding to each business scenario can be obtained. It should be understood that the third business network may include a routing network and a gating network. The routing network can be used to perform a routing feature extraction network on the embedded vector features to obtain routing extraction features corresponding to the embedded vector features. The routing extraction features can be features obtained by performing routing feature extraction processing on the embedded vector features, and the routing extraction features can carry information for weight extraction. The gating network can be used to determine the weight influence values corresponding to each business scenario based on the routing extraction features. It can be understood that the routing network can also be a Transformer model. For example, the routing network can be a 2depth and 8head Transformer model, and the gating network can include a fully connected layer and a softmax layer.
[0074] S105. Determine the risk identification probability of the target object based on the risk estimation probability corresponding to each business scenario and the weight impact value corresponding to each business scenario.
[0075] It should be understood that the risk identification probability can be the probability that the identified target object has a risk. The risk identification probability can be obtained by integrating the risk estimation probability corresponding to each business scenario, for example, by weighted summing the risk estimation probability corresponding to each business scenario based on the weighted impact value corresponding to each business scenario.
[0076] Specifically, based on the risk estimation probability corresponding to each business scenario and the weight influence value corresponding to each business scenario, determining the risk identification probability of the target object in the first business scenario can include the following steps: weighting the risk estimation probability corresponding to each business scenario by the weight influence value corresponding to each business scenario to obtain a weighted risk estimation value corresponding to each business scenario; a weight mapping value corresponding to a business scenario is used to weight the risk estimation probability corresponding to the corresponding business scenario; further, based on the weighted risk estimation value corresponding to each business scenario, determining the risk identification probability of the target object in the first business scenario.
[0077] Among them, the weighted risk estimation value may refer to the value obtained by weighting the risk estimation probability by the weight influence value, that is, the value obtained by multiplying the weight influence value by the risk estimation probability. The risk estimation probability corresponding to a business scenario may be used to weight the risk estimation probability corresponding to the corresponding business scenario.
[0078] It can be understood that, based on the weighted risk estimation value corresponding to each business scenario, the risk estimation probability corresponding to each business scenario can be determined by summing up the weighted risk estimation values of each business scenario to obtain the risk identification probability. Based on this, when determining the risk identification probability, the weight influence values for different business scenarios can be different. If the weight influence value of the business scenario is higher, then when the weighted sum is performed based on the risk estimation probability of each business scenario, the risk estimation probability corresponding to the business scenario with a higher weight influence value has a greater impact on the risk identification probability. Therefore, the risk estimation probability of multiple business scenarios can be combined to fit the risk identification probability that best meets the business scenario corresponding to the target association data. The present application can dynamically determine the weight influence value of the target-related data for each business scenario based on the embedded vector feature, so as to determine the distribution of the weight influence values of the current target-related data in each business scenario that best matches the risk identification probability of the business scenario corresponding to the current target-related data based on the determined distribution of the weight influence values. That is, the determined risk identification probability is more accurate, and is no longer limited to the risk identification of one business scenario. It can be used to identify risks for target objects in various business scenarios, which helps to improve the accuracy of risk identification for target objects.
[0079] For example, N business scenarios include business scenario C1, business scenario C2, and business scenario C3. Then, the risk estimation probability corresponding to each business scenario can be determined, that is, the risk estimation probability P01 corresponding to business scenario C1, the risk estimation probability P02 corresponding to business scenario C2, and the risk estimation probability P03 corresponding to business scenario C3. In addition, the weighted impact value corresponding to each business scenario can be determined, that is, the weighted impact value W01 corresponding to business scenario C1, the weighted impact value W02 corresponding to business scenario C2, and the weighted impact value W03 corresponding to business scenario C3. Further, the weighted risk estimation value PW1 corresponding to business scenario C1 can be determined based on P01×W01, the weighted risk estimation value PW2 corresponding to business scenario C2 can be determined based on P02×W02, and the weighted risk estimation value PW3 corresponding to business scenario C3 can be determined based on P03×W03, and then the risk identification probability of the target object can be determined based on the sum of PW1, PW2, and PW3.
[0080] For example, the method of determining the probability of risk identification can be expressed by the following formula (Formula 1):
[0081] Formula 1
[0082] in, It can represent the probability of risk identification, Indicates the weight impact value corresponding to business scenario i, Indicates the estimated risk probability corresponding to business scenario i. The value range of i is 1-N, and N is the total number of business scenarios.
[0083] It should be understood that see Figure 4 , Figure 4 It is a structural diagram of a target recognition network provided in an embodiment of the present application. Based on the above description, it can be known that in an embodiment of the present application, the target recognition network 400a may include a first business network 401a, a second business network 402a and a third business network 403a. The specific processing process can refer to the above-mentioned related description and will not be repeated here. Among them, the first business network 401a may include an input layer 411a, an embedding layer 412a and a feature extraction network 413a. The input layer 411a can be used to perform sequence conversion processing on the target-related data of the target object to obtain conversion sequence data. The embedding layer 412a can be used to perform embedded vector feature extraction on the conversion sequence data to obtain embedded vector features of the conversion sequence data. The feature extraction network 413a can be used to perform feature extraction processing on the embedded vector features to obtain target extraction features corresponding to the embedded vector features.
[0084] The second service network 402a may include multiple expert networks corresponding to service scenarios, such as the expert network E1 (eg Figure 4 421a in FIG), the expert network E2 (as shown in FIG. Figure 4 422a in FIG), the expert network En (as shown in FIG. Figure 4 423a in FIG), wherein one expert network corresponds to one business scenario, such as expert network E1 corresponds to business scenario C1, expert network E2 corresponds to business scenario C2, and expert network En corresponds to business scenario Cn. The expert network E1 can be used to determine the risk estimation probability P01 corresponding to the business scenario C1, the expert network E2 can be used to determine the risk estimation probability P02 corresponding to the business scenario C2, and the expert network En can be used to determine the risk estimation probability P0n corresponding to the business scenario Cn.
[0085] The third business network 403a may include a routing network 431a and a gating network 432a. The routing network 431a may be used to perform routing feature extraction processing on the embedded vector feature to obtain the routing extraction feature. The gating network 432a may be used to perform weight identification on the routing extraction feature to obtain the weight influence value corresponding to each business scenario. For example, the weight influence value W01 corresponding to the business scenario C1, the weight influence value W02 corresponding to the business scenario C2, and the weight influence value W0n corresponding to the business scenario Cn may be obtained.
[0086] Furthermore, the final risk identification probability may be obtained based on the risk estimation probability corresponding to each business scenario and the weighted impact value corresponding to each business scenario, such as the risk identification probability P11.
[0087] It should be understood that the target recognition network can be obtained by training based on a large amount of sample data. The trained target recognition network can be firstly trained based on unlabeled sample data to obtain a first business network, and then a second initial business network (i.e., an initial business network used to train the second business network) is added on the basis of the trained first business network, and then the network parameters of the first business network are fixed, and the network parameters of the second initial business network are adjusted based on the labeled sample data to obtain the second business network. Further, a third initial business network (i.e., an initial business network used to train the third business network) is added on the basis of the trained first business network and the second business network, and then the network parameters of the first business network and the second business network are fixed, and the network parameters of the third initial business network are adjusted based on the labeled sample data to obtain the third business network, so that the target recognition network can be determined based on the trained first business network, the second business network, and the third business network.
[0088] The process of training the target recognition network may include the following steps: obtaining a first training data set, adjusting the network parameters of a first initial business network associated with a basic recognition network through the first training data set, and determining the first initial business network after the parameter adjustment as the first business network; further, adding a second initial business network to the first business network, wherein the second initial business network includes N initial expert networks corresponding to N business scenarios; further, obtaining a second training data set, adjusting the network parameters of the N initial expert networks through the second training data set, determining the N initial expert networks after the parameter adjustment as the N expert networks, and determining the second business network based on the N expert networks; further, adding a third initial business network to the first business network and the second business network; further, obtaining a third training data set, adjusting the network parameters of the third initial business network through the third training data set, and determining the third initial business network after the parameter adjustment as the third business network; further, based on the trained first business network, the second business network and the third business network.
[0089] The first training data set may be training data used to train the first service network. It should be understood that the sample data in the first training data set are all sample data without any label.
[0090] The first initial business network may be an initial business network used to train the second business network. The network parameters of the first initial business network may be parameters obtained by random initialization. The network structure of the first initial business network is consistent with the network structure of the first business network, that is, it may be an underlying model structure based on Transformer.
[0091] The second training data set may be training data for training the second business network. It should be understood that the sample data in the second training data set may be sample data with a scenario label, where the scenario label may be a label for indicating the business scenario to which the sample data belongs, which is specifically represented by a scenario label value for each business scenario, and the scenario label value for each business scenario may be used to indicate whether the sample data belongs to the corresponding business scenario. For example, 0 (the first label value) indicates that the sample data does not belong to the corresponding business scenario, and 1 (the second label value) indicates that the sample data belongs to the corresponding business scenario. If the business scenario includes business scenarios C1, C2, and C3, and the sample data belongs to business scenario C1, then the scenario label value of the sample data Y1 for business scenario C1 is 1, and the scenario label values for business scenarios C2 and C3 are 0. Therefore, the scenario label of the sample data may be represented as [1,0,0].
[0092] The second initial business network may be an initial business network used to train the second business network. The network parameters of the second initial business network may be parameters obtained by random initialization. The network structure of the second initial business network is consistent with the network structure of the second business network, that is, it may include N initial expert networks corresponding to N business scenarios. It should be understood that the N initial expert networks in the second initial business network may be cascaded after the feature extraction network of the first business network. It should be understood that when training the third initial business network, the network parameters of the first business network may be fixed, and only the network parameters of the second initial business network may be adjusted, so that the second business network can learn the knowledge of the business scenario of the input data.
[0093] The third training data set may be training data for training the third business network. It should be understood that the sample data in the third training data set may be sample data with a risk label, and the risk label here may be a label for indicating whether the target object corresponding to the sample data has a risk (such as the overdue risk of a loan), for example, 0 (the first label value) indicates that the target object corresponding to the sample data has no risk, and 1 (the second label value) indicates that the target object corresponding to the sample data has a risk.
[0094] The third initial business network may be an initial third business network, and the network parameters of the third initial business network may be parameters obtained by random initialization. The network structure of the third initial business network is consistent with the network structure of the third business network described above, that is, it may include an initial routing network and an initial gating network. It should be understood that the third initial business network may be cascaded after the embedding layer of the first business network. It should be understood that when training the third initial business network, the network parameters of the first business network and the second business network may be fixed, and only the network parameters of the third initial business network may be adjusted, so that the third business network can learn the ability to determine the weight influence value of each business scenario, and the entire target recognition network can have the ability to accurately identify the risk of the target object.
[0095] It should be understood that the information included in the sample data in each training data set can refer to the introduction of the target-related data mentioned above, because the embodiment of the present application can represent the data of each sample data on each data attribute through a unique token. It should be understood that due to the different sources of the sample data or the lack of data on certain data attributes in some sample data, the data attributes corresponding to each sample data may not be exactly the same, and the data corresponding to each data attribute may be heterogeneous, then the data attributes included in different sample data are not exactly the same, and then the heterogeneous data corresponding to the data attributes in the input sample data can be processed into an input sequence of indefinite length and order-insensitive (i.e., conversion sequence data), and the order of the data attributes in the input sample data does not affect the information represented by the extracted features, therefore, when the present application is training and applying the target recognition network, it is not necessary to consider the data attributes included in the input data and the order of each data attribute, thereby improving the convenience of inputting data into the target recognition network for risk identification.
[0096] It should be understood that for the multiple expert networks and routing networks in the target recognition network, the embodiments of the present application can be to fine-tune these two parts of the network separately to ensure the learning of diverse knowledge, and train multiple expert networks through sample data of different business scenarios so that each expert network has specific learning of specific scenario knowledge. The routing network is then trained with risk labels as learning targets to complete the training of the general large model (i.e., the target recognition network), matching the most suitable scenario expert knowledge for each sample data (i.e., extracting the corresponding weight influence value for each business scenario), thereby improving the generalization of the entire target recognition network and the risk identification capabilities of each business scenario.
[0097] For example, see Figure 5 , Figure 5 This is a training process of a target recognition network provided by an embodiment of the present application. Figure 5As shown, firstly, the first initial service network (such as Figure 5 As shown in 501a in FIG. 1 , the network parameters of the first initial service network are randomly initialized. Then, the first training data set is input into the first initial service network, and the first initial service network is subjected to unlabeled self-supervised training and library model pseudo-label training to obtain the first service network (as shown in FIG. 1 ). Figure 5 Further, N initial expert networks (i.e., second initial business networks) with random initialization parameters are added (as shown in 502a in FIG. Figure 5 Specifically, N initial expert networks may be added after the feature extraction network of the first service network, and the network parameters of each initial expert network are randomly initialized. Further, a second training data set is input, the first service network is fixed, and multiple initial expert networks are trained to obtain N expert networks. The second service network (such as Figure 5 Further, a third initial service network with random initialization parameters is added (such as Figure 5 Specifically, a third initial service network may be added after the embedding layer of the first service network, and the network parameters of the third initial service network are randomly initialized. Then, a third training data set may be input, the first service network and the second service network may be fixed, the third initial service network may be trained, and a third service network may be obtained. Based on the first service network, the second network and the third network, a target recognition network (such as Figure 5 As shown in 506a in FIG. 5 ), a target recognition network can be trained thereby.
[0098] In an embodiment of the present application, the target-related data can be subjected to sequence conversion processing, and conversion sequence features that are easier to characterize the target-related data can be obtained, and N business scenarios can be determined, and then the risk estimation probability corresponding to each business scenario and the weight influence value corresponding to each business scenario can be determined, and then the risk estimation probability corresponding to each business scenario can be integrated to determine the final risk identification probability. In this way, the risk identification capabilities of multiple business scenarios can be integrated. The present application can dynamically determine the weight influence value of the target-related data for each business scenario to determine the distribution of the weight influence value of the current target-related data in each business scenario that best matches, so that the risk identification probability that best matches the business scenario corresponding to the current target-related data can be determined based on the distribution of the determined weight influence value, thereby combining the risk estimation probability of multiple business scenarios to fit the most accurate risk identification probability, and then no longer limited to the risk identification of one business scenario, and can accurately identify the risk of the target object in each business scenario, which helps to improve the accuracy of risk identification of the target object.
[0099] For further information, see Figure 6, Figure 6 is a flow chart of another data processing method provided by an embodiment of the present application. The method can be executed by an electronic device, for example, the electronic device can be Figure 1 The electronic device 11a in the embodiment of the present invention. The method may at least include the following steps S201 to S209.
[0100] S201. When target association data of a target object is obtained, a target identification network for identifying risks of the target object is obtained; the target identification network includes a first business network.
[0101] Among them, the introduction of the target identification network and the first service network can refer to the above related description, which will not be repeated here.
[0102] S202: Determine data attributes associated with target associated data through the first business network, and extract word segmentation attribute data of the target associated data based on the data attributes.
[0103] The data attributes can be introduced by referring to the above related description. The segmentation attribute data can refer to the data corresponding to each data attribute in the target associated data. The specific introduction can refer to the above related description, which will not be repeated here.
[0104] S203: Perform word segmentation conversion on the word segmentation attribute data through the first service network to obtain word segmentation conversion data corresponding to the word segmentation attribute data.
[0105] It should be understood that the word segmentation conversion data may be data obtained by performing word segmentation conversion on the word segmentation attribute data. One word segmentation attribute data may be used to convert into a corresponding word segmentation attribute data.
[0106] Specifically, the first business network includes an input layer; the word segmentation attribute data includes M word segmentation attribute data, the M word segmentation attribute data include target word segmentation attribute data, and M is a positive integer; then, performing word segmentation conversion on the word segmentation attribute data through the first business network to obtain word segmentation conversion data corresponding to the word segmentation attribute data may include the following steps: based on the data attribute to which the target word segmentation attribute data belongs, determining a mapping space corresponding to the target word segmentation attribute data; the mapping space includes multiple mapping values, each mapping value has a corresponding word segmentation attribute value interval; further, determining the word segmentation attribute value interval to which the target word segmentation attribute data belongs through the input layer, and determining the mapping value corresponding to the word segmentation attribute value interval to which the target word segmentation attribute data belongs as the word segmentation conversion data corresponding to the target word segmentation attribute data.
[0107] The input layer may be a network layer in the first business network for processing the input target associated data, so that the target associated data may be subjected to word segmentation conversion processing based on the input layer to obtain conversion sequence data corresponding to the target associated data. The target word segmentation attribute data may be any word segmentation attribute data among the M word segmentation attribute data, and the data attribute corresponding to the target word segmentation attribute data may be referred to as the target data attribute.
[0108] It should be understood that the mapping space may refer to the value range of the word segmentation conversion data to which the data corresponding to a data attribute can be mapped. For any data attribute, there may be a corresponding mapping space. The mapping space corresponding to the target word segmentation attribute data refers to the mapping space corresponding to the data attribute to which the target word segmentation attribute data belongs.
[0109] The mapping space may include multiple mapping values, and the mapping value may refer to the numerical value that the segmentation attribute data can be converted to. For example, if the mapping space is 1-10, then the multiple mapping values in the mapping space may be positive integers between 1 and 10. The segmentation attribute value interval may refer to the value interval of the segmentation attribute data used to map to a mapping value.
[0110] For example, a data attribute S1 (such as the consumption expenditure of the target object) corresponds to 10 mapping values in the mapping space. The value range of the data attribute S1 can be divided into 10 word segmentation attribute value intervals. For example, if the value range corresponding to the data attribute S1 is 1-1000, the value range of the data attribute S can be divided into 10 word segmentation attribute value intervals, such as 1-100, 101-200, 201-300, ..., 901-1000, a total of 10 word segmentation attribute value intervals. Further, the segmentation attribute value interval corresponding to each mapping value can be determined, and one mapping value corresponds to one segmentation attribute value interval, such as the segmentation attribute value interval corresponding to mapping value 1 is 1-100, and the segmentation attribute value interval corresponding to mapping value 2 is 101-200. By analogy, the segmentation attribute value interval corresponding to each mapping value can be obtained, and then the mapping value corresponding to the segmentation attribute data can be determined based on the segmentation attribute value interval to which the segmentation attribute data belongs. For example, if the value of the target segmentation attribute data is 430, the segmentation attribute value interval to which the target segmentation attribute data belongs can be determined from 10 segmentation attribute value intervals, that is, the interval 401-500, and the mapping value corresponding to the interval 401-500 is 5, then the mapping value corresponding to the target segmentation attribute data can be determined to be 5, and then the segmentation conversion data corresponding to the target segmentation attribute data is 5.
[0111] It should be understood that in the embodiment of the present application, when the segmentation attribute value interval corresponding to each mapping value is obtained based on the value range corresponding to the target data attribute and the number of mapping values in the mapping space, it can be obtained according to equal frequency division (also called equal frequency binning) or equal distance division (also called equal distance binning). Among them, the equal frequency division can be determined based on the value of the target data attribute in each sample data in the first training data set, so that the ratio of the number of sample data in each segmentation attribute value interval to the total number of sample data in the first training data set can be the same. For example, the value range corresponding to the data attribute S1 is 1-1000. If the value range of the data attribute S is divided into 10 segmentation attribute value intervals, the values of each sample data on the data attribute S1 can be counted, so that 10 segmentation attribute value intervals can be divided based on the value range corresponding to the data attribute S1 of 1-1000, and the number of sample data corresponding to each segmentation attribute value interval (that is, the ratio of the total number of sample data in the first training data set) is 1 / 10.
[0112] Equidistant division may refer to dividing the value range of the target data attribute according to the same distance, so that the value range corresponding to each word segmentation attribute value interval can be consistent. For example, the value range corresponding to the data attribute S1 is 1-1000. If the value range of the data attribute S is divided into 10 word segmentation attribute value intervals, it can be divided into 10 word segmentation attribute value intervals of 1-100, 101-200, 201-300, ..., 901-1000.
[0113] It should be understood that each data attribute of the target associated data can be determined as the target data attribute, so as to obtain the word segmentation conversion data corresponding to each data attribute, and then the conversion sequence data can be composed based on the word segmentation conversion data corresponding to each data attribute. It should be understood that different data attributes correspond to different mapping spaces, so that the data on each data attribute can be mapped to a unique token (i.e., word segmentation conversion data), so that the order of input data can be made insensitive, and the convenience of data input can be improved.
[0114] It should be understood that the above-mentioned word segmentation conversion method is innovatively introduced on the input layer of the present application, which can also be called tabular data word segmentation technology, which can process different heterogeneous data attribute features into variable-length and order-insensitive input sequences. The word segmenter corresponding to the input layer can be expressed as the following formula (Formula 2):
[0115]
[0116] Formula 2
[0117] in, represents the word segmenter of the input layer, and X represents the input target associated data. The word segmenter can be composed of word segmentation units corresponding to several data attributes. Indicates the data attributes The word segmentation unit (i.e., word segmenter), Represents the data attributes in the target associated data The corresponding word segmentation attribute data, for example, Can be , , The word segmentation unit (i.e., word segmenter) maps the input discrete or continuous data into an integer (i.e., word segmentation conversion data), which represents the feature value x (i.e., word segmentation attribute data) in the data attribute The corresponding quantile in the mapping space (also called quantile or percentile, which is an important concept in statistics. In a set of data, the quantile is the point that divides the set of data into several equal parts), and each quantile is determined as the mapping value in the mapping space, such as the data attribute The value range is 0-100, and the data attribute According to equal frequency binning (the proportion of sample data in each segmentation attribute value interval is the same) or equal interval binning (the interval distance of each segmentation attribute value interval is equal), it is divided into 10 segmentation attribute value intervals, and then The function maps feature x to any integer token from 1 to 10 (i.e., word segmentation conversion data, which is a globally unique integer, i.e., token_id). In addition, each data attribute of the input data of the target recognition network can be processed into a logically connected continuous integer space (i.e., the mapping space corresponding to each data attribute). The serialization of the heterogeneous features of the input data (i.e., data attributes with different representation methods, such as numerical, categorical, textual, time series, and image data attributes) can be completed. Through the serialization of heterogeneous features, different heterogeneous features can be uniformly processed and used together to maximize the use of feature information. Then the converted sequence data obtained by the above conversion is input into the embedding layer (i.e., the embedding layer) for information extraction and learning, and then passed into the feature extraction Transformer layer (i.e., the feature extraction network) to finally obtain the output embedding (i.e., the target extraction feature) to represent the compressed information of all data attributes.
[0118] It should be understood that in the embodiment of the present application, the first service network can be obtained by training with a large amount of unlabeled sample data.
[0119] Specifically, the target extraction feature is obtained by performing feature extraction processing on the embedded vector feature by the feature extraction network in the first business network; the first business network belongs to a target identification network used to identify risks of the target object; then, the embodiment of the present application also includes the following steps: obtaining a first training data set; the first sample data included in the first training data set is sample data without a risk label; further, obtaining K training tasks for the first initial business network; K is a positive integer; further, performing data processing on the first sample data through the first initial business network to obtain training extraction features associated with each training task, and determining K loss information corresponding to the K training tasks based on the training extraction features; one training task corresponds to one loss information; further, determining the first target loss information based on the K loss information, adjusting the network parameters of the first initial business network based on the first target loss information, and determining the first initial business network after the parameter adjustment as the first business network.
[0120] Among them, the first sample data may refer to the sample data of the first training data set, and the first sample data does not have a risk label. The introduction of the risk label can refer to the above-mentioned related description and will not be repeated here. Therefore, the present application can perform self-supervised training and library model pseudo-label training on the first initial business network based on the first sample data to obtain the first business network.
[0121] It should be understood that the embodiment of the present application can train the first initial service network through K training tasks. For example, the K training tasks may include one or more of the following: word segmentation mask recognition task, word segmentation replacement recognition task, noise sequence task, and pseudo label task.
[0122] Among them, the training extraction feature can be a feature obtained by processing the first sample data for each training task. The training extraction feature can be an extraction feature corresponding to the first sample data itself, or it can be an extraction feature determined for the transformed data after the first sample data is transformed.
[0123] Among them, the loss information can be used to measure the difference between the model prediction and the real data. The loss information for each training task can be determined based on the difference between the processing result determined by the training extraction feature and the training target of the corresponding training task. The first target loss information can be the loss information used to train the first initial business network based on K loss information. The first target loss information can be obtained by averaging the K loss information or by weighted summing the K loss information. The weight corresponding to each loss information can be determined according to business needs, which is not limited here.
[0124] Specifically, the K training tasks include a word segmentation and masking recognition task; then, performing data processing on the first sample data through the first initial business network to obtain training extraction features associated with each training task, and determining K loss information corresponding to the K training tasks based on the training extraction features, may include the following steps: performing sequence conversion processing on the first sample data through the first initial business network to obtain first sample conversion sequence data of the first sample data; the first sample conversion sequence data includes sample word segmentation conversion data at multiple sequence positions; further, randomly determining a first sequence position to be word segmented and masked from multiple sequence positions corresponding to the first sample conversion sequence data; further, performing word segmentation and masking on the sample word segmentation conversion data at the first sequence position in the first sample conversion sequence data based on the word segmentation and masking information; Masking processing is performed to obtain the first training sequence data; further, embedding vector feature extraction is performed on the first training sequence data through the first initial business network to obtain the first training embedding vector feature of the first training sequence data, and feature extraction processing is performed on the first training embedding vector feature to obtain the first training extraction feature corresponding to the first training embedding vector feature; the first training extraction feature belongs to the training extraction feature; further, through the recognition component associated with the word segmentation masking recognition task, the first training extraction feature is subjected to masked word segmentation recognition processing to obtain the predicted word segmentation conversion data corresponding to the word segmentation masking information in the first training sequence data; further, based on the sample word segmentation conversion data and the predicted word segmentation conversion data at the first sequence position in the first sample conversion sequence data, the loss information corresponding to the word segmentation masking recognition task is determined.
[0125] Among them, the word segmentation mask recognition task can be a task for randomly masking the conversion sequence data corresponding to the first sample data and predicting the masked word segmentation conversion data, which can also be called an MLM task. It should be understood that the word segmentation mask recognition task can be a task that allows the first initial business network to receive sequence data containing random masks, and the word segmentation mask recognition task makes the training goal of the first initial business network predict these masked words. Such a task setting requires the first initial business network to learn and understand the context information in the input sequence data, thereby capturing the relationship information between the various data attributes in the input sequence data, so that the first initial business network can extract better features.
[0126] It should be understood that the first sample conversion sequence data may be sequence data obtained by performing sequence conversion processing on the first sample data, and the first sample conversion sequence data includes sample word segmentation conversion data at multiple sequence positions. The method for performing sequence conversion processing on the first sample data may refer to the relevant description of performing sequence conversion processing on the first target associated data, which will not be described in detail here. Among them, the sample word segmentation conversion data may be data converted from the word segmentation attribute data in the first sample data, and the sequence position may refer to the position of each sample word segmentation conversion data in the first sample conversion sequence data, such as the first position, the second position, etc.
[0127] The first sequence position may be a sequence position in the first sample conversion sequence data to be subjected to word segmentation masking processing, and the first sequence position may be randomly determined. The number of the first sequence positions may be one or more, which is not limited here. The word segmentation masking processing here may be replacing the word segmentation conversion data in the first sample conversion sequence data by word segmentation masking information (mask). The word segmentation masking information may be information used to replace the word segmentation conversion data, and may also be referred to as word segmentation mask information.
[0128] The first training sequence data may be sequence data in which the sample word segmentation conversion data at the first sequence position in the first sample conversion sequence data is replaced with word segmentation masking information.
[0129] The first training embedding vector feature may be a feature obtained by extracting the embedding vector feature from the first training sequence data. The method for determining the first training embedding vector feature may refer to the above description of determining the embedding vector feature, which will not be described in detail here. For example, the first training embedding vector feature may be obtained by extracting the embedding vector feature from the first training sequence data based on the initial embedding layer in the first initial service network.
[0130] The first training extracted feature may be a feature obtained by performing feature extraction processing on the first training embedded vector feature, which may be a training extracted feature associated with a word segmentation mask recognition task, and the first training extracted feature belongs to the above-mentioned training extracted feature. The method for determining the first training extracted feature may refer to the above-mentioned description of determining the target extracted feature, which will not be described in detail here. In other words, the first training embedded vector feature may be subjected to feature extraction processing based on the initial feature extraction network in the first initial business network to obtain the first training extracted feature.
[0131] Among them, the recognition component associated with the word segmentation mask recognition task can be a component for performing masked word segmentation recognition based on the first training extraction feature, which can essentially be a classifier for predicting the word segmentation conversion data corresponding to the word segmentation mask information at the second sequence position from multiple word segmentation sets. The word segmentation set can refer to a set of mapping values corresponding to the mapping space corresponding to all data attributes, that is, it can refer to a set of all tokens.
[0132] It should be understood that the predicted word segmentation conversion data may be the word segmentation conversion data corresponding to the word segmentation shielding information in the first training sequence data predicted based on the first initial business network. Further, the sample word segmentation conversion data at the first sequence position in the first sample conversion sequence data (i.e., the training target of the word segmentation shielding recognition task) may be obtained, and based on the sample word segmentation conversion data at the first sequence position in the first sample conversion sequence data (i.e., the training target of the word segmentation shielding recognition task) and the predicted word segmentation conversion data, the loss information corresponding to the word segmentation shielding recognition task may be determined.
[0133] For example, for the first sample data, the first sample conversion sequence data of length d can be converted, and r sequence positions are sampled from 1 to d positions with a uniformly distributed probability as the first sequence position (taking r=2, the first sequence position is 2,4 as an example), and the token at the first sequence position (i.e., the sample word segmentation conversion data) is replaced with the missing token (i.e., the word segmentation masking information), recorded as MASK. And the original token (i.e., the original sample word segmentation conversion data at the first sequence position in the first sample conversion sequence data) is recorded as the label of the sample data for the word segmentation masking recognition task. Then, the data at the first sequence position in the first sample conversion sequence data is replaced by the word segmentation masking information, and the obtained first training sequence data can be recorded as , the label (i.e., training target) is Then, the MLM task (i.e., word segmentation masking recognition task) needs to predict the original token of the position removed by the MASK, that is, the predicted word segmentation conversion data can be predicted.
[0134] Furthermore, the loss information of the word segmentation masking recognition task can be determined based on the predicted word segmentation conversion data and the determined label, as can be expressed by the following formula (Formula 3):
[0135] Formula 3
[0136] in, represents the loss function of the MLM task (i.e., word segmentation and masking recognition task), where r is the set of sampled token positions (i.e., the first sequence position), Indicates the real token corresponding to the first sequence position m, Represents the hidden layer features of the first sequence position m. Represents a simple multi-classification feedforward neural network, which inputs hidden layer features and outputs predicted tokens (i.e. predicted word segmentation conversion data). Crossentropy is a multi-classification cross entropy function.
[0137] Specifically, the K training tasks include a word segmentation replacement recognition task; then, performing data processing on the first sample data through the first initial business network to obtain training extraction features associated with each training task, and determining K loss information corresponding to the K training tasks based on the training extraction features, may include the following steps: performing sequence conversion processing on the first sample data through the first initial business network to obtain first sample conversion sequence data of the first sample data; the first sample conversion sequence data includes sample word segmentation conversion data at multiple sequence positions; further, randomly determining a second sequence position to be replaced with word segmentation from the multiple sequence positions corresponding to the first sample conversion sequence data; further, performing word segmentation replacement processing on the sample word segmentation conversion data at the second sequence position in the first sample conversion sequence data based on the random word segmentation information. processing to obtain the second training sequence data; the random word segmentation information is different from the sample word segmentation conversion data at the second sequence position; further, the second training sequence data is embedded with the first initial business network to extract the feature, to obtain the second training embedded vector feature of the second training sequence data, and the second training embedded vector feature is subjected to feature extraction processing to obtain the second training extracted feature corresponding to the second training embedded vector feature; the second training extracted feature belongs to the training extracted feature; further, the second training extracted feature is subjected to replacement position recognition processing by the recognition component associated with the word segmentation replacement recognition task to obtain the word segmentation replacement prediction results corresponding to multiple sequence positions; further, based on the word segmentation replacement prediction results corresponding to the second sequence position and multiple sequence positions, the loss information corresponding to the word segmentation replacement recognition task is determined.
[0138] Among them, the word segmentation replacement recognition task can be a task for randomly replacing the conversion sequence data corresponding to the first sample data and predicting whether each word segmentation conversion data is replaced, which can also be called an RTD task. It should be understood that the word segmentation replacement recognition task can be a task that enables the first initial business network to receive sequence data containing randomly replaced word segmentation conversion data (i.e., tokens), and the word segmentation replacement recognition task can make the training goal of the first initial business network predict whether replacement occurs at each sequence position. Such a task setting enables the first initial business network to further learn the cross-information between data on various data attributes.
[0139] It should be understood that the introduction of the first sample conversion sequence data can refer to the above related description, which will not be repeated here.
[0140] The second sequence position may be a sequence position in the first sample conversion sequence data where word segmentation replacement processing is to be performed, and the second sequence position may be randomly determined. The number of the second sequence positions may be one or more, which is not limited here. The word segmentation replacement processing performed here may be replacing the word segmentation conversion data in the first sample conversion sequence data by random word segmentation information. The random word segmentation information may be information used to replace the word segmentation conversion data, and the random word segmentation information is different from the original sample word segmentation conversion data at the second sequence position, and the random word segmentation information may belong to the above-mentioned word segmentation set (i.e., the set of mapping values corresponding to the mapping space corresponding to all data attributes, that is, the set of all tokens).
[0141] The second training sequence data may be sequence data in which the sample word segmentation conversion data at the second sequence position in the first sample conversion sequence data is replaced with random word segmentation information.
[0142] The second training embedded vector feature may be a feature obtained by extracting embedded vector features from the second training sequence data. The method for determining the second training embedded vector feature may refer to the above description of determining the embedded vector feature, which will not be described in detail here. For example, the second training embedded vector feature may be obtained by extracting embedded vector features from the second training sequence data based on the initial embedding layer in the second initial service network.
[0143] The second training extracted feature may be a feature obtained by performing feature extraction processing on the second training embedded vector feature, which may be a training extracted feature associated with a word segmentation replacement recognition task, and the second training extracted feature belongs to the above-mentioned training extracted feature. The method for determining the second training extracted feature may refer to the above-mentioned description of determining the target extracted feature, which will not be described in detail here. In other words, the second training extracted feature may be obtained by performing feature extraction processing on the second training embedded vector feature based on the initial feature extraction network in the second initial business network.
[0144] Among them, the recognition component associated with the word segmentation replacement recognition task can be a component used to perform word segmentation replacement recognition based on the second training extraction features, which can essentially be a classifier used to predict whether each sequence position is to perform word segmentation replacement.
[0145] It should be understood that the word segmentation replacement prediction result can be a prediction result for indicating whether the sample word segmentation conversion data at the corresponding sequence position is subjected to word segmentation replacement processing. For example, the word segmentation replacement prediction result corresponding to each sequence position can be expressed as a probability for indicating the word segmentation replacement corresponding to each sequence position, or a first numerical value (such as 0) and a second numerical value (such as 1) can be used to indicate whether each sequence position predicted to be subjected to word segmentation replacement processing. For example, if the first sample conversion sequence data includes word segmentation conversion data at 5 sequence positions, the word segmentation replacement prediction results corresponding to each sequence position can be expressed as [0, 1, 0, 1, 0].
[0146] Furthermore, the training target of the word segmentation replacement recognition task can be determined based on the second sequence position, and the training target can include a position label of whether each sequence position actually performs word segmentation replacement, such as a position label of a first value (such as 0) indicating that the corresponding sequence position does not perform word segmentation replacement, and a position label of a second value (such as 1) indicating that the corresponding sequence position performs word segmentation replacement. For example, the first sample conversion sequence data includes word segmentation conversion data at 5 sequence positions, and the second sequence position is the 1st and 3rd sequence positions, then the training target of the word segmentation replacement recognition task can be expressed as [1, 0, 1, 0, 0]. Furthermore, based on the second sequence position and the word segmentation replacement prediction result corresponding to each sequence position, the loss information corresponding to the word segmentation replacement recognition task can be determined. Specifically, the training target of the word segmentation replacement recognition task and the word segmentation replacement prediction result corresponding to each sequence position can be determined based on the second sequence position, and the loss information corresponding to the word segmentation replacement recognition task can be determined.
[0147] For example, for the first sample data, the first sample conversion sequence data with a length of d can be converted, and r sequence positions are sampled from 1 to d positions with a uniformly distributed probability as the second sequence position (taking r=2, the first sequence position is 2,4 as an example), and the token (i.e., sample word segmentation conversion data) at the second sequence position is replaced with a random token (i.e., random word segmentation information), which is recorded as . And whether each sequence position is replaced is used as the label of the sample data for the word segmentation replacement recognition task. Then, the data at the second sequence position in the first sample conversion sequence data is replaced by random word segmentation information, and the obtained second training sequence data can be recorded as , the label (i.e., training target) is , where 0 means that the sample word segmentation conversion data at this sequence position has not been replaced, and vice versa, 1 means that the sample word segmentation conversion data at this sequence position has been replaced. Then, the RTD task (i.e., word segmentation replacement recognition task) needs to predict whether the token (i.e., sample word segmentation conversion data) at each sequence position has been replaced, that is, the word segmentation replacement prediction result corresponding to each sequence position can be predicted.
[0148] Furthermore, the loss information of the word segmentation replacement recognition task can be determined based on the word segmentation replacement prediction result corresponding to each sequence position and the determined label, as can be expressed by the following formula (Formula 4):
[0149] Formula 4
[0150] in, represents the loss function of the RTD task (i.e., word segmentation replacement recognition task), where d is the total token length of the first sample conversion sequence data (i.e., the total number of sample analysis conversion data in the first sample conversion sequence data), Represents the position label corresponding to sequence position m, where the value range of m is 1-d. Represents the hidden layer features of sequence position m. Represents a simple multi-classification feedforward neural network, which inputs hidden layer features and outputs word replacement prediction results of 0 or 1. BinaryEntropy is a binary cross entropy function.
[0151] Specifically, the K training tasks include a noise sequence task; then, performing data processing on the first sample data through the first initial business network to obtain training extraction features associated with each training task, and determining K loss information corresponding to the K training tasks based on the training extraction features, can include the following steps: randomly adding noise information to the word segmentation attribute data of the first sample data to obtain noise sample data, performing sequence conversion processing on the noise sample data to obtain noise conversion sequence data of the noise sample data; further, performing sequence conversion processing on the first sample data to obtain first sample conversion sequence data of the first sample data; further, performing embedding vector feature extraction on the noise conversion sequence data through the first initial business network to obtain The third training embedding vector feature of the noise conversion sequence data, the third training embedding vector feature is subjected to feature extraction processing to obtain a third training extracted feature corresponding to the third training embedded vector feature; the third training extracted feature belongs to the training extracted feature; further, the first sample conversion sequence data is subjected to embedding vector feature extraction through the first initial business network to obtain a fourth training embedded vector feature of the first sample conversion sequence data, the fourth training embedded vector feature is subjected to feature extraction processing to obtain a fourth training extracted feature corresponding to the fourth training embedded vector feature; the fourth training extracted feature belongs to the training extracted feature; further, based on the third training extracted feature and the fourth training extracted feature, the loss information corresponding to the noise sequence task is determined.
[0152] Among them, the noise sequence task can be a task for adding noise to the first sample data, so as to bring the features corresponding to the first sample data after adding noise closer to the features corresponding to the original first sample data as much as possible, and can also be called a CL task. It should be understood that the noise sequence task can be a task that enables the first initial service network to receive the sequence data corresponding to the first sample data to which noise is added, and the sequence data corresponding to the original first sample data. Such a task setting can enable the first initial service network to learn the features extracted from the original sequence and the noise sequence, so that the features of the two sequence data are further similar, thereby learning more information knowledge about the data attributes in the sequence.
[0153] The noise sample data may be sample data obtained by randomly adding noise information to the first sample data. The added noise information may be Gaussian distribution noise. For example, the training sample data may be , add random Gaussian distribution noise to the characteristics of the sample, and then we can get the noise sample data ,in, Represents the added noise information.
[0154] The noise conversion sequence data may be sequence data obtained by performing sequence conversion on the noise sample data, and its specific processing process may refer to the above description of performing sequence conversion on the target associated data, which will not be described in detail here. It should be understood that the method for determining the first sample conversion sequence data may refer to the above description, which will not be described in detail here.
[0155] Among them, the third training embedded vector feature can be a feature obtained by extracting embedded vector features from noise conversion sequence data. The method for determining the third training embedded vector feature can refer to the relevant description of determining the embedded vector feature above, which is not repeated here. The third training extracted feature can be a feature obtained by performing feature extraction processing on the third training embedded vector feature, which can be a training extracted feature associated with the noise sequence task, and the third training extracted feature belongs to the above-mentioned training extracted feature. The method for determining the third training extracted feature can refer to the relevant description of determining the target extracted feature above, which is not repeated here.
[0156] Among them, the fourth training embedded vector feature can be a feature obtained by extracting the embedded vector feature of the first sample conversion sequence data. The method for determining the fourth training embedded vector feature can refer to the relevant description of determining the embedded vector feature above, which is not repeated here. The fourth training extracted feature can be a feature obtained by performing feature extraction processing on the fourth training embedded vector feature, which can be a training extracted feature associated with the noise sequence task, and the fourth training extracted feature belongs to the above-mentioned training extracted feature. The method for determining the fourth training extracted feature can refer to the relevant description of determining the target extracted feature above, which is not repeated here.
[0157] It should be understood that, based on the third training extracted features and the fourth training extracted features, determining the loss information corresponding to the noise sequence task can be to determine the distance between the third training extracted features and the fourth training extracted features, and then determine the loss information of the noise sequence task based on the distance between the third training extracted features and the fourth training extracted features. It can be understood that determining the distance between the third training extracted features and the fourth training extracted features can be determined based on the recognition component associated with the noise sequence task.
[0158] For example, for the first sample data , add random Gaussian distribution noise to the sample word segmentation attribute data on the data attribute of the first sample data to obtain the noise sample data .Will and Perform sequence conversion processing respectively to obtain the first sample conversion sequence data and noise conversion sequence data Then, the CL task can identify the original sequence through the first initial business network The corresponding features (i.e. the fourth training extracted features) and the sequence of added noise The corresponding features (i.e., the third training extracted features) are then used to bring the third training extracted features and the fourth training extracted features as close as possible.
[0159] Furthermore, the loss information based on the third training feature extraction and the fourth training feature extraction can be expressed by the following formula (Formula 5):
[0160] Formula 5
[0161] in, represents the loss function of the CL task, represents the fourth training extraction feature corresponding to the first sample conversion sequence data, represents the third training extracted feature corresponding to the noise conversion sequence data. KL represents the KL divergence function, which is used to measure the distance between two vectors (ie, the third training extracted feature and the fourth training extracted feature).
[0162] Specifically, the K training tasks include a pseudo-label task; then, the first sample data is processed through the first initial business network to obtain the training extraction features associated with each training task, and the K loss information corresponding to the K training tasks is determined based on the training extraction features, which may include the following steps: performing sequence conversion processing on the first sample data to obtain first sample conversion sequence data of the first sample data; further, embedding vector feature extraction is performed on the first sample conversion sequence data through the first initial business network to obtain a fourth training embedding vector feature of the first sample conversion sequence data, and feature extraction processing is performed on the fourth training embedding vector feature to obtain a fourth training extraction feature corresponding to the fourth training embedding vector feature; the fourth training extraction feature belongs to the training extraction feature; further, based on the recognition component associated with the pseudo-label task, the fourth training extraction feature is risk classified to obtain the training risk classification probability corresponding to the first sample data; further, a library model associated with the first initial business network is obtained, and the first sample data is risk classified based on the library model to obtain the pseudo-label risk estimation probability corresponding to the first sample data; further, based on the training risk classification probability and the pseudo-label risk estimation probability, the loss information corresponding to the pseudo-label task is determined.
[0163] Among them, the pseudo-label task can be a task for determining the pseudo-label of the first sample data based on the library model, and a task for training based on the pseudo-label and the result obtained by identifying the first initial business network, which can also be called a library model pseudo-label task. Among them, the library model can be a model obtained by historical training for risk classification of target objects, and the model structure of the library model can be any classification model result, which is not limited here. Therefore, the task setting can allow the first initial business network to also learn the knowledge of the historical model (i.e., the library model).
[0164] It should be understood that the related introduction of the first sample conversion sequence data can refer to the above related description, which will not be repeated here. The method of determining the fourth training embedding vector feature and the fourth training extraction feature can refer to the above related description, which will not be repeated here.
[0165] Among them, the recognition component associated with the pseudo-label task can be a component for performing risk classification processing based on the fourth training extracted feature, and its essence can be a classifier for judging whether the sample object corresponding to the first sample data has risk. The training risk classification probability can be the probability obtained by performing risk classification processing on the fourth training extracted feature based on the recognition component associated with the pseudo-label task, which is used to indicate the probability that the sample object corresponding to the first sample data predicted by the recognition component has risk.
[0166] It should be understood that the pseudo-label risk estimation probability can be the probability obtained by risk classification based on the library model, which is used to indicate the probability that the sample object corresponding to the first sample data predicted by the library model has a risk. It should be understood that the number of the library models can be one or more. If the number of library models is multiple, the first sample data can be risk classified based on each library model to obtain the library model risk classification probability corresponding to each library model, and then the library model risk classification probabilities corresponding to each library model are averaged to determine the pseudo-label training estimation probability. It should be understood that the pseudo-label training estimation probability is also the training target of the first sample data for the pseudo-label task. Furthermore, the loss information corresponding to the pseudo-label task can be determined based on the training risk classification probability and the pseudo-label risk estimation probability (i.e., the training target for the pseudo-label task).
[0167] For example, for the first sample data, its pseudo label comes from the average of the scores of all models in the library model (i.e., the library model risk classification probability corresponding to each library model) as the pseudo label of the first sample data. A classification head (i.e., the recognition component associated with the pseudo label task) can be connected to the feature extraction Transformer layer (i.e., the feature extraction network in the first initial business network), and the first sample data is scored by the recognition component (i.e., the training risk classification probability is determined), and then the loss between the pseudo label and the training risk classification probability determined by the recognition component is calculated, so that the first initial business network can also learn the knowledge of the historical model.
[0168] Furthermore, based on the training risk classification probability and the pseudo-label risk estimation probability (i.e., the training target for the pseudo-label task), the method for determining the loss information corresponding to the pseudo-label task can be expressed by referring to the following formula (Formula 6):
[0169] Formula 6
[0170] in, represents the loss function of the pseudo-label task, represents the prediction score (i.e., the training risk classification probability) determined by the recognition component associated with the pseudo-label task after the first initial business network, Represents the average score of multiple library models in the model library (i.e., the estimated probability of pseudo-label risk).
[0171] For example, see Figure 7 , Figure 7 Schematic diagram of a training process of a first service network provided in an embodiment of the present application. Figure 7 As shown, first, a first training data set may be obtained, the first training data set including first sample data 701a, and then the first sample data 701a may be input into the first initial business network 700a. It should be understood that recognition components associated with various training tasks may be connected to the first initial business network, such as recognition components CL1 associated with the word segmentation and masking recognition task (such as Figure 7 702a in FIG. 1 ), the recognition component CL2 associated with the word segmentation replacement recognition task (as shown in FIG. 1 ). Figure 7 704a in FIG. 1 ), the recognition component CL3 associated with the noise sequence task (as shown in FIG. 1 ). Figure 7 706a in FIG), the recognition component CL4 associated with the pseudo-label task (as shown in FIG. Figure 7 As shown in 708a in FIG. 1 ). It should be understood that the first initial business network can extract the training extraction features associated with each training task. The specific determination method can refer to the above-mentioned related description and will not be repeated here. Thus, the loss information L1 corresponding to the word segmentation masking recognition task can be determined based on the recognition component CL1 (such as Figure 7 As shown in 703a in FIG. 1 ), based on the recognition component CL2, the loss information L2 corresponding to the word segmentation replacement recognition task is determined (as shown in FIG. 1 ). Figure 7 As shown in 705a in FIG. 1 ), based on the recognition component CL3, the loss information L3 corresponding to the noise sequence task is determined (eg, Figure 7 707a in FIG. 1 ). The first sample data may be input into each library model, such as input into Figure 7 The library models Mo1, Mo2, Mo3, Mob, etc. in 710a are used to determine the pseudo-label risk estimation probability based on the average of the library model risk estimation probabilities obtained by risk classification of the first sample data by each library model, and then the loss information L4 (such as Figure 7 709a in FIG. 1 ). Further, first target loss information for training the first initial service network may be determined based on the loss information L1, the loss information L2, the loss information L3, and the loss information L4. For example, the average value of the plurality of loss information may be determined as the first target loss information.
[0172] S204: Serialize the word segmentation conversion data corresponding to the word segmentation attribute data through the first business network to obtain conversion sequence data of the target associated data.
[0173] It should be understood that the serialization process here can be used to indicate that a sequence is generated based on each word segmentation conversion data, so as to obtain conversion sequence data. It should be understood that the conversion sequence data can include word segmentation conversion data corresponding to M data attributes.
[0174] S205 , extract embedded vector features from the conversion sequence data to obtain embedded vector features of the conversion sequence data.
[0175] S206: Perform feature extraction processing on the embedded vector feature to obtain a target extraction feature corresponding to the embedded vector feature.
[0176] Among them, the relevant introduction of steps S205-S206 can refer to the relevant description of the above steps S102-S103, which will not be repeated here.
[0177] S207, determining N business scenarios associated with the target-related data, performing risk estimation processing on the target extraction features based on each business scenario, and obtaining the risk estimation probability corresponding to each business scenario; N is a positive integer.
[0178] It should be understood that, based on the above description, the risk estimation probability corresponding to each business scenario can be determined based on the second business network in the target identification network.
[0179] Specifically, the target extraction feature is obtained by performing feature extraction processing on the embedded vector feature by the feature extraction network in the first business network; the first business network belongs to a target identification network used to identify risks of the target object; the target identification network also includes a second business network cascaded after the feature extraction network; then, based on each business scenario, the target extraction feature is subjected to risk estimation processing respectively to obtain the risk estimation probability corresponding to each business scenario, which may include the following steps: determining the second business network from the target identification network; the second business network includes an expert network corresponding to each business scenario; further, through the expert network corresponding to each business scenario, the target extraction feature is subjected to risk estimation processing to obtain the risk estimation probability corresponding to each business scenario; an expert network is used to determine the risk estimation probability corresponding to a business scenario.
[0180] It should be understood that the introduction of the target recognition network can refer to the above-mentioned related description, which will not be repeated here. It should be understood that in the embodiment of the present application, the target extraction features can be processed for risk estimation through the expert network corresponding to each business scenario to obtain the risk estimation probability corresponding to each business scenario, and an expert network is used to determine the risk estimation probability corresponding to a business scenario.
[0181] It should be understood that in the embodiments of the present application, a multi-expert network module based on hybrid expert technology (MOE) is creatively designed, so that the target recognition network can better learn and adapt to different scenarios, match the best scenario history knowledge for the input data, that is, design multiple expert networks corresponding to business scenarios to score the input data, and each expert network has unique scenario knowledge, so that the input data can be optimally matched to the scenario. The number of expert networks for business scenarios can be determined according to the number of scenario classifications of the existing sample data. Each expert network accepts features (i.e., target extraction features) from the feature extraction Transformer layer (i.e., the feature extraction network in the first business network) and gives its own prediction score (i.e., risk estimation probability).
[0182] It should be understood that in the embodiment of the present application, when performing risk estimation processing through the second business network in the target recognition network, it is necessary to train the second initial business network based on a large amount of sample data to obtain the second business network. When training the second business network, the business scenarios estimated by each expert network can be combined with the real business scenarios of the sample data to learn multiple expert networks with unique scene knowledge, so that the second business network learns the knowledge of multiple business scenarios. It should be understood that when training the expert network, in order to avoid "pollution" affecting the first business network, and to make the model more focused on the training of the expert network, the embodiment of the present application is an original design of a strategy based on large model fine-tuning, that is, all network parameters other than the non-expert network are frozen, and only the expert network parameters are trained and updated. Sample data of multiple business scenarios for training the expert network (that is, the second training data set can include sample data under each business scenario), and the scene labels of each sample data are determined to be modified. Specifically, for sample data of known business scenarios, the scenario label value corresponding to the business scenario to which the sample data belongs is set to the second value (such as 1), and the other scenario labels corresponding to the business scenarios to which the sample data does not belong are set to the first value (such as 0), forming a set of scenario labels, so as to use the scenario labels as the training targets of the second initial business network. For example, if there are currently 4 business scenarios and the sample data belongs to business scenario C1, the scenario label value corresponding to the business scenario C1 is determined to be 1, and the scenario label values corresponding to the remaining business scenarios are determined to be 0, so that the scenario labels for expert network training can be set to [1, 0, 0, 0], and vice versa.
[0183] Specifically, the embodiments of the present application may further include the following steps: obtaining a second training data set; the second sample data included in the second training data set is associated with a scene label; the scene label includes N scene label values corresponding to N business scenarios, one business scenario corresponds to one scene label value, and the N scene label values are determined based on the business scenario to which the second sample data belongs; further, obtaining a trained first business network, performing vector feature extraction on the second sample data through the first business network to obtain a fifth training embedded vector feature of the second sample data, performing feature extraction processing on the fifth training embedded vector feature, and obtaining a first scene label value corresponding to the fifth training embedded vector feature. Five training extracted features; further, N initial expert networks corresponding to N business scenarios are obtained, and scene recognition is performed on the fifth training extracted features based on the N respectively, to obtain N scene recognition probabilities corresponding to the N initial expert networks; one business scenario corresponds to an initial expert network, and one initial expert network is used to determine a scene recognition probability; further, the second target loss information is determined based on the N scene recognition probabilities and the N scene label values, and based on the second target loss information, the network parameters of the N initial expert networks are adjusted, and the N initial expert networks after the parameter adjustment are determined as N expert networks, and the second business network is determined based on the N expert networks.
[0184] The introduction of the second training data set can refer to the above related description, which will not be repeated here. The introduction of the scene label can refer to the above related description. The second sample data may refer to the sample data in the second training data set.
[0185] It should be understood that the second initial service network may be cascaded after the trained first service network, and thus the network parameters of the first service network may be fixed and the network parameters of the second initial service network may be adjusted.
[0186] Among them, the fifth training embedding vector feature can be a feature obtained by extracting the embedding vector feature of the second sample conversion sequence data. The method for determining the fifth training embedding vector feature can refer to the relevant description of determining the embedding vector feature above, which is not repeated here. For example, the fifth training embedding vector feature can be obtained by extracting the embedding vector feature of the second sample conversion sequence data based on the embedding layer in the first business network. It should be understood that to determine the fifth training vector feature, the second sample data can be subjected to sequence conversion processing through the first business network to obtain the second sample conversion sequence data of the second sample data; further, the embedding vector feature of the second sample conversion sequence data can be extracted through the first business network to obtain the fifth training embedding vector feature of the second sample conversion sequence data. Among them, the second sample conversion sequence data can be the sequence data obtained by performing sequence conversion processing on the second sample data through the first business network, that is, the sequence data corresponding to the second sample data. The second sample conversion sequence data includes sample segmentation conversion data at multiple sequence positions. The method for performing sequence conversion processing on the second sample data can refer to the relevant description of performing sequence conversion processing on the first target associated data above, which is not repeated here.
[0187] The fifth training extracted feature may be a feature obtained by performing feature extraction processing on the fifth training embedded vector feature. The method for determining the fifth training extracted feature may refer to the above-mentioned description of determining the target extracted feature, which will not be described in detail here. In other words, the fifth training embedded vector feature may be subjected to feature extraction processing based on the initial feature extraction network in the first service network to obtain the fifth training extracted feature.
[0188] It should be understood that the initial expert network can be an expert network that has not completed training in the second initial business network. It should be understood that one business scenario corresponds to one initial expert network, and the network structure of the initial expert network corresponding to each business scenario is consistent with the network result of the expert network corresponding to the corresponding business scenario.
[0189] It should be understood that the scene recognition probability may refer to the probability that the second sample data identified by the initial expert network belongs to the corresponding business scene. An initial expert network is used to determine a scene recognition probability. For example, the second initial business network includes an initial expert network CL1 corresponding to business scene C1, and an initial expert network CL2 corresponding to business scene C2. The output of the initial expert network CL1 is used to indicate the probability that the second sample data belongs to business scene C1 (i.e., the scene recognition probability), and the output of the initial expert network CL2 is used to indicate the probability that the second sample data belongs to business scene C2 (i.e., the scene recognition probability).
[0190] It should be understood that the second target loss information may be loss information used to train the second business network. The second target loss information may be determined based on the difference between the N scene recognition probabilities and the N scene label values. Then, the parameters of the second initial business network may be continuously adjusted until the second target loss information converges, and the second initial business network after the parameter adjustment may be determined as the second business network.
[0191] For example, based on N scene recognition probabilities and N scene label values, the method for determining the second target loss information can refer to the following formula (Formula 7):
[0192] Formula 7
[0193] in, represents second target loss information associated with the second initial service network, is the scenario label of the second sample data in multiple business scenarios, is the scene recognition probability output by the second sample data on each expert network.
[0194] For example, see Figure 8 , Figure 8 is a training diagram of a second service network provided in an embodiment of the present application. Figure 8 As shown, the second initial business network 802a can be connected after the trained first business network 801a, and the second initial business network 802a can include multiple initial expert networks, such as the initial expert network E1 corresponding to the business scenario C1, the initial expert network E2 corresponding to the business scenario C2, and the initial expert network En corresponding to the business scenario Cn. Further, a second training data set can be obtained, and the second training data set includes the second sample data 800a, and then the second sample data 800a can be input into the trained first business network 801a, so that the training extraction features corresponding to the first sample data can be obtained, and then the training extraction features can be input into each initial expert network, so that the scene recognition probability corresponding to each business scenario can be predicted based on each initial expert network, such as the scene recognition probability P41, the scene recognition probability P42, ..... scene recognition probability P4n, etc. can be obtained. Further, based on the scene label of the first sample data and the scene recognition probability of each business scenario, the second target loss information can be determined, so that the second initial business network 802a is trained based on the second target loss information.
[0195] S208: Perform weight extraction processing on the embedded vector features to obtain a weight influence value corresponding to each business scenario.
[0196] It should be understood that, based on the above description, the weight influence value corresponding to each business scenario may be determined based on the third business network in the target identification network.
[0197] Specifically, the embedded vector feature is obtained by extracting the embedded vector feature of the conversion sequence data by the embedding layer in the first business network; the conversion sequence data is obtained by performing sequence conversion processing on the target associated data by the input layer in the first business network; the first business network belongs to a target recognition network for identifying risks of the target object; the target recognition network also includes a third business network cascaded after the embedding layer; then, performing weight extraction processing on the embedded vector feature to obtain the weight influence value corresponding to each business scenario can include the following steps: determining the third business network from the target recognition network; further, performing weight extraction processing on the embedded vector feature through the third initial business network to obtain the weight influence value corresponding to each business scenario.
[0198] It is understood that the introduction of the target recognition network can refer to the above-mentioned related description, which will not be repeated here. It should be understood that in the embodiment of the present application, the weighted influence value corresponding to each business scenario is obtained by performing weight extraction processing on the embedded vector features through the third initial business network.
[0199] It should be understood that in the embodiment of the present application, after the expert network training is completed, a corresponding set of weight influence values need to be output through the third business network (i.e., the routing network and the gating network). This set of weights represents the influence of the data output by each expert network on risk identification (that is, it can represent the probability of each expert network having the best prediction accuracy in the current input data), thereby further enabling the target recognition network to have more multi-business scenario adaptation capabilities and knowledge expression for samples. Specifically, the routing network accepts the embedding (embedding vector features) from the embedding layer (i.e., the embedding layer in the first business network), and outputs it to the gating network after passing through the routing network. The gating network finally outputs a set of weights equal to the number of expert networks. This set of weight influence values is scored and integrated with each expert network to obtain the final model output score (i.e., risk identification probability). The model output score expresses the risk probability of the current input data (such as the risk probability of overdue). The closer the risk identification probability is to 1, the higher the overdue rate.
[0200] Specifically, the third business network includes a routing network and a gating network; then, performing weight extraction processing on the embedded vector features through the third initial business network to obtain the weight influence value corresponding to each business scenario can include the following steps: performing routing feature extraction processing on the embedded vector features through the routing network to obtain routing extraction features corresponding to the embedded vector features; further, performing weight identification on the routing extraction features through the gating network to obtain the weight influence value corresponding to each business scenario.
[0201] It should be understood that the route extraction feature can be a feature obtained by the routing network performing route feature extraction processing on the embedded vector feature, and can implicitly contain relevant information for weight identification. Then, the route extraction feature can be weighted through the gating network to obtain the weight impact value corresponding to each business scenario.
[0202] It should be understood that in the embodiment of the present application, when performing risk estimation processing through the third business network in the target recognition network, it is necessary to train the third initial business network based on a large amount of sample data to obtain the third business network. When training the routing network and the gating network, the purpose is to allow the routing network and the gating network to learn the matching degree of different samples for different experts, so as to better match the samples with the knowledge of the most likely scenario expert network.
[0203] Specifically, the target recognition network also includes a second business network for determining the risk estimation probability corresponding to each business scenario; then, the embodiment of the present application may also include the following steps: obtaining a third training data set; the third sample data included in the third training data set is associated with a risk label; further, obtaining a trained first business network, performing vector feature extraction on the third sample data through the first business network, obtaining a sixth training embedded vector feature of the third sample data, performing feature extraction processing on the sixth training embedded vector feature, and obtaining a sixth training extracted feature corresponding to the sixth training embedded vector feature; further, obtaining a trained second business network, performing risk estimation processing on the sixth training extracted feature through N expert networks in the second business network, and obtaining a training risk estimation corresponding to each business scenario. Probability; an expert network is used to determine the training risk estimation probability corresponding to a business scenario; further, the sixth embedding vector feature is weighted extracted through the third initial business network to obtain the training weight influence value corresponding to each business scenario; further, based on the training risk estimation probability corresponding to each business scenario and the training weight influence value corresponding to each business scenario, the training risk identification probability of the sample object associated with the third sample data is determined; further, based on the risk label and the training risk identification probability, the third target loss information is determined, based on the third target loss information, the network parameters of the third initial business network are adjusted, and the third initial business network after the parameter adjustment is determined as the third business network, and based on the first business network, the second business network and the third business network, the target identification network is determined.
[0204] The introduction of the third training data set can refer to the above related description, which will not be repeated here. The introduction of the scene label can refer to the above related description. The third sample data may refer to the sample data in the third training data set.
[0205] It should be understood that the third initial service network can be cascaded after the embedded layer of the first service network after the trained first service network and the second service network, and the network parameters of the first service network and the second service network can be fixed, and the network parameters of the third initial service network can be adjusted.
[0206] Among them, the sixth training embedding vector feature can be a feature obtained by extracting the embedding vector feature of the second sample conversion sequence data. The method for determining the sixth training embedding vector feature can refer to the relevant description of determining the embedding vector feature above, which is not repeated here. For example, the embedding vector feature of the second sample conversion sequence data can be extracted based on the embedding layer in the first business network to obtain the sixth training embedding vector feature. Among them, to determine the sixth training embedding vector feature, the third sample data can be processed by the first business network to perform sequence conversion processing to obtain the third sample conversion sequence data of the third sample data, and the embedding vector feature can be extracted from the third sample conversion sequence data to obtain the sixth training embedding vector feature of the third sample conversion sequence data. Among them, the third sample conversion sequence data can be the sequence data obtained by performing sequence conversion processing on the third sample data through the first business network, that is, the sequence data corresponding to the third sample data. The third sample conversion sequence data includes sample segmentation conversion data at multiple sequence positions. The method for performing sequence conversion processing on the third sample data can refer to the relevant description of performing sequence conversion processing on the first target associated data above, which is not repeated here.
[0207] The sixth training extracted feature may be a feature obtained by performing feature extraction processing on the sixth training embedded vector feature. The method for determining the sixth training extracted feature may refer to the above-mentioned description of determining the target extracted feature, which will not be described in detail here. In other words, the sixth training embedded vector feature may be subjected to feature extraction processing based on the initial feature extraction network in the first service network to obtain the sixth training extracted feature.
[0208] It should be understood that the training risk estimation probability may refer to the probability obtained by performing risk estimation processing on the first stream training extracted features based on the trained second service network. The method for determining the training risk estimation probability may refer to the above description of determining the training risk estimation probability, which will not be repeated here.
[0209] Among them, the training weight influence value can be obtained by performing weight extraction processing on the sixth embedded vector feature based on the third initial business network. The method for determining the training weight influence value corresponding to each business scenario can refer to the above-mentioned description of determining the weight influence value corresponding to each business scenario, which will not be repeated here.
[0210] It should be understood that the third target loss information may be loss information used to train the third business network. The third target loss information may be determined based on the difference between the risk label and the training risk identification probability. Then, the parameters of the third initial business network may be continuously adjusted until the third target loss information converges, and the third initial business network after the parameter adjustment may be determined as the third business network.
[0211] It should be understood that an initial recognition model can be constructed based on the trained first business network, the trained second business network and the third initial business network, and the network parameters of the third initial business network can be adjusted in the above manner, so as to determine the target recognition network based on the initial recognition network after parameter adjustment. Among them, all network structure parameters except the routing network and the gating network can be frozen, a large number of historical labeled samples (i.e., the third training data set) can be input, and the routing network and the gating network can be trained and updated with large model fine-tuning technology until convergence. At this point, a general large model of financial risk control with multi-scenario, multi-sample, and multi-model knowledge (i.e., target recognition network) can be obtained.
[0212] For example, the method of determining the third target loss information based on the risk label and the training risk identification probability can be expressed by referring to the following formula (Formula 8):
[0213] Formula 8
[0214] in, Information may be lost for a third target. represents the risk label of the third sample data itself, Represents the training risk recognition probability output after the fusion of the multi-expert network and the gating network.
[0215] For example, see Fig. 9 , Fig. 9 : is a training diagram of a third service network provided in an embodiment of the present application. Fig. 9 As shown, the second initial business network 902a can be connected after the trained first business network 901a, and the third initial business network 903a can be connected after the embedded layer of the trained first business network 901a. Further, a third training data set can be obtained, and the third training data set includes third sample data 900a, and then the first sample data 900a can be input into the trained first business network 901a, so that the training extraction features corresponding to the third sample data can be obtained, and then the training extraction features can be input into each initial expert network in the second business network (such as expert network E1, expert network E2, expert network E3, etc.), so that the training risk estimation probability corresponding to each business scenario can be obtained based on the prediction of each expert network, such as determining the training risk estimation probability P01 based on the expert network E1, determining the training risk estimation probability P02 based on the expert network E2, and determining the training risk estimation probability P0n based on the expert network En.
[0216] In addition, the training extraction features corresponding to the third sample data can be weighted extracted through the third initial business network 903a to obtain the training weight influence value corresponding to each business scenario, such as the training weight influence value W01, the training weight influence value W02, and the training weight influence value W0n. Furthermore, the training risk identification probability P21 can be determined based on each training risk estimation probability and each training weight influence value, so that the third target loss information can be determined based on the training risk identification probability and the risk label of the third sample data, and then the third initial business network can be trained based on the third target loss information.
[0217] S209: Determine the risk identification probability of the target object based on the risk estimation probability corresponding to each business scenario and the weight impact value corresponding to each business scenario.
[0218] It should be understood that in the embodiment of the present application, the target recognition network can also be migrated to some small models for risk identification in specific business scenarios, so that the risk identification effect of the target recognition network can be achieved with a smaller model architecture. In other words, distillation learning can be performed based on the target recognition network to obtain a target migration model. The target migration model may refer to a model learned from the target recognition network for risk identification in the target business scenario. The embodiment of the present application does not limit the model structure of the target migration model. For example, the target migration model may be a classification model based on a neural network. In some scenarios, some financial institutions with financial risk control needs (such as those that can be called target modeling objects) can be used to perform distillation learning from the target recognition network to a small model for a specific scenario, so as to identify risks based on the small model in a specific scenario. Usually, these financial institutions will provide a small portion (several thousand or tens of thousands) of sample data with risk labels for customized financial risk control modeling (i.e., target migration model).
[0219] Specifically, the N business scenarios include a target business scenario; then, an embodiment of the present application may further include the following steps: obtaining a target recognition network for identifying risks for a target object, and obtaining an initial migration model for distillation learning from the target recognition network; further, obtaining a fourth training data set associated with the target business scenario; the fourth training data set includes fourth sample data; further, performing data processing on the fourth sample data through the target recognition network to obtain a risk identification probability corresponding to the fourth sample data, and determining the risk identification probability corresponding to the fourth sample data as a pseudo sample label for training the initial migration model; further, performing data processing on the fourth sample data through the initial migration model to obtain a migration risk identification probability corresponding to the fourth sample data; further, adjusting model parameters of the initial migration model based on the migration risk identification probability and the pseudo sample label, and determining the initial migration model after parameter adjustment as the target migration model of the target recognition network; the target migration model is used to identify risks for target-related data in the target business scenario.
[0220] It should be understood that the target business scenario can be any business scenario among the N business scenarios. The initial migration model can be an untrained migration model, and this application does not limit the model structure of the initial migration model. For example, the initial migration model can be any classification model.
[0221] The fourth training data set may be a data set for training the initial migration model. The fourth sample data may be sample data in the fourth training data set. The fourth sample data may be scenario sample data with risk labels in the target business scenario (i.e., sample data in a business scenario), or may be scenario sample data without risk labels in the target business scenario, or may be enhanced sample data determined based on the scenario sample data in the target business scenario. Among them, the enhanced sample data may be sample data similar to the scenario sample data in the target business scenario. The method for determining the risk identification probability of the fourth sample data may refer to the relevant description of determining the risk identification probability of the target associated data, which will not be elaborated here.
[0222] Among them, the pseudo-label samples can be sample labels (i.e., training targets) used to train the initial migration model, so that the initial migration model can be trained with the risk identification probability determined by the target recognition network as the target, so that the trained target migration model can learn the knowledge used by the target recognition network for risk identification, thereby improving the learning efficiency of the migration model and the accuracy of risk identification. The trained target migration model can be used to identify risks for target objects in target business scenarios.
[0223] It should be understood that in an embodiment of the present application, a larger number of enhanced sample data can be found based on the scenario sample data in the target business scenario for training the initial migration model. That is, similar enhanced sample data can be determined based on the scenario sample data (labeled or unlabeled) to serve as a supplement to solve the "cold start" problem caused by the small sample size.
[0224] Specifically, the fourth sample data includes scene sample data generated in the target business scenario; then, obtaining the fourth training data set associated with the target business scenario may include the following steps: obtaining the scene sample data in the target business scenario; further, determining the sample data to be identified from the sample data set used for sample selection; further, performing similarity matching processing on the sample data to be identified and the scene sample data through the enhanced sample generation network to obtain a similarity value between the scene sample data and the sample data to be identified; further, if the similarity value meets the sample enhancement condition, the sample data to be identified is determined as enhanced sample data associated with the target business scenario; further, the enhanced sample data and the scene sample data are determined as the fourth sample data associated with the target business scenario.
[0225] The scenario sample data may refer to sample data in a business scenario, and the scenario sample data may be sample data with risk labels or sample data without risk labels, which is not limited here. The sample data set may be a data set for selecting enhanced samples, and the sample data set may be a set of data from various scenarios from multiple channels.
[0226] The sample data to be identified may be any sample data in the sample data set, and then it may be detected whether it can be used as sample data for enhanced sample data.
[0227] The enhanced sample generation network can be a network for finding sample data similar to the scene sample data. In an embodiment of the present application, the enhanced sample generation network can be an enhanced sample generation network based on the WD network, which can be a pre-trained neural network model, which can calculate the similarity of the two sample data input (i.e., similarity value), such as the distance (such as WD distance, i.e., Euclidean distance) of the features corresponding to the two sample data. To obtain the similarity value, that is, the similarity value can be expressed as a distance, and the closer the distance is to 0, the more similar the two sample data are, and vice versa. The advantage of calculating the WD distance is that no matter how far the distribution distance of the two data is, there will be no problem of gradient disappearance. By determining the enhanced sample data through the enhanced sample generation network, and obtaining the target migration model through the enhanced sample data training, a financial risk control model (i.e., target migration model) can be constructed that has both strong generalization and strong fitting learning for specific scenarios (such as target business scenarios). It can be understood that the generalization here can refer to the model's ability to predict new data. In actual financial risk control scenarios, the amount of modeling sample data of financial institutions is often small and can only represent the performance of some users in the institutional risk control scenarios. At this time, the target migration model needs to have a certain ability to predict new data to ensure that in future business, it has good risk control capabilities when facing samples that did not appear during training.
[0228] For example, see Fig.10 , Fig.10 This is an intentional enhancement of the effect of generating a network provided by the embodiment of the present application. Fig.10 As shown, the enhanced sample generation network 1004a may include a feature extractor (such as Fig.10 1005a) and distance calculation components (such as Fig.10 1006a in the figure). First, the scene sample data 1002a of the target business scene can be obtained, and the sample data to be identified 1003a can be determined from the sample data set 1001a. For example, a sample data can be randomly determined from the sample data set 1001a as the sample data to be identified. Then, both the scene sample data and the sample data to be identified can be input into the enhanced sample generation network 1004a, and the features of the two sample data can be extracted based on the feature extractor 1005a, such as the scene sample features corresponding to the scene sample data and the sample features to be identified corresponding to the sample data to be identified. The features of the two sample data (i.e., the scene sample features and the sample features to be identified) are processed for similarity through the distance calculation component 1006a to obtain a similarity value, so that it can be determined based on the similarity value whether the sample data to be identified 1003a can be used as enhanced sample data.
[0229] It should be understood that the sample enhancement condition may be a condition that the similarity value needs to satisfy when the sample data to be identified is determined as enhanced sample data. For example, if the similarity value is expressed as a distance, the closer the similarity value is to 0, the more similar the two sample data are. Then, the sample enhancement condition may be that the similarity value is less than or equal to the similarity threshold. For example, the sample enhancement condition may be that the similarity value is less than 0.1, that is, the similarity threshold is 0.1. Furthermore, the enhanced sample data and the scene sample data may be used to construct a fourth training data set for training to obtain a target migration model.
[0230] It should be understood that the enhanced sample generation network can be pre-trained based on a large amount of sample data. When the enhanced sample generation network is used, two task sample data (also referred to as two task tasks) can be randomly extracted from multiple sample data sources (i.e., Source sources) in the existing sample library (these sample databases can be sample data obtained based on multiple data sources) each time. Each task sample data enters the initial enhanced sample generation network and outputs a prediction value, which is used to characterize the similarity between the two task sample data and can be represented by a value of 0-1. It should be understood that if part of the sample data comes from the same sample data source, the sample label corresponding to the part of the sample data is 0, and if part of the sample data comes from different sample data sources, the sample label corresponding to the part of the sample data is 1. Then, task sample data can be repeatedly extracted from multiple sample data sources in the stock sample library to train the initial enhanced sample generation network until the initial enhanced sample generation network converges to obtain a trained enhanced sample generation network. Then, the trained enhanced sample generation network can be used to select enhanced sample data similar to the scenario sample data of the target business scenario from the massive sample pool (i.e., the sample data set) to supplement the training of the initial migration model.
[0231] For example, the method for determining the loss information of the initial enhanced sample generation network can be expressed by the following formula (Formula 9):
[0232] Formula 9
[0233] in, Represents the loss information used to obtain the enhanced sample generation network. P represents the dataset P (i.e., the dataset consisting of sample data from one sample data source), and Q represents the dataset Q (i.e., the dataset consisting of sample data from another sample data source). Indicates that the sample data in the dataset P is extracted by the feature extractor. Expectations after mapping, Indicates that the sample data in the dataset P is extracted by the feature extractor. Expectations after mapping.
[0234] It can be understood that if the fourth training data set includes labeled scene sample data, when the labeled scene sample data for training the migration model is obtained, the target recognition network can be fine-tuned based on the labeled scene sample data, and then the pseudo-sample labels of each fourth sample data can be determined based on the fine-tuned target recognition network.
[0235] Specifically, the fourth sample data includes scenario sample data generated under the target business scenario; then, inputting the fourth sample data into the target recognition network to obtain the second training risk recognition probability corresponding to the fourth sample data may include the following steps: if the scenario sample data is marked with a risk label, then fine-tuning the parameters of the target recognition network based on the scenario sample data to obtain the target recognition network after parameter fine-tuning; further, inputting the fourth sample data into the target recognition network after parameter fine-tuning to obtain the training risk recognition probability corresponding to the fourth sample data.
[0236] It should be understood that this method of fine-tuning the target recognition network based on the labeled scene sample data, and then determining the pseudo sample labels of each fourth sample data based on the fine-tuned target recognition network, can be called a small sample learning mode. The small sample here refers to the scene sample data in the target business scenario containing a small number of labeled samples.
[0237] It should be understood that when fine-tuning the target recognition network (such as a large credit risk control model) through a small amount of labeled scene sample data and enhanced sample data, the fine-tuning process may include: determining loss information through labeled scene sample data and loss function (such as cross entropy loss function), freezing the feature extraction network (feature extraction module) in the first business network in the target recognition network, and fine-tuning and optimizing the expert network (expert head module) and the routing network (risk router) in the third business network based on the loss information. The process of determining the loss information for fine-tuning the target recognition network can refer to the following formula (Formula 10):
[0238] Formula 10
[0239] Among them, Loss1 represents the loss information of fine-tuning the target recognition network, and its essence can be determined by the cross entropy loss function. Among them, y is the real risk label of the labeled scene sample data, such as 0 or 1, 0 means that the scene sample data has no risk, 1 means that the scene sample data has risk, and p is the risk recognition probability of the scene sample data predicted by the model.
[0240] Furthermore, after fine-tuning, the large model (i.e., the target recognition network) can generate pseudo-labels for the scene sample data (labeled + unlabeled) and enhanced samples of the target business scenario. The purpose of generating pseudo-labels is to use them as labels for subsequent distillation learning of the initial migration model.
[0241] Furthermore, the pseudo-labels of each sample data in the fourth training data set can be used to transfer the risk control capability of the large model (i.e., the target recognition network) to the small model (domain small model) (i.e., the target migration model) of the target business scenario, so that the small model (i.e., the target migration model) has the same risk control fraud capability as the large model (i.e., the target recognition network). Here, the logits-based distillation learning technology can be used. It should be understood that distillation learning can be a model compression technology, which is mainly used to transfer the knowledge of a large model (usually referred to as a teacher model, such as the target recognition network here) to a small model (usually referred to as a student model, such as the target migration model here). The main goal of this method is to make the small model imitate the behavior and performance of the large model as much as possible.
[0242] When performing distillation learning, the loss information for training the initial migration model can be expressed by the following formula (Formula 11):
[0243] Formula 11
[0244] Among them, Loss2 represents the loss information for training the initial migration model, which can be essentially determined by the MSE regression function. Among them, ysoft_i is the pseudo label determined by the large model (i.e., the target recognition network, or the fine-tuned target recognition network) for sample data i, is the output probability determined by the small model i to be trained (i.e. the current training model, i.e. the initial migration model mentioned above) for sample data i, and T is the temperature coefficient. The temperature coefficient is used to adjust the knowledge distillation process between the teacher model (such as the target recognition network) and the student model (such as the target migration model). It mainly affects the smoothness of the output probability distribution of the migration model. In distillation learning, the temperature coefficient (Temperature) is a hyperparameter.
[0245] It should be understood that if there is no scene sample data labeled with risk labels in the fourth training data set, a cold start scenario will occur when training the initial migration model. For this scenario, it is difficult to fine-tune the target recognition model using labeled sample data. In this case, by using zero-sample learning, the pseudo-sample labels of each unlabeled scene sample data and enhanced sample data in the fourth training data set can be directly determined through the target recognition network, and then the initial migration model can be trained with sample data with pseudo-sample labels in accordance with the above method.
[0246] For example, see Fig.11 , Fig.11 Schematic diagram of the effect of distillation learning provided by the embodiment of the present application. Fig.11 As shown, first, a fourth training data set 1101a can be obtained. The fourth sample data in the fourth training data set can include scene sample data 1102a and enhanced sample data 1103a, so that each fourth sample data in the fourth training data set can be input into the target recognition network 1104a to obtain the risk recognition probability corresponding to each sample data, and then determine the corresponding pseudo sample label 1106a based on the risk recognition probability corresponding to each sample data. One fourth sample data can generate a corresponding pseudo sample label. Further, the initial migration model can be trained based on each fourth sample data with a pseudo sample label. Specifically, each fourth sample data in the fourth training data set can be input into the initial migration model (such as Fig.11 As shown in 1105a in the figure), an output result corresponding to each sample data is obtained (such as the migration risk identification probability), and the model parameters of the initial migration model are adjusted based on the output result of the migration model and the pseudo sample label to obtain the target migration model, so that the target migration model can learn the knowledge of risk identification of the target recognition network, quickly fit the target migration module, and improve the training efficiency of the target migration model.
[0247] It should be understood that in business scenarios such as financial risk control and credit security, due to the long user lending performance cycle and the high cost of sample labeling, people who build models for risk identification often face the problem of model cold start caused by the lack of labeled samples. On the other hand, models that are directly modeled using sample data from specific business scenarios often do not have generalization capabilities, and general model products do not have the specificity to learn specific scenarios, and cannot achieve both generalization and specific scenario risk control performance. This application can make full use of the data and model knowledge of these multiple scenarios through large model technology to build a general large model that contains diverse knowledge (i.e., the above-mentioned target recognition network). However, the large model has a complex structure and many parameters, which results in more data processing resources being occupied and low efficiency, making it difficult for financial institutions to use it directly. Therefore, the present application can be based on a general large model (i.e., the above-mentioned target recognition network), and for each specific scenario, through an original adaptive model method training, generate a small model corresponding to the target scenario with fewer parameters (i.e., the target migration model), thereby constructing a customized financial risk control model (i.e., the target migration model) that has both strong generalization and risk control capabilities for specific scenarios and fewer parameters, so as to enhance risk control capabilities and improve the security of financial services.
[0248] The use of the embodiments of the present application can have many convenient advantages. (1) Technical advantages: a. Improve efficiency: Traditional financial risk control models usually require separate model training and optimization for each scenario, while the present application can process multiple scenarios at the same time through the multi-expert network fusion strategy of the hybrid expert model technology (MOE), greatly improving the efficiency of model construction and risk control. b. Reduce costs: The present application uses the pre-training strategy of self-supervision and library model distillation to effectively utilize the knowledge of unlabeled samples and multimodal model libraries, reducing the need for labeled samples, thereby reducing the cost of data acquisition and processing. c. Improve performance: Compared with traditional risk control models, the method proposed in this application can better handle heterogeneous sample data and improve the generalization ability and scenario risk control ability of the model. d. Improve quality: The method of this application can construct a financial risk control model with strong generalization and specific scenario learning capabilities (i.e., the target migration model mentioned above), improve the accuracy and reliability of risk assessment and control, and thus improve the quality of financial services.
[0249] (2) Certainty of the effect of the implementation of the application technology: The application technology is based on the latest deep learning and natural language processing technologies, and combines multiple innovative technical strategies, such as the Transformer-based large model architecture, the pre-training strategy of self-supervision and library model distillation, the multi-expert network fusion strategy of the hybrid expert model technology (MOE), and the adaptive generation model method based on the WD network and distillation learning, which effectively solves the cold start problem and builds a financial risk control model (i.e., target migration model) with strong generalization and specific scenario learning capabilities. The major improvement and innovation of the current financial risk control model construction technology can promote the advancement of financial risk control technology. Through the application technology, financial institutions can conduct risk assessment and control more accurately and efficiently, thereby improving their scientific management level.
[0250] In an embodiment of the present application, the target-related data can be subjected to sequence conversion processing, and conversion sequence features that are easier to characterize the target-related data can be obtained, and N business scenarios can be determined, and then the risk estimation probability corresponding to each business scenario and the weight influence value corresponding to each business scenario can be determined, and then the risk estimation probability corresponding to each business scenario can be integrated to determine the final risk identification probability. In this way, the risk identification capabilities of multiple business scenarios can be integrated. The present application can dynamically determine the weight influence value of the target-related data for each business scenario to determine the distribution of the weight influence value of the current target-related data in each business scenario that best matches, so that the risk identification probability that best matches the business scenario corresponding to the current target-related data can be determined based on the distribution of the determined weight influence value, thereby combining the risk estimation probability of multiple business scenarios to fit the most accurate risk identification probability, and then no longer limited to the risk identification of one business scenario, and can accurately identify the risk of the target object in each business scenario, which helps to improve the accuracy of risk identification of the target object.
[0251] See also Fig.12 , Fig.12 Schematic diagram of a data processing device provided in an embodiment of the present application. Fig.12 As shown, the data processing device 1 may be a device running on an electronic device (for example, the above Figure 1 The data processing device 1 is a computer program (including program code) of the server 100a in the embodiment of the present application. For example, the data processing device 1 is an application software. It can be understood that the data processing device 1 can be used to execute the corresponding steps of the data processing method provided in the embodiment of the present application. Fig.12 As shown, the data processing device 1 may include: an embedding vector extraction module 11, a feature extraction module 12, a risk estimation module 13, a weight extraction module 14, and a risk identification module 15;
[0252] The embedded vector extraction module 11 is used to extract vector features from target-related data of the target object to obtain embedded vector features corresponding to the target-related data;
[0253] A feature extraction module 12 is used to perform feature extraction processing on the embedded vector feature to obtain a target extraction feature corresponding to the embedded vector feature;
[0254] The risk estimation module 13 is used to determine N business scenarios associated with the target-related data, and perform risk estimation processing on the target extraction features based on each business scenario to obtain the risk estimation probability corresponding to each business scenario; N is a positive integer;
[0255] A weight extraction module 14 is used to perform weight extraction processing on the embedded vector feature to obtain a weight influence value corresponding to each business scenario;
[0256] The risk identification module 15 is used to determine the risk identification probability of the target object based on the risk estimation probability corresponding to each business scenario and the weighted impact value corresponding to each business scenario.
[0257] Wherein, the embedding vector extraction module 11 is specifically used for:
[0258] Performing sequence conversion processing on the target associated data to obtain conversion sequence data of the target associated data;
[0259] Embedded vector features are extracted from the conversion sequence data to obtain embedded vector features of the conversion sequence data.
[0260] Wherein, the embedding vector extraction module 11 is specifically used for:
[0261] When the target association data of the target object is obtained, a target identification network for risk identification of the target object is obtained; the target identification network includes a first business network;
[0262] Determining data attributes associated with the target-related data through the first business network, and extracting word segmentation attribute data of the target-related data based on the data attributes;
[0263] Performing word segmentation conversion on the word segmentation attribute data through the first business network to obtain word segmentation conversion data corresponding to the word segmentation attribute data;
[0264] The word segmentation conversion data corresponding to the word segmentation attribute data is serialized through the first business network to obtain conversion sequence data of the target associated data.
[0265] Wherein, the first business network includes an input layer; the word segmentation attribute data includes M word segmentation attribute data, the M word segmentation attribute data include target word segmentation attribute data, and M is a positive integer;
[0266] Wherein, the embedding vector extraction module 11 is specifically used for:
[0267] Based on the data attribute to which the target word segmentation attribute data belongs, determining a mapping space corresponding to the target word segmentation attribute data; the mapping space includes a plurality of mapping values, each mapping value having a corresponding word segmentation attribute value interval;
[0268] The word segmentation attribute value interval to which the target word segmentation attribute data belongs is determined through the input layer, and the mapping value corresponding to the word segmentation attribute value interval to which the target word segmentation attribute data belongs is determined as the word segmentation conversion data corresponding to the target word segmentation attribute data.
[0269] The target extraction feature is obtained by performing feature extraction processing on the embedded vector feature by a feature extraction network in the first business network; the first business network belongs to a target identification network for risk identification of the target object;
[0270] The data processing device further includes: a first training module 16;
[0271] The first training module 16 is specifically used for:
[0272] Acquire a first training data set; the first sample data included in the first training data set is sample data without risk labels;
[0273] Obtain K training tasks for the first initial service network; K is a positive integer;
[0274] Processing the first sample data through the first initial business network to obtain training extraction features associated with each training task, and determining K loss information corresponding to the K training tasks based on the training extraction features; one training task corresponds to one loss information;
[0275] First target loss information is determined based on the K loss information, network parameters of the first initial service network are adjusted based on the first target loss information, and the first initial service network after the parameter adjustment is determined as the first service network.
[0276] The K training tasks include a word segmentation and masking recognition task;
[0277] Among them, the first training module 16 is specifically used for:
[0278] Performing sequence conversion processing on the first sample data through the first initial business network to obtain first sample conversion sequence data of the first sample data; the first sample conversion sequence data includes sample word segmentation conversion data at multiple sequence positions;
[0279] Randomly determine a first sequence position to be subjected to word segmentation shielding from a plurality of sequence positions corresponding to the first sample conversion sequence data;
[0280] Based on the word segmentation masking information, the sample word segmentation conversion data at the first sequence position in the first sample conversion sequence data is subjected to word segmentation masking processing to obtain first training sequence data;
[0281] Performing embedding vector feature extraction on the first training sequence data through the first initial service network to obtain a first training embedding vector feature of the first training sequence data, performing feature extraction processing on the first training embedding vector feature to obtain a first training extraction feature corresponding to the first training embedding vector feature; the first training extraction feature belongs to the training extraction feature;
[0282] By using a recognition component associated with the word segmentation mask recognition task, the first training extracted features are processed by masked word segmentation recognition to obtain predicted word segmentation conversion data corresponding to the word segmentation mask information in the first training sequence data;
[0283] Based on the sample word segmentation conversion data at the first sequence position in the first sample conversion sequence data and the predicted word segmentation conversion data, loss information corresponding to the word segmentation mask recognition task is determined.
[0284] The K training tasks include a word segmentation replacement recognition task;
[0285] Among them, the first training module 16 is specifically used for:
[0286] Performing sequence conversion processing on the first sample data through the first initial business network to obtain first sample conversion sequence data of the first sample data; the first sample conversion sequence data includes sample word segmentation conversion data at multiple sequence positions;
[0287] Randomly determine a second sequence position to be replaced with a word segmentation from a plurality of sequence positions corresponding to the first sample conversion sequence data;
[0288] Based on the random word segmentation information, the sample word segmentation conversion data at the second sequence position in the first sample conversion sequence data is subjected to word segmentation replacement processing to obtain second training sequence data; the random word segmentation information is different from the sample word segmentation conversion data at the second sequence position;
[0289] Performing embedding vector feature extraction on the second training sequence data through the first initial service network to obtain a second training embedding vector feature of the second training sequence data, performing feature extraction processing on the second training embedding vector feature to obtain a second training extraction feature corresponding to the second training embedding vector feature; the second training extraction feature belongs to the training extraction feature;
[0290] Performing replacement position recognition processing on the second training extracted features through a recognition component associated with the word segmentation replacement recognition task to obtain word segmentation replacement prediction results corresponding to the multiple sequence positions;
[0291] Based on the word segmentation replacement prediction results corresponding to the second sequence position and the multiple sequence positions, loss information corresponding to the word segmentation replacement recognition task is determined.
[0292] Wherein, the K training tasks include a noise sequence task;
[0293] The first training module 16 is specifically used for:
[0294] Randomly adding noise information to the word segmentation attribute data of the first sample data to obtain noise sample data, and performing sequence conversion processing on the noise sample data to obtain noise conversion sequence data of the noise sample data;
[0295] Performing sequence conversion processing on the first sample data to obtain first sample conversion sequence data of the first sample data;
[0296] Performing embedding vector feature extraction on the noise conversion sequence data through the first initial business network to obtain a third training embedding vector feature of the noise conversion sequence data, performing feature extraction processing on the third training embedding vector feature to obtain a third training extraction feature corresponding to the third training embedding vector feature; the third training extraction feature belongs to the training extraction feature;
[0297] Performing embedding vector feature extraction on the first sample conversion sequence data through the first initial service network to obtain a fourth training embedding vector feature of the first sample conversion sequence data, performing feature extraction processing on the fourth training embedding vector feature to obtain a fourth training extraction feature corresponding to the fourth training embedding vector feature; the fourth training extraction feature belongs to the training extraction feature;
[0298] Based on the third training extracted features and the fourth training extracted features, loss information corresponding to the noise sequence task is determined.
[0299] Wherein, the K training tasks include pseudo label tasks;
[0300] The first training module 16 is specifically used for:
[0301] Performing sequence conversion processing on the first sample data to obtain first sample conversion sequence data of the first sample data;
[0302] Performing embedding vector feature extraction on the first sample conversion sequence data through the first initial service network to obtain a fourth training embedding vector feature of the first sample conversion sequence data, performing feature extraction processing on the fourth training embedding vector feature to obtain a fourth training extraction feature corresponding to the fourth training embedding vector feature; the fourth training extraction feature belongs to the training extraction feature;
[0303] Based on the recognition component associated with the pseudo label task, performing risk classification processing on the fourth training extracted feature to obtain a training risk classification probability corresponding to the first sample data;
[0304] Acquire a library model associated with the first initial business network, perform risk classification processing on the first sample data based on the library model, and obtain a pseudo-label risk estimation probability corresponding to the first sample data;
[0305] Based on the training risk classification probability and the pseudo label risk estimation probability, loss information corresponding to the pseudo label task is determined.
[0306] The target extraction feature is obtained by performing feature extraction processing on the embedded vector feature by a feature extraction network in the first business network; the first business network belongs to a target recognition network for risk identification of the target object; the target recognition network also includes a second business network cascaded after the feature extraction network;
[0307] Among them, the risk estimation module 13 is specifically used for:
[0308] Determining a second business network from the target identification network; the second business network includes an expert network corresponding to each business scenario;
[0309] Through the expert network corresponding to each business scenario, the target extraction features are processed for risk estimation to obtain the risk estimation probability corresponding to each business scenario; an expert network is used to determine the risk estimation probability corresponding to a business scenario.
[0310] Wherein, the data processing device further includes a second training module 17;
[0311] The second training module 17 is specifically used for:
[0312] Acquire a second training data set; the second sample data included in the second training data set is associated with a scene label; the scene label includes N scene label values corresponding to the N business scenarios, one business scenario corresponds to one scene label value, and the N scene label values are determined based on the business scenario to which the second sample data belongs;
[0313] Obtain a trained first business network, perform vector feature extraction on the second sample data through the first business network to obtain a fifth training embedding vector feature of the second sample data, perform feature extraction processing on the fifth training embedding vector feature, and obtain a fifth training extraction feature corresponding to the fifth training embedding vector feature;
[0314] Obtaining N initial expert networks corresponding to N business scenarios, and performing scene recognition on the fifth training extracted features based on the N networks respectively, to obtain N scene recognition probabilities corresponding to the N initial expert networks; one business scenario corresponds to one initial expert network, and one initial expert network is used to determine one scene recognition probability;
[0315] Based on the N scene recognition probabilities and the N scene label values, second target loss information is determined; based on the second target loss information, network parameters of the N initial expert networks are adjusted; the N initial expert networks after parameter adjustment are determined as the N expert networks; and the second business network is determined based on the N expert networks.
[0316] The embedded vector feature is obtained by extracting the embedded vector feature of the conversion sequence data by the embedding layer in the first business network; the conversion sequence data is obtained by the input layer in the first business network performing sequence conversion processing on the target associated data; the first business network belongs to a target identification network for risk identification of the target object; the target identification network also includes a third business network cascaded after the embedding layer;
[0317] The weight extraction module 14 is specifically used for:
[0318] Determining a third service network from the target identification network;
[0319] The embedded vector features are subjected to weight extraction processing through the third business network to obtain a weight influence value corresponding to each business scenario.
[0320] The target identification network further includes a second business network for determining the risk estimation probability corresponding to each business scenario;
[0321] The data processing device 1 further includes: a third training module 18;
[0322] The third training module 18 is specifically used for:
[0323] Acquire a third training data set; third sample data included in the third training data set is associated with a risk label;
[0324] Obtain a trained first business network, perform vector feature extraction on the third sample data through the first business network to obtain a sixth training embedding vector feature of the third sample data, perform feature extraction processing on the sixth training embedding vector feature, and obtain a sixth training extraction feature corresponding to the sixth training embedding vector feature;
[0325] Obtain a trained second business network, and perform risk estimation processing on the sixth training extracted features through N expert networks in the second business network to obtain a training risk estimation probability corresponding to each business scenario; one expert network is used to determine a training risk estimation probability corresponding to one business scenario;
[0326] Performing weight extraction processing on the sixth embedding vector feature through the third initial business network to obtain a training weight influence value corresponding to each business scenario;
[0327] Determining a training risk identification probability of a sample object associated with the third sample data based on a training risk estimation probability corresponding to each business scenario and a training weight influence value corresponding to each business scenario;
[0328] Based on the risk label and the training risk identification probability, third target loss information is determined; based on the third target loss information, network parameters of the third initial business network are adjusted; the third initial business network after the parameter adjustment is determined as the third business network; and based on the first business network, the second business network and the third business network, the target identification network is determined.
[0329] Wherein, the third service network includes a routing network and a gating network;
[0330] The weight extraction module 14 is specifically used for:
[0331] Performing routing feature extraction processing on the embedded vector feature through the routing network to obtain routing extraction features corresponding to the embedded vector feature;
[0332] The routing extraction features are weighted by the gating network to obtain a weight impact value corresponding to each business scenario.
[0333] Wherein, the N business scenarios include a target business scenario;
[0334] The data processing device 1 further includes a model migration module 19;
[0335] The model migration module 19 is specifically used for:
[0336] Acquire a target recognition network for performing risk identification on a target object, and acquire an initial migration model for performing distillation learning from the target recognition network;
[0337] Acquire a fourth training data set associated with the target business scenario; the fourth training data set includes fourth sample data;
[0338] Processing the fourth sample data through the target recognition network to obtain a risk recognition probability corresponding to the fourth sample data, and determining the risk recognition probability corresponding to the fourth sample data as a pseudo sample label for training the initial migration model;
[0339] Processing the fourth sample data by using the initial migration model to obtain a migration risk identification probability corresponding to the fourth sample data;
[0340] Based on the migration risk identification probability and the pseudo sample label, the model parameters of the initial migration model are adjusted, and the initial migration model after parameter adjustment is determined as the target migration model of the target identification network; the target migration model is used to identify risks of target-related data in the target business scenario.
[0341] Wherein, the fourth sample data includes scenario sample data generated in the target business scenario;
[0342] The model migration module 19 is specifically used for:
[0343] Obtaining scenario sample data under the target business scenario;
[0344] Determining sample data to be identified from a sample data set used for sample selection;
[0345] Performing similarity matching processing on the sample data to be identified and the scene sample data by enhancing the sample generation network to obtain a similarity value between the scene sample data and the sample data to be identified;
[0346] If the similarity value satisfies the sample enhancement condition, the sample data to be identified is determined as enhanced sample data associated with the target business scenario;
[0347] The enhanced sample data and the scenario sample data are determined as fourth sample data associated with the target business scenario.
[0348] Among them, the risk identification module 15 is specifically used for:
[0349] The risk estimation probability corresponding to each business scenario is weighted by the weight impact value corresponding to each business scenario to obtain a weighted risk estimation value corresponding to each business scenario; the weight mapping value corresponding to a business scenario is used to weight the risk estimation probability corresponding to the corresponding business scenario;
[0350] Based on the weighted risk estimation value corresponding to each business scenario, the risk identification probability of the target object is determined.
[0351] See also Fig.13 , Fig.13 Schematic diagram of the structure of a computer device provided in an embodiment of the present application. Fig.13 As shown, the computer device 1000 may include: a processor 1001, a network interface 1004 and a memory 1005. In addition, the above-mentioned computer device 1000 may also include: a user interface 1003, and at least one communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. Among them, the user interface 1003 may include a display screen (Display), a keyboard (Keyboard), and the user interface 1003 may optionally include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or it may be a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 1005 may optionally also be at least one storage device located away from the aforementioned processor 1001. As Fig.13 As shown, the memory 1005 as a computer-readable storage medium may include an operating system, a network communication module, a user interface module, and a device control application program.
[0352] In such Fig.13 In the computer device 1000 shown, the network interface 1004 can provide a network communication function; the user interface 1003 is mainly used to provide an input interface for the user; and the processor 1001 can be used to call the device control application stored in the memory 1005 to execute the description of the data processing method in any of the corresponding embodiments above, which will not be repeated here. In addition, the description of the beneficial effects of using the same method will not be repeated.
[0353] In addition, it should be pointed out here that: the embodiment of the present application also provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program executed by the data processing device 1 mentioned above, and the computer program includes program instructions. When the processor executes the program instructions, it can execute the description of the data processing method in the previous embodiment, so it will not be repeated here. In addition, the description of the beneficial effects of using the same method will not be repeated. For technical details not disclosed in the computer-readable storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application.
[0354] The computer-readable storage medium may be the data processing device provided in any of the aforementioned embodiments or the internal storage unit of the computer device, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device. Further, the computer-readable storage medium may also include both the internal storage unit of the computer device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.
[0355] In addition, it should be noted here that: the embodiment of the present application also provides a computer program product or a computer program, which includes a computer instruction, and the computer instruction is stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method provided by any of the corresponding embodiments in the foregoing text. In addition, the description of the beneficial effects of adopting the same method will not be repeated. For technical details not disclosed in the computer program product or computer program embodiment involved in this application, please refer to the description of the method embodiment of this application.
[0356] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories), or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.
[0357] The terms "first", "second", etc. in the description, claims, and drawings of the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, device, product, or equipment that includes a series of steps or units is not limited to the listed steps or modules, but optionally includes steps or modules that are not listed, or optionally includes other step units inherent to these processes, methods, devices, products, or equipment.
[0358] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0359] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.
Claims
1. A data processing method, characterized in that: The method comprises: Extracting vector features from target-related data of the target object to obtain embedded vector features corresponding to the target-related data; Performing feature extraction processing on the embedded vector feature to obtain a target extraction feature corresponding to the embedded vector feature; Determine N business scenarios associated with the target-related data, and perform risk estimation processing on the target extraction features based on each business scenario to obtain the risk estimation probability corresponding to each business scenario; N is a positive integer; Performing weight extraction processing on the embedded vector features to obtain a weight influence value corresponding to each business scenario; Based on the risk estimation probability corresponding to each business scenario and the weighted impact value corresponding to each business scenario, the risk identification probability of the target object is determined.
2. The method according to claim 1, characterized in that The extracting vector features of the target-related data of the target object to obtain the embedded vector features corresponding to the target-related data includes: Performing sequence conversion processing on the target associated data to obtain conversion sequence data of the target associated data; Embedded vector features are extracted from the conversion sequence data to obtain embedded vector features of the conversion sequence data.
3. The method according to claim 2, characterized in that The performing sequence conversion processing on the target associated data to obtain conversion sequence data of the target associated data includes: When the target association data of the target object is obtained, a target identification network for risk identification of the target object is obtained; the target identification network includes a first business network; Determining data attributes associated with the target-related data through the first business network, and extracting word segmentation attribute data of the target-related data based on the data attributes; Performing word segmentation conversion on the word segmentation attribute data through the first business network to obtain word segmentation conversion data corresponding to the word segmentation attribute data; The word segmentation conversion data corresponding to the word segmentation attribute data is serialized through the first business network to obtain conversion sequence data of the target associated data.
4. The method according to claim 3, characterized in that The first business network includes an input layer; the word segmentation attribute data includes M word segmentation attribute data, the M word segmentation attribute data include target word segmentation attribute data, and M is a positive integer; The performing word segmentation conversion on the word segmentation attribute data through the first business network to obtain word segmentation conversion data corresponding to the word segmentation attribute data includes: Based on the data attribute to which the target word segmentation attribute data belongs, determining a mapping space corresponding to the target word segmentation attribute data; the mapping space includes a plurality of mapping values, each mapping value having a corresponding word segmentation attribute value interval; The word segmentation attribute value interval to which the target word segmentation attribute data belongs is determined through the input layer, and the mapping value corresponding to the word segmentation attribute value interval to which the target word segmentation attribute data belongs is determined as the word segmentation conversion data corresponding to the target word segmentation attribute data.
5. The method according to claim 1, characterized in that The target extraction feature is obtained by performing feature extraction processing on the embedded vector feature by a feature extraction network in the first business network; the first business network belongs to a target identification network for risk identification of the target object; The method further comprises: Acquire a first training data set; the first sample data included in the first training data set is sample data without risk labels; Obtain K training tasks for the first initial service network; K is a positive integer; Processing the first sample data through the first initial business network to obtain training extraction features associated with each training task, and determining K loss information corresponding to the K training tasks based on the training extraction features; one training task corresponds to one loss information; First target loss information is determined based on the K loss information, network parameters of the first initial service network are adjusted based on the first target loss information, and the first initial service network after the parameter adjustment is determined as the first service network.
6. The method according to claim 5, characterized in that The K training tasks include a word segmentation and masking recognition task; The performing data processing on the first sample data through the first initial business network to obtain a training extraction feature associated with each training task, and determining K pieces of loss information corresponding to the K training tasks based on the training extraction features, includes: Performing sequence conversion processing on the first sample data through the first initial business network to obtain first sample conversion sequence data of the first sample data; the first sample conversion sequence data includes sample word segmentation conversion data at multiple sequence positions; Randomly determine a first sequence position to be subjected to word segmentation shielding from a plurality of sequence positions corresponding to the first sample conversion sequence data; Based on the word segmentation masking information, the sample word segmentation conversion data at the first sequence position in the first sample conversion sequence data is subjected to word segmentation masking processing to obtain first training sequence data; Performing embedding vector feature extraction on the first training sequence data through the first initial service network to obtain a first training embedding vector feature of the first training sequence data, performing feature extraction processing on the first training embedding vector feature to obtain a first training extraction feature corresponding to the first training embedding vector feature; the first training extraction feature belongs to the training extraction feature; By using a recognition component associated with the word segmentation mask recognition task, the first training extracted features are processed by masked word segmentation recognition to obtain predicted word segmentation conversion data corresponding to the word segmentation mask information in the first training sequence data; Based on the sample word segmentation conversion data at the first sequence position in the first sample conversion sequence data and the predicted word segmentation conversion data, loss information corresponding to the word segmentation mask recognition task is determined.
7. The method according to claim 5, characterized in that The K training tasks include a word segmentation replacement recognition task; The performing data processing on the first sample data through the first initial business network to obtain a training extraction feature associated with each training task, and determining K pieces of loss information corresponding to the K training tasks based on the training extraction features, includes: Performing sequence conversion processing on the first sample data through the first initial business network to obtain first sample conversion sequence data of the first sample data; the first sample conversion sequence data includes sample word segmentation conversion data at multiple sequence positions; Randomly determine a second sequence position to be replaced with a word segmentation from a plurality of sequence positions corresponding to the first sample conversion sequence data; Based on the random word segmentation information, the sample word segmentation conversion data at the second sequence position in the first sample conversion sequence data is subjected to word segmentation replacement processing to obtain second training sequence data; the random word segmentation information is different from the sample word segmentation conversion data at the second sequence position; Performing embedding vector feature extraction on the second training sequence data through the first initial service network to obtain a second training embedding vector feature of the second training sequence data, performing feature extraction processing on the second training embedding vector feature to obtain a second training extraction feature corresponding to the second training embedding vector feature; the second training extraction feature belongs to the training extraction feature; Performing replacement position recognition processing on the second training extracted features through a recognition component associated with the word segmentation replacement recognition task to obtain word segmentation replacement prediction results corresponding to the multiple sequence positions; Based on the word segmentation replacement prediction results corresponding to the second sequence position and the multiple sequence positions, loss information corresponding to the word segmentation replacement recognition task is determined.
8. The method according to claim 5, characterized in that The K training tasks include a noise sequence task; The performing data processing on the first sample data through the first initial business network to obtain a training extraction feature associated with each training task, and determining K pieces of loss information corresponding to the K training tasks based on the training extraction features, includes: Randomly adding noise information to the word segmentation attribute data of the first sample data to obtain noise sample data, and performing sequence conversion processing on the noise sample data to obtain noise conversion sequence data of the noise sample data; Performing sequence conversion processing on the first sample data to obtain first sample conversion sequence data of the first sample data; Performing embedding vector feature extraction on the noise conversion sequence data through the first initial business network to obtain a third training embedding vector feature of the noise conversion sequence data, performing feature extraction processing on the third training embedding vector feature to obtain a third training extraction feature corresponding to the third training embedding vector feature; the third training extraction feature belongs to the training extraction feature; Performing embedding vector feature extraction on the first sample conversion sequence data through the first initial service network to obtain a fourth training embedding vector feature of the first sample conversion sequence data, performing feature extraction processing on the fourth training embedding vector feature to obtain a fourth training extraction feature corresponding to the fourth training embedding vector feature; the fourth training extraction feature belongs to the training extraction feature; Based on the third training extracted features and the fourth training extracted features, loss information corresponding to the noise sequence task is determined.
9. The method according to claim 5, characterized in that The K training tasks include pseudo label tasks; The performing data processing on the first sample data through the first initial business network to obtain a training extraction feature associated with each training task, and determining K pieces of loss information corresponding to the K training tasks based on the training extraction features, includes: Performing sequence conversion processing on the first sample data to obtain first sample conversion sequence data of the first sample data; Performing embedding vector feature extraction on the first sample conversion sequence data through the first initial service network to obtain a fourth training embedding vector feature of the first sample conversion sequence data, performing feature extraction processing on the fourth training embedding vector feature to obtain a fourth training extraction feature corresponding to the fourth training embedding vector feature; the fourth training extraction feature belongs to the training extraction feature; Based on the recognition component associated with the pseudo label task, performing risk classification processing on the fourth training extracted feature to obtain a training risk classification probability corresponding to the first sample data; Acquire a library model associated with the first initial business network, perform risk classification processing on the first sample data based on the library model, and obtain a pseudo-label risk estimation probability corresponding to the first sample data; Based on the training risk classification probability and the pseudo label risk estimation probability, loss information corresponding to the pseudo label task is determined.
10. The method according to claim 1, characterized in that The target extraction feature is obtained by performing feature extraction processing on the embedded vector feature by a feature extraction network in the first business network; the first business network belongs to a target recognition network for risk identification of the target object; the target recognition network also includes a second business network cascaded after the feature extraction network; The step of performing risk estimation processing on the target extraction features based on each business scenario to obtain the risk estimation probability corresponding to each business scenario includes: Determining a second business network from the target identification network; the second business network includes an expert network corresponding to each business scenario; Through the expert network corresponding to each business scenario, the target extraction features are processed for risk estimation to obtain the risk estimation probability corresponding to each business scenario; an expert network is used to determine the risk estimation probability corresponding to a business scenario.
11. The method according to claim 10, characterized in that The method further comprises: Acquire a second training data set; the second sample data included in the second training data set is associated with a scene label; the scene label includes N scene label values corresponding to the N business scenarios, one business scenario corresponds to one scene label value, and the N scene label values are determined based on the business scenario to which the second sample data belongs; Obtain a trained first business network, perform vector feature extraction on the second sample data through the first business network to obtain a fifth training embedding vector feature of the second sample data, perform feature extraction processing on the fifth training embedding vector feature, and obtain a fifth training extraction feature corresponding to the fifth training embedding vector feature; Obtaining N initial expert networks corresponding to the N business scenarios, and performing scene recognition on the fifth training extracted features based on the N networks respectively, to obtain N scene recognition probabilities corresponding to the N initial expert networks; one business scenario corresponds to one initial expert network, and one initial expert network is used to determine one scene recognition probability; Based on the N scene recognition probabilities and the N scene label values, second target loss information is determined; based on the second target loss information, network parameters of the N initial expert networks are adjusted; the N initial expert networks after parameter adjustment are determined as the N expert networks; and the second business network is determined based on the N expert networks.
12. The method according to claim 1, characterized in that The embedding vector feature is obtained by extracting the embedding vector feature of the conversion sequence data by the embedding layer in the first business network; the conversion sequence data is obtained by the input layer in the first business network performing sequence conversion processing on the target association data; the first business network belongs to a target identification network for risk identification of the target object; the target identification network also includes a third business network cascaded after the embedding layer; The performing weight extraction processing on the embedded vector features to obtain the weight impact value corresponding to each business scenario includes: Determining a third service network from the target identification network; The embedded vector features are subjected to weight extraction processing through the third business network to obtain a weight influence value corresponding to each business scenario.
13. The method according to claim 12, characterized in that The target identification network also includes a second business network for determining the risk estimation probability corresponding to each business scenario; The method further comprises: Acquire a third training data set; third sample data included in the third training data set is associated with a risk label; Acquire a trained first business network, perform vector feature extraction on the third sample data through the first business network to obtain a sixth training embedded vector feature of the third sample data, perform feature extraction processing on the sixth training embedded vector feature, and obtain a sixth training extracted feature corresponding to the sixth training embedded vector feature; Obtain a trained second business network, and perform risk estimation processing on the sixth training extracted features through N expert networks in the second business network to obtain a training risk estimation probability corresponding to each business scenario; one expert network is used to determine a training risk estimation probability corresponding to one business scenario; Performing weight extraction processing on the sixth embedding vector feature through the third initial business network to obtain a training weight influence value corresponding to each business scenario; Determining a training risk identification probability of a sample object associated with the third sample data based on a training risk estimation probability corresponding to each business scenario and a training weight influence value corresponding to each business scenario; Based on the risk label and the training risk identification probability, third target loss information is determined; based on the third target loss information, network parameters of the third initial business network are adjusted; the third initial business network after the parameter adjustment is determined as the third business network; and based on the first business network, the second business network and the third business network, the target identification network is determined.
14. The method according to claim 12, characterized in that The third service network includes a routing network and a gating network; The performing weight extraction processing on the embedded vector features through the third business network to obtain the weight influence value corresponding to each business scenario includes: Performing routing feature extraction processing on the embedded vector feature through the routing network to obtain routing extraction features corresponding to the embedded vector feature; The routing extraction features are weighted by the gating network to obtain a weight impact value corresponding to each business scenario.
15. The method according to claim 1, characterized in that The N business scenarios include a target business scenario; The method further comprises: Acquire a target recognition network for performing risk identification on a target object, and acquire an initial migration model for performing distillation learning from the target recognition network; Acquire a fourth training data set associated with the target business scenario; the fourth training data set includes fourth sample data; Processing the fourth sample data through the target recognition network to obtain a risk recognition probability corresponding to the fourth sample data, and determining the risk recognition probability corresponding to the fourth sample data as a pseudo sample label for training the initial migration model; Processing the fourth sample data by using the initial migration model to obtain a migration risk identification probability corresponding to the fourth sample data; Based on the migration risk identification probability and the pseudo sample label, the model parameters of the initial migration model are adjusted, and the initial migration model after parameter adjustment is determined as the target migration model of the target identification network; the target migration model is used to identify risks of target-related data in the target business scenario.
16. The method according to claim 15, characterized in that The fourth sample data includes scenario sample data generated in the target business scenario; The acquiring a fourth training data set associated with the target business scenario includes: Obtaining scenario sample data under the target business scenario; Determining sample data to be identified from a sample data set used for sample selection; Performing similarity matching processing on the sample data to be identified and the scene sample data by enhancing the sample generation network to obtain a similarity value between the scene sample data and the sample data to be identified; If the similarity value satisfies the sample enhancement condition, the sample data to be identified is determined as enhanced sample data associated with the target business scenario; The enhanced sample data and the scenario sample data are determined as fourth sample data associated with the target business scenario.
17. The method according to claim 1, characterized in that The determining the risk identification probability of the target object based on the risk estimation probability corresponding to each business scenario and the weight impact value corresponding to each business scenario includes: The risk estimation probability corresponding to each business scenario is weighted by the weight impact value corresponding to each business scenario to obtain a weighted risk estimation value corresponding to each business scenario; the weight mapping value corresponding to a business scenario is used to weight the risk estimation probability corresponding to the corresponding business scenario; Based on the weighted risk estimation value corresponding to each business scenario, the risk identification probability of the target object is determined.
18. A data processing device, characterized in that: The device comprises: An embedded vector extraction module is used to extract vector features from target-related data of a target object to obtain embedded vector features corresponding to the target-related data; A feature extraction module, used to perform feature extraction processing on the embedded vector feature to obtain a target extraction feature corresponding to the embedded vector feature; A risk estimation module, used to determine N business scenarios associated with the target-related data, and perform risk estimation processing on the target extraction features based on each business scenario to obtain the risk estimation probability corresponding to each business scenario; N is a positive integer; A weight extraction module, used to perform weight extraction processing on the embedded vector features to obtain a weight influence value corresponding to each business scenario; The risk identification module is used to determine the risk identification probability of the target object based on the risk estimation probability corresponding to each business scenario and the weighted impact value corresponding to each business scenario.
19. A computer device, characterized in that: including memory and processor; The memory is connected to the processor, the memory is used to store a computer program, and the processor is used to call the computer program so that the computer device executes the method according to any one of claims 1 to 17.
20. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded and executed by a processor, so that a computer device having the processor executes the method according to any one of claims 1 to 17.
Citation Information
Patent Citations
Operation optimizing method capable of coordinating operation risk and wind energy consumption of power system
CN104600747A
Power system risk assessment method based on new energy multi-scene risk feature guidance
CN111612289A
