Business data processing method, device, electronic device and readable storage medium

Abstract

The present application provides a business data processing method, device, electronic device and readable storage medium, which relates to the field of data processing. In the method, firstly, retrieved asset data is obtained; in response to the selection operation of a preset target labeling model, the retrieved asset data is labeled using the target labeling model to obtain a labeling result; according to the labeling result, a first recommendation is made using a preset recommendation strategy to obtain relevant recommendation results, and the relevant recommendation results include recommended data and labels corresponding to the recommended data; a preset asset labeling list is obtained, and the asset labeling list includes multiple asset data to be labeled and labeling labels corresponding to each asset data to be labeled; according to the recommended data, the labels corresponding to the recommended data, the asset data to be labeled and the labeling labels, a matching result is obtained; according to the matching result and the retrieved asset data, a second recommendation is made to obtain recommended asset data, which can recommend accurate asset data to users.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a business data processing method, device, electronic device and readable storage medium. Background Art

[0002] The asset data of an enterprise is data that the enterprise legally owns or controls. It is usually divided into different categories according to different properties. For example, it can be divided into different asset data such as products, markets, orders, costs, revenues, services, channels, etc. according to attributes. Asset data inventory and intelligent labeling can directly or indirectly bring economic benefits and provide new development opportunities for enterprises. Intelligent recommendation is an application scenario of asset data inventory and intelligent labeling of enterprises, which recommends appropriate asset data to users and realizes the value of enterprise assets. Intelligent recommendation recommends the recommended content that users want through user input query data. However, when users query data, they are not very clear about the search results and usually enter some related content. However, these related contents are relatively vague and cannot recommend accurate asset data to users. Summary of the invention

[0003] The present application provides a business data processing method, device, electronic device and readable storage medium, which can recommend accurate asset data to users.

[0004] The technical solution of the embodiment of the present application is as follows:

[0005] In a first aspect, an embodiment of the present application provides a method for processing business data, the method comprising:

[0006] Get retrieve asset data;

[0007] In response to a selection operation of a preset target marking model, the retrieved asset data is marked using the target marking model to obtain a marking result;

[0008] According to the labeling result, a first recommendation is performed using a preset recommendation strategy to obtain a relevant recommendation result, wherein the relevant recommendation result includes the recommended data and a label corresponding to the recommended data;

[0009] Obtaining a preset asset marking list, wherein the asset marking list includes a plurality of asset data to be marked and a marking label corresponding to each of the asset data to be marked;

[0010] Obtaining a matching result according to the recommended data, the tag corresponding to the recommended data, the asset data to be marked, and the marked tag;

[0011] A second recommendation is made based on the matching result and the retrieved asset data to obtain recommended asset data.

[0012] In the above technical scheme, firstly, retrieved asset data is obtained, and the retrieved asset data is the search content input by the user. The retrieved asset data is obtained to provide data support for subsequently recommending accurate asset data to the user; in response to the selection operation of the preset target labeling model, the retrieved asset data is labeled using the target labeling model to obtain the labeling result. The retrieved asset data is first labeled, and the obtained labeling result indicates the category to which it belongs, which is conducive to accurate recommendation; according to the labeling result, a first recommendation is made using a preset recommendation strategy to obtain relevant recommendation results, and the relevant recommendation results include recommended data and labels corresponding to the recommended data. The recommendation strategy is first used to make a rough recommendation, which is conducive to improving the accuracy of the recommendation; a preset asset labeling list is obtained, and the asset labeling list includes multiple asset data to be labeled and labeling labels corresponding to each asset data to be labeled, which provides data support for subsequent accurate recommendation; according to the recommended data, the labels corresponding to the recommended data, the asset data to be labeled and the labeling labels, a matching result is obtained; a second recommendation is made based on the matching result and the retrieved asset data to obtain recommended asset data. By making the second recommendation, relatively accurate recommended asset data can be obtained.

[0013] In some embodiments of the present application, before obtaining the preset asset tagging list, the method further includes:

[0014] Acquire multiple asset data to be marked, and pre-process each of the asset data to be marked to obtain multiple pre-processed asset data;

[0015] Using the target labeling model to label each of the pre-processed asset data to obtain a labeling label;

[0016] The asset marking list is obtained according to each of the asset data to be marked and the marking tags corresponding to each of the asset data to be marked.

[0017] In the above technical solution, by preprocessing the asset data to be tagged, that is, performing data screening and data cleaning, not only can the amount of calculation be reduced, but also the accuracy of tagging can be improved. Then, the preprocessed asset data is tagged using the target tagging model to obtain the tagging label, and the asset data to be tagged and the tagging label form an asset tagging list, which is conducive to subsequent matching according to the asset tagging list to recommend accurate asset data.

[0018] In some embodiments of the present application, before labeling each of the pre-processed asset data using the target labeling model to obtain a labeling label, the method further includes:

[0019] Acquire an asset sample data set, wherein the asset sample data set includes a plurality of asset sample data and a label corresponding to each of the asset sample data;

[0020] Preprocessing each of the asset sample data to obtain a plurality of preprocessed sample data;

[0021] Performing a first sample balancing process on each of the preprocessed sample data to obtain a training data set;

[0022] Performing a second sample balancing process on each of the preprocessed sample data to obtain a verification data set;

[0023] Using the training data set to adjust the parameters of the preset first initial labeling model to obtain a second initial labeling model;

[0024] The second initial marking model is verified using the verification data set, and when the accuracy is greater than a preset accuracy threshold, the target marking model is obtained.

[0025] In the above technical solution, the generalization of the model can be improved by using the asset sample data set and preprocessing the asset sample data, that is, data screening and data cleaning. Then, the sample is balanced to obtain the training set and the verification set. The training set is used for parameter adjustment, and the verification set is used for verification to obtain the target labeling model, which has good generalization and robustness.

[0026] In some embodiments of the present application, performing a first sample balancing process on each of the preprocessed sample data to obtain a training data set includes:

[0027] Deduplication processing is performed on the preprocessed sample data according to the category, the Chinese name of the field, and the English name of the field to obtain first deduplication sample data;

[0028] Selecting N categories with the largest number from the first deduplicated sample data to obtain selected sample data, where N is an integer greater than or equal to 2;

[0029] Deduplication of the selected sample data is performed according to the category and the Chinese name of the field to obtain second deduplication sample data;

[0030] Removing the categories whose number does not exceed a preset number in the second deduplication sample data to obtain the training data set;

[0031] The performing second sample balancing processing on each of the preprocessed sample data to obtain a verification data set includes:

[0032] The categories whose number in the preprocessed sample data does not exceed a preset number are removed to obtain the verification data set.

[0033] In the above technical solution, duplicates are first removed according to category, field Chinese name and field English name, and then according to category and field Chinese name, to achieve effective deduplication and avoid duplicate records. By removing categories whose number does not exceed the preset number in the data, the impact of invalid data on the accuracy of the model is reduced.

[0034] In some embodiments of the present application, before labeling the retrieved asset data using the target labeling model to obtain a labeling result, the method further includes:

[0035] Performing vector processing on the retrieved asset data to obtain an asset data vector;

[0036] Performing data dimensionality upgrading processing on the retrieved asset data to obtain a multi-dimensional data vector;

[0037] The asset data vector and the multi-dimensional data vector are fused and then input into the target marking model.

[0038] In the above technical solution, by performing data dimensionality upgrading, the associated data of the retrieved asset data can be increased to clarify the recommendation target, and then data fusion is performed to increase the data representation ability, which is conducive to improving the accuracy of the model output.

[0039] In some embodiments of the present application, obtaining a matching result according to the recommended data, the tag corresponding to the recommended data, the asset data to be marked, and the marked tag includes:

[0040] The tags corresponding to the recommended data and the marked tags are classified, and the similarity between the recommended data and the asset data to be marked is calculated to filter out data with consistent tag categories and similarity less than a preset threshold to obtain the matching result.

[0041] In the above technical solution, accurate recommendations are made based on categories and similarities to obtain matching results, which is conducive to subsequently recommending more accurate asset data to users based on the matching results.

[0042] In some embodiments of the present application, the recommendation strategy is at least one of quantity restriction, matching degree restriction, and combination restriction.

[0043] In the above technical solutions, quantity restriction, matching degree restriction and combination restriction have different advantages. Users can choose corresponding strategy recommendations according to their needs so that the recommended content meets user needs.

[0044] In a second aspect, an embodiment of the present application provides a service data processing device, the device comprising:

[0045] A first data acquisition module, used for acquiring and retrieving asset data;

[0046] A marking module, used for marking the retrieved asset data using the target marking model in response to a selection operation of a preset target marking model to obtain a marking result;

[0047] A first recommendation module, configured to make a first recommendation based on the labeling result and using a preset recommendation strategy to obtain a relevant recommendation result, wherein the relevant recommendation result includes the recommended data and a label corresponding to the recommended data;

[0048] A second data acquisition module is used to acquire a preset asset marking list, wherein the asset marking list includes a plurality of asset data to be marked and a marking label corresponding to each of the asset data to be marked;

[0049] A matching module, used for obtaining a matching result according to the recommended data, the tag corresponding to the recommended data, the asset data to be marked and the marked tag;

[0050] The second recommendation module is used to make a second recommendation based on the matching result and the retrieved asset data to obtain recommended asset data.

[0051] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor, a memory, a user interface, and a network interface, wherein the memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes any one of the methods provided in the first aspect above.

[0052] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, and when the instructions are executed, any one of the methods provided in the first aspect is executed.

[0053] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0054] 1. The invention adopts the method of obtaining the retrieved asset data, and using the target labeling model to label the retrieved asset data, making the first recommendation based on the labeling results, obtaining the relevant recommendation results, thereby screening out the asset data similar to the retrieved asset data, and then obtaining the matching results based on the relevant recommendation results and the asset labeling list, and making the second recommendation based on the matching results and the retrieved asset data to obtain the technical means of recommended asset data. Therefore, the problem that the input related content in the related technology is relatively vague and cannot recommend accurate asset data to users is effectively solved. The retrieved asset data is labeled by the target labeling model, and the first recommendation is made based on the labeling results, and the asset data similar to the retrieved asset data is screened out, and then the second recommendation is made based on the asset labeling list. Through multi-layer recommendation, accurate asset data can be recommended to users.

[0055] 2. Using data preprocessing and sample balancing to process data can improve the generalization and robustness of the model.

[0056] 3. Vector processing and data dimensionality upgrading before inputting the data into the model can improve the accuracy of the training model. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 This is one of the flowcharts of a business data processing method provided by an embodiment of the present application;

[0058] Figure 2 This is the second flow chart of the business data processing method provided by an embodiment of the present application;

[0059] Figure 3 This is a flowchart of a business data processing method provided by an embodiment of the present application;

[0060] Figure 4 This is a fourth flow chart of a business data processing method provided by an embodiment of the present application;

[0061] Figure 5 This is a schematic diagram of label division of a business data processing method provided by an embodiment of the present application;

[0062] Figure 6 yes Figure 4 A schematic diagram of a sub-step flow chart of step S930;

[0063] Figure 7 This is a sample balancing schematic diagram of a business data processing method provided by an embodiment of the present application;

[0064] Figure 8 This is a schematic diagram of the overall process of a business data processing method provided by an embodiment of the present application;

[0065] Fig. 9It is a structural diagram of a business data processing device provided by an embodiment of the present application;

[0066] Fig.10 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0067] In order to enable technicians in this field to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0068] In the description of the embodiments of the present application, words such as "for example" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "for example" or "for example" is intended to present related concepts in a specific way.

[0069] In the description of the embodiments of the present application, the meaning of the term "multiple" refers to two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprise", "have" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0070] The embodiment of the present application provides a business data processing method, device, electronic device and readable storage medium. The business data processing method first obtains the retrieved asset data, the retrieved asset data is the search content input by the user, and the retrieved asset data is obtained to provide data support for subsequently recommending accurate asset data to the user; in response to the selection operation of the preset target labeling model, the retrieved asset data is labeled using the target labeling model to obtain the labeling result, the retrieved asset data is first labeled, and the obtained labeling result indicates the category to which it belongs, which is conducive to accurate recommendation; according to the labeling result, a first recommendation is made using a preset recommendation strategy to obtain relevant recommendation results, the relevant recommendation results include recommended data and labels corresponding to the recommended data, and a rough recommendation is first made using the recommendation strategy, which is conducive to improving the accuracy of the recommendation; a preset asset labeling list is obtained, the asset labeling list includes multiple asset data to be labeled and labeling labels corresponding to each asset data to be labeled, and provides data support for subsequent accurate recommendation; according to the recommended data, the labels corresponding to the recommended data, the asset data to be labeled and the labeling labels, a matching result is obtained; according to the matching result and the retrieved asset data, a second recommendation is made to obtain recommended asset data, and by making the second recommendation, relatively accurate recommended asset data can be obtained. Compared with the prior art in which the input associated content is relatively vague and thus makes it impossible to recommend accurate asset data to users, the embodiment of the present application labels the retrieved asset data through a target labeling model, makes a first recommendation based on the labeling results, screens out asset data similar to the retrieved asset data, and then makes a second recommendation based on the asset labeling list. Through multi-layer recommendations, accurate asset data can be recommended to users.

[0071] It should be noted that the business data processing method is applied to the fields of banking, commercial asset recommendation, etc., and can also be applied to other asset recommendation fields. First, through asset inventory tagging management, the sorted asset tagging list is used for intelligent recommendation, which can recommend accurate asset data to users.

[0072] The following, in conjunction with the accompanying drawings, further illustrates the technical solution provided in the embodiment of the present application by taking the banking industry as an example.

[0073] Reference Figure 1 , Figure 1 The flowchart of the business data processing method provided by the embodiment of the present application is shown in FIG. The business data processing method is applied to a business data processing device, and the business data processing method is executed by a processor in an electronic device or a readable storage medium. The business data processing method includes steps S100, S200, S300, S400, S500, and S600.

[0074] Step S100, obtaining and retrieving asset data.

[0075] In one embodiment, the retrieved asset data is a brief summary or specific description of the data that the user wants to obtain. The retrieved asset data may include loan policy search, banking business financial management search, deposit policy search, etc. The user may enter the retrieved asset data in a preset webpage or APP and obtain the retrieved asset data through a preset interface, which is conducive to subsequent recommendation based on the retrieved asset data and providing data support. Among them, the preset interface includes the input() function.

[0076] Step S200, in response to the selection operation of the preset target labeling model, the retrieved asset data is labeled using the target labeling model to obtain a labeling result.

[0077] In one embodiment, the user selects a target labeling model on a web page. Different models are recommended in different ways. The selected target labeling model can make the recommended data more accurate. In response to the selection operation of the preset target labeling model, the retrieved asset data is labeled using the target labeling model to obtain a labeling result. The labeling result is a specific subcategory obtained after the data is divided according to attributes or categories. The category division can be based on industry classification standards. Since the target labeling model is pre-trained, the target labeling model can accurately mark the category of the retrieved asset data, which is conducive to subsequent recommendations based on the labeling results. The target labeling model can be a BERT model, a variant of the BERT model, or other deep learning models, as long as it can meet the accurate labeling requirements, which will not be elaborated here.

[0078] like Figure 2 As shown, before the target labeling model is used to label the retrieved asset data and the labeling result is obtained, the business data processing method further includes but is not limited to the following steps:

[0079] Step S710, performing vector processing on the retrieved asset data to obtain an asset data vector.

[0080] In one embodiment, before the retrieved asset data is input into the target marking model, the retrieved asset data is first vectorized, specifically: the retrieved asset data is vectorized using a preset data representation algorithm to obtain an asset data vector. By data vectorization, the retrieved asset data is converted into a data type that can be processed by the target marking model, which is conducive to improving calculation efficiency.

[0081] Step S720, perform data dimensionality upgrading processing on the retrieved asset data to obtain a multi-dimensional data vector.

[0082] In one embodiment, data dimensioning is to expand related data for the data, such as Figure 8As shown in the figure, the data dimension upgrade of user data can be expanded to roles, departments, concerned businesses, historical search records, jurisdiction systems, etc. The data dimension upgrade of the retrieved asset data can be processed according to the pre-established knowledge graph to obtain a multi-dimensional data vector. By performing data dimension upgrade processing, the search range of the retrieved asset data can be increased, which is conducive to improving the accuracy of recommendations. Among them, the knowledge graph can reflect the association relationship between each data, so as to use the knowledge graph for rapid data dimension upgrade. The knowledge graph is pre-built according to the specific business scenario, and its construction process is similar to that of other knowledge graphs, which will not be repeated here.

[0083] Step S730: The asset data vector and the multi-dimensional data vector are fused and input into the target labeling model.

[0084] In one embodiment, the asset data vector and the multi-dimensional data vector are fused. The asset data vector and the multi-dimensional data vector can be used for feature connection and then input into the target labeling model. The target labeling model performs labeling and can output a more accurate labeling category, thereby improving the accuracy of the recommendation.

[0085] Step S300, based on the labeling result, a first recommendation is performed using a preset recommendation strategy to obtain a relevant recommendation result, which includes the recommended data and a label corresponding to the recommended data.

[0086] In one embodiment, the first recommendation is a rough recommendation based on the retrieved asset data. Data of different categories from the tagged labels are first removed, and only content related to the retrieved asset data is recommended. According to the tagged results obtained in step S200, the first recommendation is made using a preset recommendation strategy to obtain relevant recommendation results, which is conducive to subsequent advancement based on the relevant recommendation results, thereby recommending accurate asset data to users. The relevant recommendation results include the recommended data and the labels corresponding to the recommended data. The labels corresponding to the recommended data belong to the same category as the tagged results. The recommended data has a strong correlation with the retrieved asset data, which is represented by the matching degree. The higher the matching degree, the stronger the correlation.

[0087] In another embodiment, the recommendation strategy is at least one of quantity restriction, matching degree restriction, and combination restriction. When the recommendation strategy is quantity restriction, the first a data items are recommended according to the matching degree, and the recommendation strategy has a faster recommendation speed. Wherein, a is an integer greater than 0 and less than 100, and a can be 5 items, etc., which can be adjusted according to the situation, and will not be described in detail here. When the recommendation strategy is matching degree restriction, data with a matching degree greater than a preset association threshold is recommended, and the recommendation strategy has a faster recommendation speed and a higher recommendation accuracy. Wherein, the association threshold can be 70%, etc., which can be adjusted according to the situation, and will not be described in detail here. When the recommendation strategy is combination restriction, the matching degree is greater than the preset association threshold, and the first a data items are recommended according to the matching degree, and the recommendation strategy has a slower recommendation speed and the highest recommendation accuracy. Exemplarily, the default matching degree arrangement of a data item is 10, and the association threshold is 70%, then the different strategies are: the quantity restriction is the first 5 items arranged according to the matching degree, the matching degree restriction is the matching degree greater than 70%, and the combination restriction has a matching degree ≥ 70% and the first 5 items with the highest matching degree.

[0088] Step S400, obtaining a preset asset marking list, the asset marking list including a plurality of asset data to be marked and a marking label corresponding to each asset data to be marked.

[0089] In one embodiment, the preset asset tagging list is obtained by tagging multiple asset data to be tagged to obtain tagging labels, and each asset data to be tagged and the tagging labels corresponding to each asset data to be tagged form a record of the asset tagging list. The preset asset tagging list is obtained through a preset data acquisition interface to provide data preparation for subsequent recommendation of more accurate asset data. The preset data acquisition interface can be a read() function.

[0090] like Figure 3 As shown, before obtaining the preset asset marking list, the business data processing method also includes but is not limited to the following steps: Step S810, obtaining multiple asset data to be marked, preprocessing each asset data to be marked, and obtaining multiple preprocessed asset data.

[0091] In one embodiment, the asset data to be marked includes different data assets such as products, markets, orders, costs, revenues, services, channels, etc., and multiple asset data to be marked can be obtained through crawler technology, or by sorting out historical data of banking business. Then, multiple asset data to be marked are sorted and saved in CSV, txt or other table formats, and multiple asset data to be marked are obtained using a data reading interface to provide data support for subsequent marking to obtain an asset marking list.

[0092] In one embodiment, each asset data to be marked is preprocessed, and the preprocessing includes data cleaning and data vectorization. Data cleaning performs data screening and removal of messy data on multiple asset data to be marked. Data screening is to select data columns that affect the marking classification of the asset data to be marked, such as the Chinese name of the field, the English name of the field, the Chinese name of the table, the English name of the table, the field type and the field length, etc., and filter out important field data, which can reduce the amount of calculation while ensuring accuracy. Removing messy data can be removing blank lines, garbled characters, notes and duplicate data. Among them, removing duplicate data can remove duplicate record rows according to fields such as category, field Chinese name and field English name, and only retain one row of records in the same data. By performing data preprocessing to obtain multiple preprocessed asset data, the amount of calculation can be reduced and the marking accuracy can be improved.

[0093] Step S820: Use the target labeling model to label each pre-processed asset data to obtain a labeling label.

[0094] In one embodiment, the target labeling model is a trained model, and the pre-processed asset data obtained in step S810 is labeled using the target labeling model. The pre-processed asset data is directly input into the target labeling model, and the target labeling model outputs the labeling label, which is conducive to the subsequent construction of the asset labeling list.

[0095] Step S830, obtaining an asset marking list according to each asset data to be marked and the marking label corresponding to each asset data to be marked.

[0096] In one embodiment, each asset data to be tagged and the tagged label corresponding to the asset data to be tagged are taken as a record, and multiple asset data to be tagged and their corresponding tagged labels constitute multiple records, and multiple records form an asset tagged list. Since multiple asset data to be tagged are data of different attributes of the industry, the asset tagged list includes relatively rich asset data and its tagged labels, which is conducive to the subsequent use of the asset tagged list to match with relevant recommendation results, and recommend accurate asset data to users.

[0097] like Figure 4 As shown, before using the target labeling model to label each pre-processed asset data to obtain a labeling label, the business data processing method also includes but is not limited to the following steps: Step S910, obtaining an asset sample data set, the asset sample data set includes multiple asset sample data and a labeling label corresponding to each asset sample data.

[0098] In one embodiment, the asset sample data set includes multiple asset sample data, and the multiple asset sample data include different data assets such as products, markets, orders, costs, revenues, services, channels, etc. The multiple asset sample data can be obtained by crawler technology, and then the multiple asset sample data are sorted and saved in csv, txt or other table formats. The asset sample data set also includes a marking label corresponding to each asset sample data, and the marking label is obtained by manually annotating the multiple asset sample data, and is stored in a file corresponding to each asset sample data. The asset sample data set is obtained by using the data reading interface to provide data support for subsequent model training.

[0099] It should be noted that the asset sample data is manually labeled, and the labeling rules are divided according to the standards published by the banking industry. The asset sample data is matched with the corresponding sub-categories to obtain the labeling labels. Figure 5 As shown, the categories include: general for participants, general for individuals, general for organizations, customer information-basic information, risk information-credit risk, management personality-individual customers, corporate customers, financial peer customers, employee information, institutional information, merchant information-basic information, terminal information, etc. Each category corresponds to a corresponding level. For example, merchant information-basic information corresponds to level 2. When the asset sample data is input as merchant information, the subcategory under the merchant information, the basic information subcategory, is further determined. The label annotated with the asset sample data is expressed as merchant information-basic information- / -2.

[0100] Step S920: pre-process each asset sample data to obtain a plurality of pre-processed sample data.

[0101] In one embodiment, each asset sample data is preprocessed, and the preprocessing includes data cleaning and data vectorization. Data cleaning performs data screening and removal of messy data on multiple asset sample data. Data screening is to select data columns that affect the labeling classification of asset sample data, such as the Chinese name of the field, the English name of the field, the Chinese name of the table, the English name of the table, the field type and the field length, etc., and filter out important field data, which can reduce the amount of calculation while ensuring accuracy. Removing messy data can be removing blank lines, garbled characters, notes and duplicate data. Among them, removing duplicate data can remove duplicate record rows according to fields such as category, field Chinese name and field English name, and only retain one row of records in the same data. By performing data preprocessing to obtain multiple preprocessed sample data, the amount of calculation can be reduced and the accuracy of model labeling can be improved.

[0102] Step S930: Perform a first sample equalization process on each preprocessed sample data to obtain a training data set.

[0103] like Figure 6As shown, a first sample balancing process is performed on each preprocessed sample data to obtain a training data set, including but not limited to the following steps: Step S931, deduplication processing is performed on the preprocessed sample data according to category, field Chinese name and field English name to obtain first deduplication sample data.

[0104] In some possible embodiments of the present application, the preprocessed sample data obtained in step S920 is deduplicated according to the category, field Chinese name, and field English name. For example, when the category, field Chinese name, and field English name of multiple preprocessed sample data are the same, the multiple preprocessed sample data are duplicate data, and the duplicate data are deleted to ensure only one preprocessed sample data of the same data, and obtain the first deduplicated sample data, which is beneficial to the subsequent training set based on the first deduplicated sample data.

[0105] Step S932: Select N categories with the highest number from the first deduplicated sample data to obtain selected sample data, where N is an integer greater than or equal to 2.

[0106] In some possible embodiments of the present application, N is an integer greater than or equal to 2, and N is not greater than 10. The N categories with the highest number are selected from the first deduplication sample data. When N is 2, the two categories with the highest number are selected to obtain the selected sample data, which is conducive to the subsequent secondary deduplication based on the selected sample data. It should be noted that this category is a large category, which also includes multiple small categories.

[0107] Step S933, deduplicate the selected sample data according to the category and the Chinese name of the field to obtain second deduplicated sample data.

[0108] In some possible embodiments of the present application, since the Chinese name of the field can be translated into different English, based on the selected sample obtained in step S932, deduplication processing is performed again according to the category and the Chinese name of the field to obtain the second deduplication sample data. This can avoid the existence of data with the same meaning in English and thus form duplicate data, and the second deduplication sample data has no duplicate data.

[0109] Step S934, remove the categories whose number does not exceed a preset number in the second deduplication sample data to obtain a training data set.

[0110] In some possible embodiments of the present application, the second deduplication sample data is obtained according to step S933, the number of the second deduplication sample data is counted, and in each small category, the categories whose number in the second deduplication sample data does not exceed a preset number are removed, and the preset number can be 10 or 20. The categories whose number in the second deduplication sample data is less than or equal to 10 are removed to obtain a training data set. Constructing a training data set through the above steps is conducive to improving the accuracy of the model.

[0111] Step S940: Perform a second sample balancing process on each preprocessed sample data to obtain a verification data set.

[0112] In one embodiment, a second sample balancing process is performed on each preprocessed sample data to obtain a verification data set, which specifically includes but is not limited to: removing categories in the preprocessed sample data whose number does not exceed a preset number, which can be 10 or 20. Removing categories in the preprocessed sample data whose number is less than or equal to 10 to obtain a verification data set is beneficial to improving the accuracy of the model.

[0113] like Figure 7 As shown in the figure, assuming that there are 5789 preprocessed sample data, the preprocessed sample data is deduplicated according to the category, field Chinese name and field English name, and then the two categories with the highest number are selected from the deduplicated data; the selected data is deduplicated according to the category and field Chinese name, and the deduplicated data volume is 2252; then the categories with no more than 10 items in the data are removed to obtain a training data set with 2250 items. The categories with no more than 10 items in the preprocessed sample data are removed to obtain a validation data set with 5780 items.

[0114] Step S950, using the training data set to adjust the parameters of the preset first initial labeling model to obtain a second initial labeling model.

[0115] In one embodiment, the training data set includes preprocessed sample data without duplication and whose data volume meets the preset quantity, and the labeling label corresponding to the preprocessed sample data. The preprocessed sample data and the labeling label are used to adjust the parameters of the preset first initial labeling model to obtain the second initial labeling model, which is conducive to the subsequent acquisition of the target labeling model. The first initial labeling model can be a BERT model, a variant of the BERT model, or other deep learning models, as long as it can meet the accurate labeling requirements, which will not be elaborated here; the model parameter adjustment is the parameter adjustment method of the BERT model, which will not be elaborated here.

[0116] Step S960, using the verification data set to verify the second initial marking model, when the accuracy is greater than a preset accuracy threshold, a target marking model is obtained.

[0117] In one embodiment, the training data set includes a preset number of preprocessed sample data, but does not include labeling tags corresponding to the preprocessed sample data. The second initial labeling model is verified using the verification data set. When the accuracy is greater than a preset accuracy threshold, it indicates that the model has good generalization ability, and the target labeling model is obtained, which also has a high accuracy in labeling the retrieved asset data.

[0118] In another embodiment, when the accuracy is less than or equal to a preset accuracy threshold, the training data set is subjected to data dimensionality upgrading, and the first initial labeling model is adjusted using the upgraded training data set so that the accuracy is greater than the preset accuracy threshold.

[0119] In another embodiment, when a bank provides new sample data, the sample data has no corresponding label. Categories whose number does not exceed a preset number in the new sample data are removed, and the target labeling model is tested with an accuracy of up to 82%. The target labeling model has good generalization ability.

[0120] Step S500, obtaining a matching result according to the recommended data, the tag corresponding to the recommended data, the asset data to be marked and the marked tag.

[0121] In one embodiment, a matching result is obtained based on the recommended data, the label corresponding to the recommended data, the asset data to be marked and the marked label, which specifically includes but is not limited to: firstly, the label corresponding to the recommended data and the marked label are classified, and a preset classification algorithm is used to determine whether the label corresponding to the recommended data and the marked label are the same subcategory, and the similarity algorithm is used to calculate the similarity between the recommended data and the asset data to be marked, and the data with the same label category and similarity less than a preset threshold are screened out to obtain a matching result, which is conducive to the subsequent accurate asset data recommendation based on the matching result. The classification algorithm can be a support vector machine or a deep learning algorithm; the similarity algorithm can be a Euclidean distance, a character similarity algorithm, etc., which will not be described in detail here.

[0122] Step S600: Perform a second recommendation based on the matching result and the retrieved asset data to obtain recommended asset data.

[0123] In one embodiment, the second recommendation is an accurate recommendation based on the matching result, and among the data of the same category, the asset data that better meets the user's needs is recommended. It is beneficial that the matching result obtained in step S500 has a high similarity and the same label, and is highly matched with the retrieved asset data, and the recommended data ranked first is recommended to obtain the recommended asset data. The recommended asset data is screened multiple times and better meets the relevant requirements of the retrieved asset data, so as to recommend accurate asset data to the user.

[0124] It should be noted that the corresponding tags in the recommended asset data can be used as tags for retrieving asset data, and the retrieved asset data and its corresponding tags can be added to the asset tagging list. Increasing the amount of data in the asset tagging list can improve the accuracy of recommendations.

[0125] like Figure 8 As shown, Figure 8The overall process diagram of a business data processing method provided by an embodiment of the present application is shown. First, the retrieved asset data is obtained, and in response to the selection operation of a preset target labeling model, vector processing is performed on the retrieved asset data, and dimensionality is increased. The processed data is fused and input into the target labeling model for labeling processing to obtain a labeling result. The obtained labeling result indicates the category to which it belongs, which is conducive to accurate recommendation. According to the labeling result, a first recommendation is performed using a strategy among preset quantity restrictions, matching degree restrictions, and combination restrictions to obtain relevant recommendation results. The relevant recommendation results include recommended data and labels corresponding to the recommended data. A rough recommendation is first performed using the recommendation strategy, which is conducive to improving the accuracy of the recommendation.

[0126] In one embodiment, multiple asset data to be marked are obtained, each asset data to be marked is preprocessed to obtain multiple preprocessed asset data, each preprocessed asset data is marked using a target marking model to obtain a marking label; the marked data is confirmed and published, the asset data is counted, and effective asset management is achieved, and an asset marking list is obtained based on each asset data to be marked after confirmation and publication and the marking labels corresponding to each asset data to be marked. A preset asset marking list is obtained, and the asset marking list includes multiple asset data to be marked and the marking labels corresponding to each asset data to be marked; a matching result is obtained based on the recommended data, the labels corresponding to the recommended data, the asset data to be marked, and the marking labels; a second recommendation is made based on the matching result and the retrieved asset data to obtain recommended asset data. By making the second recommendation, more accurate recommended asset data can be obtained.

[0127] like Fig. 9As shown, the embodiment of the present application provides a business data processing device 100, which obtains the retrieved asset data through the first data acquisition module 110. The retrieved asset data is the search content input by the user. The retrieved asset data is obtained to provide data support for the subsequent recommendation of accurate asset data to the user; the labeling module 120 is used to respond to the selection operation of the preset target labeling model, and the retrieved asset data is labeled using the target labeling model to obtain the labeling result. The retrieved asset data is first labeled, and the obtained labeling result indicates the category to which it belongs, which is conducive to accurate recommendation; then the first recommendation module 130 is used to make a first recommendation based on the labeling result using the preset recommendation strategy to obtain Relevant recommendation results, the relevant recommendation results include the recommended data and the labels corresponding to the recommended data. The recommendation strategy is first used for rough recommendation, which is conducive to improving the accuracy of the recommendation; the second data acquisition module 140 is used to obtain the preset asset labeling list, and the asset labeling list includes multiple asset data to be labeled and the labeling labels corresponding to each asset data to be labeled, so as to provide data support for subsequent accurate recommendations; then the matching module 150 is used to obtain the matching results according to the recommended data, the labels corresponding to the recommended data, the asset data to be labeled and the labeling labels; finally, the second recommendation module 160 is used to make a second recommendation according to the matching results and the retrieved asset data to obtain the recommended asset data, so as to obtain more accurate recommended asset data.

[0128] It should be noted that the first data acquisition module 110 is connected to the tagging module 120, the tagging module 120 is connected to the first recommendation module 130, the first recommendation module 130 is connected to the second data acquisition module 140, the second data acquisition module 140 is connected to the matching module 150, and the matching module 150 is connected to the second recommendation module 160. The above-mentioned business data processing method is applied to the business data processing device 100, which tags the retrieved asset data through the target tagging model, makes a first recommendation based on the tagging result, screens out asset data similar to the retrieved asset data, and then makes a second recommendation based on the asset tagging list, and can recommend accurate asset data to users through multi-layer recommendations.

[0129] It should also be noted that: when the device provided in the above embodiment realizes its function, only the division of the above functional modules is used as an example. In actual application, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0130] The present application also discloses an electronic device. Fig.10 , Fig.10The electronic device 500 may include: at least one processor 501 , at least one network interface 504 , a user interface 503 , a memory 505 , and at least one communication bus 502 .

[0131] The communication bus 502 is used to realize the connection and communication between these components.

[0132] The user interface 503 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 503 may also include a standard wired interface and a wireless interface.

[0133] The network interface 504 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0134] Among them, the processor 501 may include one or more processing cores. The processor 501 uses various interfaces and lines to connect various parts in the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 505, and calling data stored in the memory 505. Optionally, the processor 501 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 501 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 501, and it can be implemented separately through a chip.

[0135] Among them, the memory 505 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory). Optionally, the memory 505 includes a non-transitory computer-readable storage medium. The memory 505 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 505 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 505 may also be optionally at least one storage device located away from the aforementioned processor 501. Refer to Fig.10 , the memory 505 as a computer storage medium may include an operating system, a network communication module, a user interface module and an application program of a business data processing method.

[0136] exist Fig.10 In the electronic device 500 shown, the user interface 503 is mainly used to provide an input interface for the user and obtain the data input by the user; and the processor 501 can be used to call an application program storing a business data processing method in the memory 505, and when executed by one or more processors 501, the electronic device 500 executes one or more methods in the above-mentioned embodiments. It should be noted that for the aforementioned method embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for the present application.

[0137] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0138] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0139] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0140] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0141] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: various media that can store program codes, such as USB flash drives, mobile hard drives, magnetic disks or optical disks.

[0142] The above are only exemplary embodiments of the present disclosure and cannot be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification and the truth of practice, those skilled in the art will easily think of other embodiments of the present disclosure.

[0143] This application is intended to cover any variation, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the art not described in the present disclosure. The description and examples are to be regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A business data processing method, characterized in that: The method comprises: Get retrieve asset data; In response to a selection operation of a preset target marking model, the retrieved asset data is marked using the target marking model to obtain a marking result; According to the labeling result, a first recommendation is performed using a preset recommendation strategy to obtain a relevant recommendation result, wherein the relevant recommendation result includes the recommended data and a label corresponding to the recommended data; Obtaining a preset asset marking list, wherein the asset marking list includes a plurality of asset data to be marked and a marking label corresponding to each of the asset data to be marked; Obtaining a matching result according to the recommended data, the tag corresponding to the recommended data, the asset data to be marked, and the marked tag; Perform a second recommendation based on the matching result and the retrieved asset data to obtain recommended asset data; Before obtaining the preset asset marking list, the method further includes: Acquire multiple asset data to be marked, and pre-process each of the asset data to be marked to obtain multiple pre-processed asset data; Using the target labeling model to label each of the pre-processed asset data to obtain a labeling label; Obtaining the asset marking list according to each of the asset data to be marked and the marking tags corresponding to each of the asset data to be marked; Before labeling each of the pre-processed asset data using the target labeling model to obtain a labeling label, the method further includes: Acquire an asset sample data set, wherein the asset sample data set includes a plurality of asset sample data and a label corresponding to each of the asset sample data; Preprocessing each of the asset sample data to obtain a plurality of preprocessed sample data; Performing a first sample balancing process on each of the preprocessed sample data to obtain a training data set; Performing a second sample balancing process on each of the preprocessed sample data to obtain a verification data set; Using the training data set to adjust the parameters of the preset first initial labeling model to obtain a second initial labeling model; Using the verification data set to verify the second initial marking model, and when the accuracy is greater than a preset accuracy threshold, obtaining the target marking model; The step of performing a first sample balancing process on each of the pre-processed sample data to obtain a training data set includes: Deduplication processing is performed on the preprocessed sample data according to the category, the Chinese name of the field, and the English name of the field to obtain first deduplication sample data; Selecting N categories with the largest number from the first deduplicated sample data to obtain selected sample data, where N is an integer greater than or equal to 2; Deduplication of the selected sample data is performed according to the category and the Chinese name of the field to obtain second deduplication sample data; Removing the categories whose number does not exceed a preset number in the second deduplication sample data to obtain the training data set; The step of performing a second sample balancing process on each of the preprocessed sample data to obtain a verification data set includes: The categories whose number in the preprocessed sample data does not exceed a preset number are removed to obtain the verification data set.

2. The method according to claim 1, characterized in that Before labeling the retrieved asset data using the target labeling model to obtain a labeling result, the method further includes: Performing vector processing on the retrieved asset data to obtain an asset data vector; Performing data dimensionality upgrading processing on the retrieved asset data to obtain a multi-dimensional data vector; The asset data vector and the multi-dimensional data vector are fused and then input into the target marking model.

3. The method according to claim 1, characterized in that The obtaining a matching result according to the recommended data, the tag corresponding to the recommended data, the asset data to be marked, and the marked tag includes: The tags corresponding to the recommended data and the marked tags are classified, and the similarity between the recommended data and the asset data to be marked is calculated to filter out data with consistent tag categories and similarity less than a preset threshold to obtain the matching result.

4. The method according to claim 1, characterized in that The recommendation strategy is at least one of quantity restriction, matching degree restriction, and combination restriction.

5. A business data processing device, used to implement the method according to any one of claims 1 to 4, characterized in that: The device comprises: A first data acquisition module (110), used for acquiring and retrieving asset data; A marking module (120), configured to, in response to a selection operation of a preset target marking model, mark the retrieved asset data using the target marking model to obtain a marking result; A first recommendation module (130), configured to make a first recommendation based on the labeling result using a preset recommendation strategy to obtain a relevant recommendation result, wherein the relevant recommendation result includes the recommended data and a label corresponding to the recommended data; A second data acquisition module (140) is used to acquire a preset asset marking list, wherein the asset marking list includes a plurality of asset data to be marked and a marking label corresponding to each of the asset data to be marked; A matching module (150), configured to obtain a matching result according to the recommended data, the tag corresponding to the recommended data, the asset data to be marked and the marked tag; A second recommendation module (160), configured to make a second recommendation based on the matching result and the retrieved asset data to obtain recommended asset data; The second data acquisition module (140) is further used to acquire a plurality of asset data to be marked, and pre-process each of the asset data to be marked to obtain a plurality of pre-processed asset data; Using the target labeling model to label each of the pre-processed asset data to obtain a labeling label; Obtaining the asset marking list according to each of the asset data to be marked and the marking tags corresponding to each of the asset data to be marked; Before labeling each of the pre-processed asset data using the target labeling model to obtain a labeling label, it is also used to obtain an asset sample data set, wherein the asset sample data set includes a plurality of asset sample data and a labeling label corresponding to each of the asset sample data; Preprocessing each of the asset sample data to obtain a plurality of preprocessed sample data; Performing a first sample balancing process on each of the preprocessed sample data to obtain a training data set; Performing a second sample balancing process on each of the preprocessed sample data to obtain a verification data set; Using the training data set to adjust the parameters of the preset first initial labeling model to obtain a second initial labeling model; The second initial marking model is verified using the verification data set, and when the accuracy is greater than a preset accuracy threshold, the target marking model is obtained.

6. An electronic device, characterized in that: The electronic device (500) comprises a processor (501), a memory (505), a user interface (503), a communication bus (502) and a network interface (504); the processor (501), the memory (505), the user interface (503) and the network interface (504) are respectively connected to the communication bus (502); the memory (505) is used to store instructions; the user interface (503) and the network interface (504) are used to communicate with other devices; the processor (501) is used to execute the instructions stored in the memory (505) so that the electronic device (500) executes the method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1 to 4 is performed.

Images