Enterprise credit model optimization method and system based on big data

Abstract

The invention relates to the technical field of enterprise credit evaluation, in particular to an enterprise credit model optimization method and system based on big data, and the method comprises the steps: obtaining training data corresponding to a target enterprise credit model, and determining a model data state according to the coverage integrity and data singularity of the training data, determining an optimization analysis mode according to the model data state; in the data supplementary analysis, determining a search mode corresponding to each search keyword according to the data richness and the effective evaluation index so as to obtain search data, and determining a supplementary mode according to the effective search data quantity and the dislocation coefficient of the associated search data combination; determining an adjustment mode according to the correlation coefficient and the comparison coefficient of the supplementary data; in data rejection analysis, noise indexes are rejected according to an index fluctuation coefficient and an index offset rate; training the target enterprise credit model by adopting the optimized data; the stability and reliability of the enterprise credit model can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of enterprise credit assessment, and in particular to an optimization method and system for an enterprise credit model based on big data. Background Art

[0002] In the era of big data, more and more data is publicly released by enterprises, which provides rich data for constructing and optimizing enterprise credit models. However, the training data of current enterprise credit models generally faces severe challenges such as data incompleteness and data imbalance, resulting in significant volatility in the evaluation of enterprise credit by the models. The stability of the prediction results is greatly reduced when facing new data, and thus the true credit status of enterprises cannot be accurately reflected. Therefore, how to optimize the training data of enterprise training models to improve the stability and reliability of enterprise credit models is a technical problem that needs to be solved urgently by those skilled in the art.

[0003] Chinese Patent Publication No. CN114219562A discloses a training method of a model, an enterprise credit assessment method and device, equipment, and medium, including: obtaining enterprise sample data, preprocessing the enterprise sample data to obtain training data, and improving the accuracy of model training by optimizing the enterprise sample data. Then, calculating a first target value for a first objective function of an original training model according to the training data; fine-tuning the parameters to be adjusted of a second objective function of the original training model according to the first target value, and using the fine-tuned second objective function as the model parameters of the original training model to update the original training model to obtain an enterprise credit assessment model; wherein, the enterprise credit assessment model is used for evaluating the credit of enterprises. It can be seen that the above technical solution has the following problems: when optimizing the enterprise sample data, the problem of unstable model training results caused by unbalanced training data is not considered, and the data supplement method cannot be adaptively adjusted according to the actual state of the training data, resulting in poor stability and reliability of the enterprise credit model. Summary of the Invention

[0004] Therefore, the present invention provides an optimization method and system for an enterprise credit model based on big data to overcome the problem in the prior art that the imbalance of training data is not considered, and thus the data supplement method cannot be adaptively adjusted according to the actual state of the training data, resulting in poor stability and reliability of the enterprise credit model.

[0005] To achieve the above object, the present invention provides an optimization method for an enterprise credit model based on big data, including:

[0006] Obtain the training data corresponding to the target enterprise credit model, determine the model data status based on the coverage integrity and data singularity of the training data, and determine the optimization analysis method according to the model data status. The optimization analysis method is data supplementation analysis or data elimination analysis;

[0007] In the data supplementation analysis, determine the search keywords according to the combined data ratio, determine the search method corresponding to each search keyword according to the data richness and the effective evaluation index to obtain the search data, and determine the supplementation method according to the number of effective search data in the associated search data combination and the dislocation coefficient;

[0008] Determine the adjustment method according to the correlation coefficient and comparison coefficient of the supplementary data. The adjustment method is to adjust the search depth or the backtracking range;

[0009] The search method is to perform matching search according to the keyword correlation degree and the link evaluation value or to perform independent search according to the link validity. The supplementation method is to select supplementary data according to the law representativeness and data dislocation value or to determine whether to use the search data as supplementary data according to the anomaly index;

[0010] In the data elimination analysis, eliminate the noise indicators according to the index fluctuation coefficient and the index offset rate;

[0011] Use the optimized data to train the target enterprise credit model.

[0012] Furthermore, determining the optimization analysis method according to the model data status includes:

[0013] If the model data status is that the coverage integrity is less than the preset coverage integrity or the data singularity is greater than or equal to the preset data singularity, the optimization analysis method is data supplementation analysis;

[0014] If the model data status is that the coverage integrity is greater than or equal to the preset coverage integrity and the data singularity is less than the preset data singularity, the optimization analysis method is data elimination analysis.

[0015] Furthermore, the method for confirming the data singularity includes:

[0016] If the data volume reference value is greater than or equal to the preset data volume reference value, determine the data singularity according to the ratio of the balance coefficient to the data volume reference value;

[0017] If the data volume reference value is less than the preset data volume reference value, determine the data singularity according to the data correlation degree.

[0018] Furthermore, determining the search keywords according to the combined data ratio includes:

[0019] Determine the associated data combinations according to industry keywords, and conduct keyword selection analysis for each associated data combination. When conducting keyword selection analysis for a single associated data combination,

[0020] If the proportion of combined data is greater than or equal to the preset proportion of combined data, determine the search keywords according to the index comparison coefficient and the index distribution proportion;

[0021] If the proportion of combined data is less than the preset proportion of combined data, determine the search keywords according to the level dislocation coefficient.

[0022] Furthermore, determine the search methods corresponding to each search keyword according to the data richness and the effective evaluation index, including:

[0023] For a single search keyword,

[0024] If the data richness is less than the preset data richness or the effective evaluation index is less than the preset effective evaluation index, the search method is to perform matching search according to the keyword relevance and the link evaluation value;

[0025] If the data richness is greater than or equal to the preset data richness and the effective evaluation index is greater than or equal to the preset effective evaluation index, the search method is to perform separate search according to the link validity.

[0026] Furthermore, determine the supplementation method according to the effective search data quantity and the dislocation coefficient of the associated search data combination, including:

[0027] For a single associated search data combination,

[0028] If the effective search data quantity is greater than the standard quantity and the dislocation coefficient is greater than or equal to the preset dislocation coefficient, the supplementation method is to select supplementary data according to the law representativeness and the data dislocation value;

[0029] If the effective search data quantity is equal to the standard quantity or the dislocation coefficient is less than the preset dislocation coefficient, the supplementation method is to determine whether to use the search data as supplementary data according to the anomaly index.

[0030] Furthermore, the confirmation methods of the associated search data combination include:

[0031] For a single enterprise name, use each search data corresponding to this enterprise name within the corresponding retrospective range of this enterprise name as the associated search data combination corresponding to this enterprise name;

[0032] The retrospective range corresponding to a single enterprise name is determined according to the dynamic comparison coefficient and the in-domain relevance corresponding to this enterprise name.

[0033] Furthermore, determine the adjustment method according to the correlation coefficient and the comparison coefficient of the supplementary data, including:

[0034] If the correlation coefficient is greater than the preset correlation coefficient and the comparison coefficient is less than the preset comparison coefficient, the adjustment method is to increase the search depth for adjustment;

[0035] If the correlation coefficient is greater than the preset correlation coefficient and the comparison coefficient is greater than or equal to the preset comparison coefficient, the adjustment method is to increase the backtracking range for adjustment.

[0036] Furthermore, the noise indicators are eliminated according to the index fluctuation coefficient and the index offset rate, including:

[0037] Special index keywords with an index fluctuation coefficient less than the preset index fluctuation coefficient and an index offset rate less than the preset index offset rate are recorded as noise indicators, and the noise indicators and the corresponding data keywords of the noise indicators are eliminated.

[0038] The present invention also provides an enterprise credit model optimization system based on big data, including:

[0039] A data acquisition module for obtaining training data corresponding to the target enterprise credit model;

[0040] An optimization analysis module, connected to the data acquisition module, for determining the model data status according to the coverage integrity and data singularity of the training data, and determining the optimization analysis method according to the model data status. The optimization analysis method is data supplement analysis or data elimination analysis;

[0041] A data supplement module, respectively connected to the data acquisition module and the optimization analysis module, for determining search keywords according to the combined data ratio in data supplement analysis, determining the search methods corresponding to each search keyword according to the data richness and the effective evaluation index to obtain search data, and determining the supplement method according to the number of effective search data in the associated search data combination and the dislocation coefficient;

[0042] A supplement adjustment module, respectively connected to the data acquisition module and the data supplement module, for determining the adjustment method according to the correlation coefficient and the comparison coefficient of the supplementary data. The adjustment method is to adjust the search depth or the backtracking range;

[0043] A data elimination module, connected to the optimization analysis module, for eliminating noise indicators according to the index fluctuation coefficient and the index offset rate in data elimination analysis;

[0044] A model optimization module, respectively connected to the data supplement module, the supplement adjustment module and the data elimination module, for training the target enterprise credit model with the optimized data.

[0045] Compared with the prior art, the beneficial effects of the present invention are as follows. In the technical solution of the present invention, the model data status is determined according to the coverage integrity and data singularity of the training data. The coverage integrity and data singularity effectively reflect the integrity of the training data. Then, different optimization analysis methods are adaptively selected according to the model data status, making the selection of the optimization analysis method more in line with the actual application scenario. When the balance degree of the training data is poor, more data can be introduced to improve the integrity of the training data. When the redundancy degree of the data is high, redundant or low-correlation features can be removed to improve the generalization ability of the model, so that the model can capture the credit status of the enterprise more comprehensively and accurately.

[0046] Furthermore, in the present invention, the search keywords are determined according to the proportion of combined data. The proportion of combined data effectively reflects the integrity of the training data corresponding to the associated data combination. Then, different confirmation methods of search keywords are adaptively selected according to the proportion of combined data, which can improve the accuracy of search keyword determination, and then can more accurately locate the training data that needs to be supplemented. This not only improves the efficiency of data supplementation but also ensures a high correlation between the supplemented data and the existing data, thereby enhancing the overall performance of the model.

[0047] Furthermore, in the present invention, the data richness and the effective evaluation index effectively reflect the richness of the training data corresponding to the search keywords and the richness of the information on the search page corresponding to the search keywords. Then, different search methods are adaptively selected according to the data richness and the effective evaluation index, making the selection of the search method more in line with the actual application scenario, being able to flexibly adjust the search strategy, and then improving the search efficiency and the accuracy of the search.

[0048] Furthermore, in the present invention, the dynamic comparison coefficient and the in-domain correlation degree can reflect the reliability of the search data and the change of the search data over time. Then, the backtracking range is determined through the dynamic comparison coefficient and the in-domain correlation degree, which helps to filter out irrelevant or low-quality search data. Then, the effectiveness of the search data in the associated search data combination can be ensured. Furthermore, the difference degree of the search data in the associated search data combination is affected by the number of effective search data and the dislocation coefficient. Then, different supplementation methods are adaptively selected according to the number of effective search data and the dislocation coefficient, making the selection of the supplementation method able to select search data with a greater representative degree, and then improving the accuracy of information supplementation.

[0049] Furthermore, in the present invention, the correlation coefficient and the comparison coefficient effectively reflect the richness of the supplemented search data. Then, different adjustment methods are adaptively selected according to the correlation coefficient and the comparison coefficient of the supplemented data, and more relevant information can be mined through the selected adjustment method, which helps to improve the efficiency and effect of information retrieval. Description of the Drawings

[0050] Figure 1 Schematic diagram of the enterprise credit model optimization method based on big data according to the present invention;

[0051] Figure 2 Flowchart of determining the optimization analysis method according to the model data status of the present invention;

[0052] Figure 3 Flowchart of determining the search method according to the data richness and the effective evaluation index of the present invention;

[0053] Figure 4 Module connection diagram of the enterprise credit model optimization system based on big data according to the present invention. Detailed implementation manners

[0054] In order to make the objectives and advantages of the present invention clearer, the present invention will be further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0055] The preferred implementation manners of the present invention will be described below with reference to the accompanying drawings. Those skilled in the art should understand that these implementation manners are only used to explain the technical principles of the present invention and do not limit the protection scope of the present invention.

[0056] It should be noted that in the description of the present invention, the terms indicating directions or positional relationships such as "upper", "lower", "left", "right", "inner", "outer", etc. are based on the directions or positional relationships shown in the drawings. This is only for convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation, and thus cannot be construed as a limitation of the present invention.

[0057] In addition, it should be noted that in the description of the present invention, unless otherwise clearly specified and limited, the terms "installation", "connection", and "connection" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two elements. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0058] Please refer to Figures 1 to 3 As shown, the present invention provides an enterprise credit model optimization method based on big data, including:

[0059] Obtain the training data corresponding to the target enterprise credit model, determine the model data status according to the coverage integrity and data singularity of the training data, and determine the optimization analysis method according to the model data status. The optimization analysis method is data supplementation analysis or data elimination analysis;

[0060] In the data supplementation analysis, determine the search keywords according to the combined data ratio, determine the search method corresponding to each search keyword according to the data richness and the effective evaluation index to obtain the search data, and determine the supplementation method according to the number of effective search data in the associated search data combination and the dislocation coefficient;

[0061] Determine the adjustment method according to the correlation coefficient and the comparison coefficient of the supplementary data. The adjustment method is to adjust the search depth or the backtracking range;

[0062] The search method is to perform matching search according to the keyword correlation degree and the link evaluation value or to perform independent search according to the link validity. The supplementation method is to select supplementary data according to the law representativeness and the data dislocation value or to determine whether to use the search data as supplementary data according to the abnormal index;

[0063] In the data elimination analysis, eliminate the noise indicators according to the index fluctuation coefficient and the index offset rate;

[0064] Use the optimized data to train the target enterprise credit model.

[0065] In the present invention, the training data in the training process of the target enterprise credit model is optimized. Each training data includes data keywords corresponding to several index keywords. The target enterprise credit model is the enterprise credit model that needs to be optimized. The index keywords include, but are not limited to, the establishment years, annual turnover, net profit rate, number of employees, industry category, and credit rating. The data keywords corresponding to the establishment years include, but are not limited to, 1 year, 5 years, and 10 years. The data keywords corresponding to the annual turnover include, but are not limited to, 1 million yuan and 20 million yuan. The data keywords corresponding to the net profit rate include, but are not limited to, 10% and 12%. The data keywords corresponding to the number of employees include, but are not limited to, 50 people and 100 people. The data keywords corresponding to the industry category include, but are not limited to, manufacturing, service, and information technology. The data keywords corresponding to the credit rating include AAA level, AA level, A level, BBB level, and BB level. This is the content that those skilled in the art can easily understand, and will not be elaborated here.

[0066] In the present invention, a number of historical records are correspondingly set. Any one of the historical records records data such as the data singularity, data volume reference value, index comparison coefficient, level dislocation coefficient, data richness, and effective evaluation index during at least one historical process of optimizing the enterprise credit model. And each historical record corresponds to a qualified mark, which records whether the enterprise credit model optimization process meets the user's requirements. The qualified mark can be manually recorded. It can be understood that the user can determine whether the enterprise credit model optimization process meets the requirements according to the self-set index. The self-set index can be, but is not limited to, the misjudgment rate, which will not be elaborated here. Among them, the misjudgment rate is the number of times the target enterprise credit model wrongly evaluates the credit level after training with the optimized training data.

[0067] The optimized data is the supplementary data and training data obtained through data supplement analysis, or the training data remaining after removing the noise indicators and the corresponding data keywords of the noise indicators through data elimination analysis; the present invention performs subsequent model training with the optimized data to obtain an optimized target enterprise credit model, making the enterprise credit rating result more accurate. This is the content that those skilled in the art can easily understand and will not be elaborated specifically.

[0068] Specifically, the optimization analysis method is determined according to the model data status, including:

[0069] If the model data status is that the coverage integrity is less than the preset coverage integrity or the data singularity is greater than or equal to the preset data singularity, the optimization analysis method is data supplement analysis;

[0070] If the model data status is that the coverage integrity is greater than or equal to the preset coverage integrity and the data singularity is less than the preset data singularity, the optimization analysis method is data elimination analysis.

[0071] Among them, the model data status includes the first model data status and the second model data status. The first model data status is that the coverage integrity is less than the preset coverage integrity or the data singularity is greater than or equal to the preset data singularity, and the second model data status is that the coverage integrity is greater than or equal to the preset coverage integrity and the data singularity is less than the preset data singularity;

[0072] Coverage integrity = (the minimum value of the training data volume corresponding to each associated data combination) / (the maximum value of the training data volume corresponding to each associated data combination). The training data volume corresponding to a single associated data combination is the number of training data included in the associated data combination;

[0073] The values of the preset coverage integrity and the preset data singularity can be determined by the user according to the actual application scenario. The smaller the value of the preset coverage integrity, and the larger the value of the preset data singularity, the greater the user's need for data supplement analysis. Provide a set of values for the preset coverage integrity and the preset data singularity. The preset coverage integrity is 50%. Detect the historical records of data supplement analysis, and record the average value of the data singularity corresponding to the historical records that can meet the user's needs as the preset data singularity.

[0074] Specifically, the method for confirming the data singularity includes:

[0075] If the data volume reference value is greater than or equal to the preset data volume reference value, determine the data singularity according to the ratio of the balance coefficient to the data volume reference value;

[0076] If the data volume reference value is less than the preset data volume reference value, determine the data singularity according to the data correlation degree.

[0077] Among them, if the data volume reference value is greater than or equal to the preset data volume reference value, the data singularity = balance coefficient / data volume reference value;

[0078] If the data volume reference value is less than the preset data volume reference value, the data singularity = 1 / data correlation degree;

[0079] The data volume reference value is the total amount of training data corresponding to the target enterprise credit model. The value of the preset data volume reference value can be determined by the user according to the actual application scenario. The smaller the value of the preset data volume reference value, the greater the user's need to determine the data singularity according to the ratio of the balance coefficient to the data volume reference value. Provide a set of values for the preset data volume reference value. Detect the historical records of the user determining the data singularity according to the ratio of the balance coefficient to the data volume reference value, and record the average value of the data volume reference value corresponding to the historical records that can meet the user's needs as the preset data volume reference value;

[0080] The balance coefficient is the average value of the sub-balance coefficients corresponding to each special index keyword. The special index keyword is an index keyword in which the data keywords in the corresponding training data all contain numbers. The sub-balance coefficient corresponding to a single special index keyword is the standard deviation of the numerical values of the data keywords corresponding to this special index keyword. The numerical value corresponding to a single data keyword is the number contained in this data keyword;

[0081] The data correlation degree is the average value of the sub-correlation means corresponding to each special index keyword,

[0082] For a single special index keyword, denote this special index keyword as the target special index keyword, and denote other special index keywords except the target feature index as reference special index keywords. The sub - related mean value corresponding to the target special index keyword is the average value of the correlation coefficients between the target special index keyword and each reference special index keyword;

[0083] Among them, the calculation formula for the correlation coefficient r between any two special index keywords is:

[0084]

[0085] Among them, denote the two special index keywords as the first special index keyword and the second special index keyword respectively, and n is the number of training data; x i is the value of the data keyword corresponding to the first special index keyword of the i - th training data, and y i is the value of the data keyword corresponding to the second special index keyword of the i - th training data, is the average value of the values of each data keyword corresponding to the first special index keyword, is the average value of the values of each data keyword corresponding to the second special index keyword, and i = 1, 2, 3, ……, n.

[0086] Specifically, determining the search keyword according to the combined data proportion includes:

[0087] Determine the associated data combination according to the industry keyword, and conduct an analysis on the selection of search keywords for each associated data combination. When conducting an analysis on the selection of search keywords for a single associated data combination,

[0088] if the combined data proportion is greater than or equal to the preset combined data proportion, determine the search keyword according to the index comparison coefficient and the index distribution proportion;

[0089] if the combined data proportion is less than the preset combined data proportion, determine the search keyword according to the level dislocation coefficient.

[0090] Among them, determining the associated data combination according to the industry keyword includes: Denote the data keywords in each training data corresponding to the industry category as the industry keyword, and denote the set of each training data corresponding to the same industry keyword as an associated data combination;

[0091] The confirmation method of the combined data proportion is that for a single associated data combination, the combined data proportion corresponding to this associated data combination=(the number of training data included in this associated data combination) / (the total amount of training data corresponding to the target enterprise credit model);

[0092] The value of the preset combination data ratio can be determined by the user according to the actual application scenario. The smaller the value of the preset combination data ratio is, the greater the user's demand for determining the search keyword based on the indicator comparison coefficient and the indicator distribution ratio is. A value of the preset combination data ratio is provided, and the preset combination data ratio is 20%;

[0093] Determining the search keywords according to the index comparison coefficient and the index distribution ratio includes: for a single associated data combination, using the industry keywords and each analysis phrase corresponding to the associated data combination as the search keywords;

[0094] The analysis phrase is a single analysis word and a single missing data keyword corresponding to the analysis word, and each missing data keyword corresponds to an analysis phrase;

[0095] The analysis word is a special indicator keyword whose indicator comparison coefficient among the special indicator keywords corresponding to the associated data combination is less than the preset indicator comparison coefficient or whose indicator distribution ratio is less than the preset indicator distribution ratio;

[0096] The missing data keyword is confirmed in the following manner: for a single analysis word, the analysis word is recorded as a target analysis word, the data keyword in each training data corresponding to the target analysis word is recorded as the target data keyword, and the target data keyword whose frequency coefficient is less than the preset frequency coefficient is recorded as the missing data keyword corresponding to the target analysis word;

[0097] For a single associated data combination, the associated data combination is recorded as a target combination, and other associated data combinations other than the target combination are recorded as reference combinations. The indicator comparison coefficient is confirmed in the following manner: for a single special indicator keyword, the values ​​of each data keyword in the single associated data combination corresponding to the special indicator keyword are recorded as the first value, and the numerical analysis coefficient corresponding to the single associated data combination = the largest first value corresponding to the associated data combination - the smallest first value corresponding to the associated data combination. The indicator comparison coefficient corresponding to the special indicator keyword is the average value of the indicator difference between the target combination and each reference combination, and the indicator difference is the absolute value of the difference between the numerical analysis coefficients corresponding to the two associated data combinations;

[0098] The indicator distribution ratio is confirmed by recording the number of different data keywords in each training data in the target combination corresponding to a single special indicator keyword as a, and recording the number of different data keywords in all training data corresponding to the special indicator keyword as b. The indicator distribution ratio = a / b;

[0099] The confirmation method of the frequency coefficient is as follows: Denote the data keywords in each training data corresponding to a single special index keyword as the first reference data keywords. For a single first reference data keyword, regard this first reference data keyword as the target word, and the frequency coefficient corresponding to the target word = (the number of times the target word appears in each training data corresponding to the target combination) / (the number of times the target word appears in each training data corresponding to the target enterprise credit model);

[0100] For the values of the preset index comparison coefficient, the preset index distribution ratio, and the preset frequency coefficient, users can determine them according to the actual application scenario. The greater the user's precision requirement for model optimization, the greater the values of the preset index comparison coefficient, the preset index distribution ratio, and the preset frequency coefficient. Provide a set of values for the preset index comparison coefficient, the preset index distribution ratio, and the preset frequency coefficient. Detect the historical records of determining search keywords based on the index comparison coefficient and the index distribution ratio, and record the average value of the index comparison coefficients corresponding to the historical records that can meet the user's needs as the preset index comparison coefficient. The preset index distribution ratio is 70%, and the preset frequency coefficient is 60%;

[0101] Determine search keywords according to the level dislocation coefficient, including: For a single associated data combination, use the level keywords whose level dislocation coefficients of the industry keywords corresponding to this associated data combination and the level keywords corresponding to this associated data combination are less than the preset level dislocation coefficient as search keywords;

[0102] The level keyword is the data keyword corresponding to the credit level;

[0103] For a single level keyword in a single associated data combination, the level dislocation coefficient = (the number of times this level keyword appears in the training data corresponding to this associated data combination) / (the total amount of the training data corresponding to this associated data combination / 5);

[0104] For the value of the preset level dislocation coefficient, users can determine it according to the actual application scenario. The greater the user's precision requirement for model optimization, the greater the value of the level dislocation coefficient. Provide a value for the preset level dislocation coefficient. Detect the historical records of determining search keywords based on the level dislocation coefficient, and record the average value of the level dislocation coefficients corresponding to the historical records that can meet the user's needs as the preset level dislocation coefficient.

[0105] Specifically, determine the search method corresponding to each search keyword according to the data richness and the effective evaluation index, including:

[0106] For a single search keyword,

[0107] If the data richness is less than the preset data richness or the effective evaluation index is less than the preset effective evaluation index, the search method is to perform a matching search based on the keyword relevance and the link evaluation value;

[0108] If the data richness is greater than or equal to the preset data richness and the effective evaluation index is greater than or equal to the preset effective evaluation index, the search method is to perform a separate search based on the link validity.

[0109] Among them, the confirmation method of data richness is that for a single search keyword, this search keyword is recorded as the target search keyword, and it is the number of enterprise names that appear in the search page corresponding to the target search keyword;

[0110] In the present invention, each search keyword corresponds to a search page. The search page is a web page obtained by searching with the search keyword. The search page contains several links, and each link corresponds to a sub-page. The present invention extracts the search information contained in each search page and sub-page through web crawler technology and text parsing technology, and each search information corresponds to a release time. The release time of a single search information is the time when the search information is uploaded to the network. The enterprise name includes, but is not limited to, XX Company, XX Factory, and XX Center. This is content that is easily understood by those skilled in the art and will not be elaborated specifically. The search information contains data keywords corresponding to each index keyword that is the same as the training data;

[0111] The confirmation method of the effective evaluation index includes:

[0112] If the number of links is less than the preset number of links, the effective evaluation index is determined according to the link depth value. Among them, the effective evaluation index and the link depth value are in a positive correlation relationship;

[0113] If the number of links is greater than or equal to the preset number of links, the effective evaluation index is determined according to the area ratio value. Among them, the effective evaluation index and the area ratio value are in a positive correlation relationship;

[0114] The number of links is the total number of links in the search page corresponding to the target search keyword. The value of the preset number of links can be determined by the user according to the actual application scenario. The larger the value of the preset number of links, the greater the need for the user to determine the effective evaluation index according to the link depth value. Provide a value for the preset number of links. The preset number of links is 10;

[0115] Record the search page corresponding to the target search keyword as the target page. Establish a tree structure diagram based on the target page. Conduct node analysis on the target page, extract all sub-pages corresponding to the target page, create several new sub-nodes for each sub-page, connect each sub-node to the root node, conduct node analysis on each sub-page, and connect each sub-page to the corresponding sub-node until the page threshold of the sub-page is 0, then stop the node analysis. Finally, establish a tree structure diagram with the target page as the root node and each sub-page as the sub-node. This is content that is easy for those skilled in the art to understand and will not be elaborated here;

[0116] The link depth value is the number of all sub-nodes in the tree structure diagram established based on the target page. The area ratio value is the area of the overlapping part between the first area and the second area. The first area is the smallest rectangular area that can contain each enterprise name in the search page corresponding to the target search keyword, and the second area is the smallest rectangular area that can contain each link in the search page corresponding to the target search keyword;

[0117] The values of the preset data richness and the preset effective evaluation index can be determined by the user according to the actual application scenario. The larger the values of the preset data richness and the preset effective evaluation index, the greater the user's need for matching searches based on the keyword correlation degree and the link evaluation value. Provide a set of values for the preset data richness and the preset effective evaluation index, detect the historical records of matching searches based on the keyword correlation degree and the link evaluation value, and record the average value of the data richness corresponding to the historical records that can meet the user's needs as the preset data richness, and record the average value of the effective evaluation index corresponding to the historical records that can meet the user's needs as the preset effective evaluation index;

[0118] Conduct matching searches based on the keyword correlation degree and the link evaluation value, including: for a single search keyword, record this search keyword as the target keyword, record the other search keywords except the target keyword as reference keywords, select the reference keyword with the highest keyword correlation degree with the target keyword and the target keyword to conduct searches simultaneously to obtain the target search page, and determine the priority coefficient of each link according to the link evaluation value of the target search page. The priority coefficient of a single link has a positive correlation with the link evaluation value corresponding to this link;

[0119] The link evaluation value corresponding to a single link is the number of enterprise names contained in the sub-page corresponding to this link;

[0120] Conduct separate searches according to the link validity, including: select links with a link validity greater than the preset link validity for searches;

[0121] Link effectiveness = (connectivity coefficient / preset connectivity coefficient) + (link evaluation value / preset link evaluation value). For a single link in a single target search page, mark this link as the target link, and mark the other links in the target search page except the target link as reference links. The connectivity coefficient corresponding to the target link is the average value of the shortest distances from the center point corresponding to the target link to the center points corresponding to each reference link. The center point corresponding to a single link is the center of the circumscribed circle of the smallest rectangle that can contain the single link; the preset connectivity coefficient is the average value of the connectivity coefficients corresponding to each link in the target search page, and the preset link evaluation value is the average value of the link evaluation values corresponding to each link in the target search page;

[0122] For any two search keywords, keyword relevance = (the number of training data in which the two search keywords appear simultaneously / the number of training data corresponding to the target enterprise credit model) + combined record reference value. The confirmation method of the combined record reference value is to detect the historical records that can meet the user's needs by matching and searching according to the keyword relevance and the link evaluation value, and mark them as reference historical records. The combined record reference value = the number of reference historical records that select the two search keywords for search / the total amount of reference historical records;

[0123] The value of the preset link effectiveness can be determined by the user according to the actual application scenario. The greater the user's demand for improving the search efficiency, the smaller the value of the preset link effectiveness. Provide a value of the preset link effectiveness. Mark the historical records that can meet the user's needs by performing a single search according to the link effectiveness as reference records, and mark the average value of the link effectiveness corresponding to the links selected for search in the reference records as the preset link effectiveness.

[0124] Specifically, determine the supplementation method according to the number of effective search data in the associated search data combination and the dislocation coefficient, including:

[0125] For a single associated search data combination,

[0126] If the number of effective search data is greater than the standard number and the dislocation coefficient is greater than or equal to the preset dislocation coefficient, the supplementation method is to select supplementary data according to the law representativeness and the data dislocation value;

[0127] If the number of effective search data is equal to the standard number or the dislocation coefficient is less than the preset dislocation coefficient, the supplementation method is to determine whether to use the search data as supplementary data according to the anomaly index.

[0128] Among them, the number of valid search data is the number of search data in a single associated search data combination. The dislocation coefficient is the maximum value among the sub-dislocation coefficients corresponding to each special index keyword in a single associated search data combination. The sub-dislocation coefficient is the standard deviation of the values of the data keywords corresponding to this special index keyword in this associated search data combination. The standard quantity is 1;

[0129] For the value of the preset dislocation coefficient, the user can determine it according to the actual application scenario. The smaller the value of the preset dislocation coefficient, the greater the user's need to select supplementary data according to the law representativeness and the data dislocation value. Provide a value of the preset dislocation coefficient, detect the historical records of selecting supplementary data according to the law representativeness and the data dislocation value, and record the average value of the dislocation coefficients corresponding to the historical records that can meet the user's needs as the preset dislocation coefficient;

[0130] Selecting supplementary data according to the law representativeness and the data dislocation value includes: using the search data with a law representativeness greater than the preset law representativeness and a data dislocation value less than the preset data dislocation value as supplementary data;

[0131] Determining whether to use the search data as supplementary data according to the anomaly index includes:

[0132] If the anomaly index is greater than or equal to the preset anomaly index, then the search data is not used as supplementary data;

[0133] If the anomaly index is less than the preset anomaly index, then the search data is used as supplementary data;

[0134] The confirmation method of the law representativeness is that for a single search data, the average value of the sub-representativeness corresponding to each data keyword included in this search data is recorded as the law representativeness corresponding to this search data. The sub-representativeness corresponding to a single data keyword is the number of training data including this data keyword;

[0135] The confirmation method of the data dislocation value is that for a single search data, the values of the data keywords corresponding to each special index keyword in this search data are recorded as the analysis values. The data dislocation value is the maximum value among the mutation rates corresponding to each analysis value. For a single analysis value, the mutation rate corresponding to this analysis value = |analysis value - the mean coefficient corresponding to this analysis value| / analysis value. The other search data in the same associated search data combination as this search data is recorded as the analysis search data. For a single analysis value, the mean coefficient corresponding to this analysis value is the average value of the values of the data keywords corresponding to the special index keyword corresponding to this analysis value in each analysis search data;

[0136] The confirmation method of the anomaly index is as follows: Detect the historical records that determine whether to use the search data as supplementary data according to the anomaly index and cannot meet the user's needs, and record them as anomaly historical records. For each data keyword corresponding to a single search data, the average value of the anomaly rates corresponding to each data keyword is recorded as the anomaly index. The anomaly rate corresponding to a single data keyword = the number of anomaly historical records containing this data keyword / the total amount of anomaly historical records;

[0137] The values of the preset law representativeness, preset data dislocation value, and preset anomaly index can be determined by the user according to the actual application scenario. The greater the user's demand for improving the model optimization accuracy, the greater the values of the preset law representativeness and preset anomaly index, and the smaller the value of the preset data dislocation value. Provide a set of values for the preset law representativeness, preset data dislocation value, and preset anomaly index. Detect the historical records of selecting supplementary data according to the law representativeness and data dislocation value, and record the supplementary data selected from the historical records that can meet the user's needs as the reference supplementary data. The average value of the law representativeness corresponding to each reference supplementary data is recorded as the preset law representativeness. The preset data dislocation value is 20%, and the preset anomaly index is 40%.

[0138] Specifically, the confirmation method of the associated search data combination includes:

[0139] For a single enterprise name, each search data corresponding to this enterprise name within the corresponding retrospective range is used as the associated search data combination corresponding to this enterprise name;

[0140] The retrospective range corresponding to a single enterprise name is determined according to the dynamic comparison coefficient and the in-domain correlation degree corresponding to this enterprise name.

[0141] Among them, the confirmation method of the retrospective range is as follows: For a single enterprise name, detect the release time corresponding to each search data corresponding to this enterprise name, and record the earliest release time as the target time point. The retrospective range is the time range between the retrospective time point and the current time point; the retrospective time point is any time point within the interval [target time point, current time point); the retrospective range and the retrospective reference value are in a positive correlation relationship, and the retrospective reference value = dynamic comparison coefficient - in-domain correlation degree;

[0142] The dynamic comparison coefficient = rating change degree + level comparison value;

[0143] The confirmation method of the rating change degree is as follows: For a single enterprise name, record each search data corresponding to this enterprise name as the same-name data, and record the same-name data with the same level keyword as the same-level combination. The rating change degree = 1 - (the maximum value in the quantity reference values corresponding to each same-level combination / the total amount of the same-name data). The quantity reference value corresponding to a single same-level combination is the number of search data included in this same-level combination;

[0144] Level comparison value = (the maximum value among the level change rates corresponding to each same-level combination) - (the minimum value among the level change rates corresponding to each same-level combination); the level change rate corresponding to a single same-level combination is the average value of the fluctuation coefficients corresponding to each special index keyword, and the fluctuation coefficient corresponding to a single special index keyword is the standard deviation of the numerical values of the data keywords corresponding to this special index keyword in this same-level combination;

[0145] For a single search data, record the numerical values of the data keywords corresponding to each special index keyword in this search data as analysis numerical values. The data dislocation value is the maximum value among the mutation rates corresponding to each analysis numerical value. For a single analysis numerical value, the mutation rate corresponding to this analysis numerical value = |analysis numerical value - the mean coefficient corresponding to this analysis numerical value| / analysis numerical value. Denote the other search data in the same associated search data combination as the search data as analysis search data. For a single analysis numerical value, the mean coefficient corresponding to this analysis numerical value is the average value of the numerical values of the data keywords corresponding to the special index keyword corresponding to this analysis numerical value in each analysis search data;

[0146] The in-domain correlation degree = 1 / (the maximum value among the floating coefficients corresponding to each special index keyword - the minimum value among the floating coefficients corresponding to each special index keyword). The floating coefficient corresponding to a single special index keyword = (the absolute value of the difference between the maximum value and the minimum value of the numerical values of the data keywords corresponding to this special index keyword in each same-name data) / (the maximum value of the numerical values of the data keywords corresponding to this special index keyword in each same-name data).

[0147] Specifically, determine the adjustment method according to the correlation coefficient and the comparison coefficient of the supplementary data, including:

[0148] If the correlation coefficient is greater than the preset correlation coefficient and the comparison coefficient is less than the preset comparison coefficient, the adjustment method is to increase the adjustment for the search depth;

[0149] If the correlation coefficient is greater than the preset correlation coefficient and the comparison coefficient is greater than or equal to the preset comparison coefficient, the adjustment method is to increase the adjustment for the backtracking range.

[0150] Among them, it should be noted that when the correlation coefficient is less than or equal to the preset correlation coefficient, no adjustment is made for the search depth and the backtracking range. The correlation coefficient is the average value of the sub-correlation degrees corresponding to each supplementary data. For a single supplementary data, denote this supplementary data as the target supplementary data, and denote the other supplementary data except the target supplementary data as the reference supplementary data. The sub-correlation degree is the average value of the number of same words corresponding to each reference supplementary data. The number of same words corresponding to a single reference supplementary data is the number of data keywords in this reference supplementary data that are the same as the target keyword;

[0151] Comparison coefficient = 1 / (average of the in-domain correlation degrees of the enterprise names corresponding to each supplementary data);

[0152] For the values of the preset correlation coefficient and the preset comparison coefficient, users can determine them according to the actual application scenario. The smaller the value of the preset correlation coefficient and the larger the value of the preset comparison coefficient, the greater the user's demand for increasing the search depth. Provide a set of values for the preset correlation coefficient and the preset comparison coefficient. Detect the historical records of increasing the search depth, and record the average value of the correlation coefficients corresponding to the historical records that can meet the user's needs as the preset correlation coefficient, and record the average value of the comparison coefficients corresponding to the historical records that can meet the user's needs as the preset comparison coefficient;

[0153] When adjusting to increase the search depth, the increase value of the search depth has a positive correlation with the correlation coefficient;

[0154] When adjusting to increase the backtracking range, the increase value of the backtracking range has a positive correlation with the correlation coefficient;

[0155] The search depth is the time for web crawling the search pages corresponding to each search keyword. It should be noted that in the initial crawling process of the present invention, the search depth corresponding to a single search keyword has a negative correlation with the data richness corresponding to that search keyword; when adjusting to increase the backtracking range, the interval between the selected backtracking time point and the target time point gradually decreases.

[0156] Specifically, eliminating the noise indicators according to the index fluctuation coefficient and the index offset rate includes:

[0157] Mark the special index keywords with an index fluctuation coefficient less than the preset index fluctuation coefficient and an index offset rate less than the preset index offset rate as noise indicators, and eliminate the noise indicators and each data keyword corresponding to the noise indicators.

[0158] Among them, the index fluctuation coefficient corresponding to a single special index keyword = (the sub-equilibrium coefficient corresponding to the special index keyword) / (the average value of the numerical values of each data keyword corresponding to the special index keyword);

[0159] For a single special index keyword, the index offset rate corresponding to the special index keyword = the sub-correlation mean value corresponding to the special index keyword - the correlation reference value. Record the historical records that can meet the user's needs and determine the elimination method according to the data smoothing ratio as the comparison historical records, and record the average value of the sub-correlation mean values of the special index keywords eliminated in the comparison historical records as the correlation reference value;

[0160] The values of the preset index fluctuation coefficient and the preset index offset rate can be determined by the user according to the actual application scenario. The greater the user's demand for the model training accuracy, the greater the values of the preset index fluctuation coefficient and the preset index offset rate. Provide the values of the preset index fluctuation coefficient and the preset index offset rate, detect the historical records of eliminating the noise index and each data keyword corresponding to the noise index, and record the average value of the index fluctuation coefficients corresponding to each noise index in each historical record that can meet the user's needs as the preset index fluctuation coefficient, and record the average value of the index offset rates corresponding to each noise index in each historical record that can meet the user's needs as the preset index offset rate.

[0161] Please refer to Figure 4 as shown, which is the module connection diagram of the enterprise credit model optimization system based on big data of the present invention. The present invention also provides an enterprise credit model optimization system based on big data, including:

[0162] A data acquisition module for obtaining the training data corresponding to the target enterprise credit model;

[0163] An optimization analysis module, which is connected to the data acquisition module, for determining the model data status according to the coverage integrity and data singularity of the training data, and determining the optimization analysis method according to the model data status. The optimization analysis method is data supplement analysis or data elimination analysis;

[0164] A data supplement module, which is respectively connected to the data acquisition module and the optimization analysis module, for, in the data supplement analysis, determining the search keywords according to the combined data ratio, determining the search methods corresponding to each search keyword according to the data richness and the effective evaluation index to obtain the search data, and determining the supplement method according to the number of effective search data in the associated search data combination and the dislocation coefficient;

[0165] A supplement adjustment module, which is respectively connected to the data acquisition module and the data supplement module, for determining the adjustment method according to the correlation coefficient and the comparison coefficient of the supplement data. The adjustment method is to adjust the search depth or the backtracking range;

[0166] A data elimination module, which is connected to the optimization analysis module, for eliminating the noise index according to the index fluctuation coefficient and the index offset rate in the data elimination analysis;

[0167] A model optimization module, which is respectively connected to the data supplement module, the supplement adjustment module and the data elimination module, for training the target enterprise credit model by using the optimized data.

[0168] So far, the technical solution of the present invention has been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easily understood by those skilled in the art that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

[0169] The above are only the preferred embodiments of the present invention and are not used to limit the present invention; for those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for optimizing enterprise credit model based on big data, characterized in that: include: Obtain the training data corresponding to the target enterprise's credit model, determine the model data status based on the coverage completeness and data uniqueness of the training data, and determine the optimization analysis method based on the model data status. The optimization analysis method is data supplementation analysis or data elimination analysis; In the data supplementation analysis, the search keywords are determined according to the proportion of combined data, the search method corresponding to each search keyword is determined according to the data richness and effective evaluation index to obtain search data, and the supplementation method is determined according to the number of effective search data and the dislocation coefficient of the associated search data combination; Determine the adjustment method according to the correlation coefficient and comparison coefficient of the supplementary data, and the adjustment method is to adjust the search depth or the backtracking range; The search method is to perform a matching search based on the keyword association and the link evaluation value or to perform a separate search based on the link validity, and the supplementary method is to select supplementary data based on the regularity representativeness and the data deviation value or to determine whether to use the search data as supplementary data based on the abnormality index; In the data elimination analysis, noise indicators are eliminated according to the indicator volatility coefficient and indicator bias rate; Use optimized data to train the target enterprise credit model.

2. The enterprise credit model optimization method based on big data according to claim 1 is characterized in that: Determine the optimization analysis method based on the model data status, including: If the model data status is that the coverage completeness is less than the preset coverage completeness or the data singleness is greater than or equal to the preset data singleness, the optimization analysis method is data supplementation analysis; If the model data status is that the coverage completeness is greater than or equal to the preset coverage completeness and the data singleness is less than the preset data singleness, the optimization analysis method is data elimination analysis.

3. The enterprise credit model optimization method based on big data according to claim 2 is characterized in that: Methods for confirming data uniqueness include: If the data volume reference value is greater than or equal to the preset data volume reference value, the data uniformity is determined according to the ratio of the equalization coefficient to the data volume reference value; If the data volume reference value is less than the preset data volume reference value, the data uniformity is determined according to the data correlation.

4. The enterprise credit model optimization method based on big data according to claim 2 is characterized in that: Determine the search keywords based on the proportion of combined data, including: Determine the associated data combination based on the industry keywords, and perform search keyword selection analysis for each associated data combination. When performing search keyword selection analysis for a single associated data combination, If the combined data ratio is greater than or equal to the preset combined data ratio, the search keyword is determined according to the indicator comparison coefficient and the indicator distribution ratio; If the proportion of combined data is less than the preset proportion of combined data, the search keyword is determined according to the level shift coefficient.

5. The enterprise credit model optimization method based on big data according to claim 4 is characterized in that: Determine the search method corresponding to each search keyword based on data richness and effective evaluation index, including: For a single search keyword, If the data richness is less than the preset data richness or the effective evaluation index is less than the preset effective evaluation index, the search method is to perform a matching search based on the keyword relevance and link evaluation value; If the data richness is greater than or equal to the preset data richness and the effective evaluation index is greater than or equal to the preset effective evaluation index, the search method is to perform a separate search based on the link validity.

6. The enterprise credit model optimization method based on big data according to claim 5 is characterized in that: The supplement method is determined according to the number of valid search data of the associated search data combination and the shift coefficient, including: Search data combinations for a single association, If the number of valid search data is greater than the standard number and the shift coefficient is greater than or equal to the preset shift coefficient, the supplementary method is to select supplementary data according to the regular representativeness and the data shift value; If the number of valid search data is equal to the standard number or the error coefficient is less than the preset error coefficient, the supplementary method is to determine whether to use the search data as supplementary data based on the abnormality index.

7. The enterprise credit model optimization method based on big data according to claim 6 is characterized in that: The confirmation method of the associated search data combination includes: For a single enterprise name, each search data corresponding to the enterprise name within the traceback range corresponding to the enterprise name is used as a combination of associated search data corresponding to the enterprise name; The traceability range corresponding to a single enterprise name is determined based on the dynamic comparison coefficient corresponding to the enterprise name and the degree of correlation within the domain.

8. The enterprise credit model optimization method based on big data according to claim 7 is characterized in that: The adjustment method is determined based on the correlation coefficient and comparison coefficient of the supplementary data, including: If the correlation coefficient is greater than the preset correlation coefficient and the comparison coefficient is less than the preset comparison coefficient, the adjustment method is to increase the search depth; If the correlation coefficient is greater than the preset correlation coefficient and the comparison coefficient is greater than or equal to the preset comparison coefficient, the adjustment method is to increase the backtracking range.

9. The enterprise credit model optimization method based on big data according to claim 2 is characterized in that: Noise indicators are eliminated based on the indicator volatility coefficient and indicator bias rate, including: Special indicator keywords whose indicator volatility coefficient is less than the preset indicator volatility coefficient and whose indicator bias rate is less than the preset indicator bias rate are recorded as noise indicators, and the noise indicators and the data keywords corresponding to the noise indicators are eliminated.

10. An optimization system using the enterprise credit model optimization method based on big data as described in any one of claims 1 to 9, characterized in that: include: Data collection module, used to obtain training data corresponding to the target enterprise credit model; An optimization analysis module, which is connected to the data acquisition module, is used to determine the model data state according to the coverage completeness and data uniformity of the training data, and to determine the optimization analysis method according to the model data state, and the optimization analysis method is data supplementation analysis or data elimination analysis; A data supplementation module, which is connected to the data acquisition module and the optimization analysis module respectively, and is used to determine the search keywords according to the proportion of combined data in the data supplementation analysis, determine the search method corresponding to each search keyword according to the data richness and the effective evaluation index to obtain the search data, and determine the supplementation method according to the number of effective search data of the associated search data combination and the shift coefficient; A supplementary adjustment module, which is connected to the data acquisition module and the data supplementary module respectively, and is used to determine an adjustment method according to the correlation coefficient and the comparison coefficient of the supplementary data, and the adjustment method is to adjust the search depth or the backtracking range; A data elimination module, which is connected to the optimization analysis module and is used to eliminate noise indicators according to the indicator fluctuation coefficient and the indicator bias rate in the data elimination analysis; The model optimization module is respectively connected to the data supplementation module, the supplementation adjustment module and the data elimination module, and is used to train the target enterprise credit model with optimized data.

Images