Risk assessment analysis method and system based on big data

Abstract

The invention belongs to the technical field of computer data processing, and provides a risk assessment analysis method and system based on big data, and the method comprises the steps: collecting financial risk assessment related data according to a preset data collection strategy, and constructing a Hadoop-based financial big data architecture; grouping the financial big data through a factor analysis method to obtain a plurality of factors, and obtaining a financial risk assessment value according to the factors and the variance contribution rates corresponding to the factors; automatically generating a financial risk assessment report by using a sequence-to-sequence method according to the financial risk assessment value; and carrying out visual display on the financial risk assessment report in combination with a preset interface layout principle. According to the method, the risk value of the financial project can be quickly and accurately calculated, and then a reliable decision basis is provided for a decision maker.

Description

Technical Field

[0001] The present application relates to the application of big data technology in the field of financial risk, and specifically to a method and system for comprehensive assessment and analysis of financial risks based on big data. Background Art

[0002] With the rapid expansion of financial services and the increasing variety of financial products, the demand for purchasing financial products is also increasing. At the same time, various financial institutions have launched a wide variety of credit products. The development of loan business has made competition more and more fierce, and the scenarios of credit business have become more and more complicated. When handling financial services such as credit loans, in order to reduce the bad debt rate, financial institutions need to conduct risk assessments on the financial services to be handled.

[0003] At present, financial business risk assessment is carried out in the business system. The iteration of risk strategy requires a lot of business system coding development. The business system not only has to handle business, but also needs to conduct risk assessment. The workload is large and the assessment efficiency is low, which can easily cause the business system to be stuck. Not only can it not respond quickly to changes in risks, but it can also not conduct comprehensive and effective financial risk assessment. Summary of the invention

[0004] In view of this, an embodiment of the present application provides a risk assessment and analysis method and system based on big data to solve the above technical problems.

[0005] A first aspect of an embodiment of the present application provides a risk assessment and analysis method based on big data, comprising:

[0006] According to the preset data collection strategy, collect financial risk assessment related data and build a financial big data architecture based on Hadoop;

[0007] The financial big data is grouped by factor analysis to obtain several factors, and the financial risk assessment value is obtained based on the factors and their corresponding variance contribution rates;

[0008] According to the financial risk assessment value, the financial risk assessment report is automatically generated using the sequence-to-sequence method;

[0009] The financial risk assessment report is combined with the preset interface layout principles for visual display.

[0010] Furthermore, according to the preset data collection strategy, the financial risk assessment related data is collected, and a financial big data architecture based on Hadoop is constructed, wherein the preset data collection strategy includes internal data integration, external data acquisition, and cooperative and shared data access.

[0011] Furthermore, the external data is obtained by screening out target web pages based on a genetic algorithm according to the relevance and importance of the web pages to be crawled and the financial risk assessment theme, and crawling the financial risk assessment related data of the target web pages.

[0012] Furthermore, the method of screening out target web pages based on a genetic algorithm according to the relevance and importance of the web pages to be crawled and the financial risk assessment theme, and crawling the financial risk assessment related data of the target web pages, includes the following steps:

[0013] Step 11, obtaining a queue of web page URLs to be crawled;

[0014] Step 12, downloading and parsing web pages one by one, and calculating the relevance value between the web pages and the financial risk assessment topic according to the keywords and keyword weights in the web pages;

[0015] Step 13, performing a selection operation based on a genetic algorithm to screen out web pages with relevance values ​​higher than a preset threshold, and forming a set A;

[0016] Step 14, performing a crossover operation based on a genetic algorithm to obtain hyperlinks contained in all web pages in set A, sorting them by topic importance, and selecting the top links to update set A;

[0017] Step 15, performing mutation operation based on genetic algorithm, querying and splicing subject keywords through search engine, selecting the top result links, and forming set B;

[0018] Step 16, performing a merging operation based on a genetic algorithm so that the set C = A + B is inherited to the next generation;

[0019] Step 17, determine whether the preset termination condition is met, if yes, output set C, if no, return to step 11;

[0020] Step 18, crawl the financial risk assessment related data of the web pages in set C.

[0021] Furthermore, the web pages are downloaded and parsed one by one, and the relevance value between the web pages and the financial risk assessment topic is calculated according to the keywords and keyword weights in the web pages, wherein the calculation formula is:

[0022]

[0023] In the above formula, Sim(p) represents the correlation value, η n represents the absolute frequency of the n-th dimension keyword in the web page, α n Represents the weight of the n-th dimension keyword.

[0024] Furthermore, the financial big data is grouped by factor analysis to obtain several factors, and the financial risk assessment value is obtained according to the factors and their corresponding variance contribution rates, which specifically include:

[0025] Obtain the correlation coefficient matrix based on financial big data, and remove those with correlation coefficients less than the preset value;

[0026] Calculate the factor loading matrix based on the correlation coefficient matrix, and construct factor variables based on the factor loading matrix;

[0027] Calculate the factor variable score for each sample, and obtain the financial risk assessment value based on the factor variable score and variance contribution rate.

[0028] Furthermore, the financial risk assessment report is automatically generated based on the financial risk assessment value and using a sequence-to-sequence method, specifically:

[0029] Extract the features of the financial risk assessment value as an initial sequence through an extractor;

[0030] After encoding and decoding the initial sequence, a financial risk assessment report is obtained.

[0031] Furthermore, the financial risk assessment report is visualized in combination with the preset interface layout principles, including:

[0032] Combining objective cognitive laws with user subjective cognition, we obtain the interface layout principles, and transform the interface layout principles into a computable multi-objective optimization function for the interface layout.

[0033] The target optimization function is encoded through a preset algorithm to seek the optimal spatial solution, and then the optimal interface layout solution for the financial risk assessment report is obtained.

[0034] Furthermore, the objective cognition law is combined with the user's subjective cognition to obtain the interface layout principle, and the interface layout principle is converted into a computable interface layout multi-objective optimization function, wherein the objective optimization function is:

[0035]

[0036] In the above formula, S i Indicates the order of operations, P i Indicates importance and operation frequency, O i represents the correlation principle, ω 1i and ω 2i Denotes the position control coefficient, d ia is the distance between the position coordinates of the geometric center of element i and the optimal layout point a, d amax Indicates the maximum distance between the element position point in the area to be laid out and the optimal layout point a.

[0037] A second aspect of the embodiment of the present application provides a risk assessment and analysis system based on big data, including:

[0038] The collection module is used to collect financial risk assessment related data according to the preset data collection strategy and build a financial big data architecture based on Hadoop;

[0039] The calculation module is used to group the financial big data through factor analysis to obtain several factors, and obtain the financial risk assessment value based on the factors and their corresponding variance contribution rates;

[0040] A reporting module, which is used to automatically generate a financial risk assessment report based on the financial risk assessment value and using a sequence-to-sequence method;

[0041] The display module is used to visualize the financial risk assessment report in combination with the preset interface layout principles.

[0042] A third aspect of an embodiment of the present application provides a terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in the first aspect when executing the computer program.

[0043] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0044] A fifth aspect of the present application provides a computer program product, which, when executed on a terminal, enables the terminal to execute the steps of the method described in the first aspect.

[0045] As can be seen from the above, the embodiment of the present application constructs a financial big data architecture based on Hadoop, groups the financial big data through factor analysis, and then calculates the financial risk assessment value. It can not only comprehensively acquire and process the data related to the financial risk assessment, making the basic data more accurate and reliable, but also quickly obtain the financial risk data, effectively reduce the amount of calculation, save calculation costs, and then quickly respond to risk changes. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0047] Figure 1 It is a flowchart of a risk assessment and analysis method based on big data provided in an embodiment of the present application;

[0048] Figure 2 It is a structural diagram of a risk assessment and analysis system based on big data provided in an embodiment of the present application;

[0049] Figure 3 It is a structural diagram of a terminal provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0051] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0052] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0053] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0054] As used in this specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0055] In a specific implementation, the terminal described in the embodiments of the present application includes, but is not limited to, other portable devices such as mobile phones, laptop computers, or tablet computers with touch-sensitive surfaces (e.g., touch screen displays and / or touch pads). It should also be understood that in some embodiments, the device is not a portable communication device, but a desktop computer with a touch-sensitive surface (e.g., touch screen displays and / or touch pads).

[0056] In the following discussion, a terminal including a display and a touch-sensitive surface is described. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, mouse and / or joystick.

[0057] Various applications that can be executed on the terminal can use at least one common physical user interface device such as a touch-sensitive surface. One or more functions of the touch-sensitive surface and corresponding information displayed on the terminal can be adjusted and / or changed between applications and / or within corresponding applications. In this way, the common physical architecture of the terminal (e.g., the touch-sensitive surface) can support various applications with user interfaces that are intuitive and transparent to the user.

[0058] It should be understood that the size of the serial numbers of each step in this embodiment does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of this application.

[0059] In order to illustrate the technical solution described in this application, a specific embodiment is provided below for illustration.

[0060] See also Figure 1 , Figure 1 Schematic diagram of a risk assessment and analysis method based on big data provided in the embodiment of the present application. Figure 1 As shown, a risk assessment and analysis method based on big data includes the following steps:

[0061] Step 1: Collect financial risk assessment related data according to the preset data collection strategy and build a financial big data architecture based on Hadoop;

[0062] Step 2: Group the financial big data by factor analysis to obtain several factors, and obtain the financial risk assessment value based on the factors and their corresponding variance contribution rates;

[0063] Step 3, automatically generating a financial risk assessment report based on the financial risk assessment value and using a sequence-to-sequence method;

[0064] Step 4: Visually display the financial risk assessment report in combination with the preset interface layout principles to facilitate decision-making.

[0065] Obviously, the embodiment of the present application constructs a financial big data architecture based on Hadoop, groups the financial big data through factor analysis, and then calculates the financial risk assessment value. It can not only comprehensively acquire and process the data related to the financial risk assessment, making the basic data more accurate and reliable, but also quickly obtain the financial risk data, effectively reduce the amount of calculation, save the calculation cost, and then quickly respond to the risk changes.

[0066] In the embodiment of the present application, step 1 is to collect financial risk assessment related data according to a preset data collection strategy, and build a financial big data architecture based on Hadoop, wherein the preset data collection strategy includes internal data integration, external data acquisition, and cooperative and shared data access;

[0067] For example, internal data integration can be the integration of data resources of internal business systems, risk control systems, project management systems, etc.

[0068] External data acquisition can be done by purchasing third-party data, crawling web data, etc., to obtain external data such as financial market trends, target object information, policy changes, etc.

[0069] Cooperation and shared data access can be to establish partnerships with other institutions, build cloud storage space stations, and achieve data sharing to make the data more comprehensive;

[0070] Specifically, in order to quickly, accurately and comprehensively obtain financial risk assessment related data from massive web page data, the target web pages can be screened out based on the genetic algorithm according to the relevance and importance of the web pages to be crawled and the financial risk assessment theme, and the financial risk assessment related data of the target web pages can be crawled;

[0071] More specifically, according to the relevance and importance of the web pages to be crawled and the financial risk assessment topic, the target web pages are screened out based on the genetic algorithm, and the financial risk assessment related data of the target web pages are crawled, including the following steps:

[0072] Step 11, obtaining a queue of web page URLs to be crawled;

[0073] Step 12, download and parse the web pages one by one, and calculate the relevance value between the web pages and the financial risk assessment topic according to the keywords and keyword weights in the web pages, wherein the calculation formula can be expressed by the following formula:

[0074]

[0075] In the above formula, Sim(p) represents the correlation value, η n represents the absolute frequency of the n-th dimension keyword in the web page, α n Indicates the weight of the n-th dimension keyword;

[0076] Step 13, performing a selection operation based on a genetic algorithm to screen out web pages with relevance values ​​higher than a preset threshold, and forming a set A;

[0077] Step 14, performing a crossover operation based on a genetic algorithm to obtain hyperlinks contained in all web pages in set A, sorting them by topic importance, and selecting the top links to update set A;

[0078] Step 15, performing mutation operation based on genetic algorithm, querying and splicing subject keywords through search engine, selecting the top result links, and forming set B;

[0079] Step 16, performing a merging operation based on a genetic algorithm so that the set C = A + B is inherited to the next generation;

[0080] Step 17, determine whether the preset termination condition is met, if yes, output set C, if no, return to step 11;

[0081] Step 18, crawl the financial risk assessment related data of the web pages in set C.

[0082] As we all know, general search engines can provide a large amount of Internet data, but they will also drown out the truly valuable data. In the embodiment of the present application, a subject crawler is used, that is, based on the target subject, subject-related web pages are collected, thereby effectively improving the importance and subject relevance of the crawled pages, and effectively improving the reliability and accuracy of the data. At the same time, combined with a genetic algorithm, the integrity and comprehensiveness of the data can be effectively improved.

[0083] In the embodiment of the present application, step 2, grouping the financial big data by factor analysis to obtain a number of factors, and obtaining the financial risk assessment value according to the factors and their corresponding variance contribution rates, specifically includes the following steps:

[0084] Step 21, obtaining a correlation coefficient matrix based on the financial big data, and eliminating those with correlation coefficients less than a preset value (e.g., 0.3);

[0085] Step 22, calculate the factor loading matrix according to the correlation coefficient matrix, and construct the factor variables according to the factor loading matrix;

[0086] Step 23, calculate the factor variable score of each sample, and obtain the financial risk assessment value based on the factor variable score and variance contribution rate. The calculation formula can be expressed as follows:

[0087] F=α1F1+α2F2+…+α m F m

[0088] In the above formula, α i Fi The variance contribution rate of i Scores for factor variables.

[0089] Since the factor analysis method can simplify complex original data into a few comprehensive factors and retain the explanatory power of the original data as much as possible, the embodiment of the present application obtains the financial risk assessment value through the factor analysis method, which helps to improve the accuracy of financial risk prediction, and the prediction effect in the period of rising financial risk is better than the prediction effect in the period of falling financial risk.

[0090] In the embodiment of the present application, step 3, based on the financial risk assessment value, automatically generates a financial risk assessment report using a sequence-to-sequence method, which may specifically be:

[0091] The features of the financial risk assessment value are extracted by the extractor as the initial sequence X, where the initial sequence X can be expressed by the following formula:

[0092] X={x1,x2,...,x N}, x n ∈R d

[0093] In the above formula, x n is the nth patch feature extracted, and d is the size of the mapped feature vector;

[0094] After encoding and decoding the initial sequence X, the financial risk assessment report Y is obtained, where the financial risk assessment report Y can be expressed by the following formula:

[0095] Y={y1,y2,..,y B},y b ∈S

[0096] In the above formula, y b Represents the tensor corresponding to the bth word in the generated sequence, and S represents the set of all words in the generated report.

[0097] Obviously, since the extractor is set to extract the features and obtain the financial risk assessment report after decoding, the quality of the generated report is more credible and reliable.

[0098] In the embodiment of the present application, step 4, visually displaying the financial risk assessment report in combination with the preset interface layout principle to facilitate auxiliary decision-making, may include the following steps:

[0099] Step 4.1, combine the objective cognitive law with the user's subjective cognition to obtain the interface layout principle, and transform the interface layout principle into a computable interface layout multi-objective optimization function, where the objective optimization function can be expressed by the following formula:

[0100]

[0101] In the above formula, S i Indicates the order of operations, P i Indicates importance and operation frequency, O i represents the correlation principle, ω 1i and ω 2i Denotes the position control coefficient, d ia is the distance between the position coordinates of the geometric center of element i and the optimal layout point a, d amax Indicates the maximum distance between the element position point in the area to be laid out and the optimal layout point a;

[0102] Step 4.2, encode the target optimization function through a preset algorithm, seek the optimal spatial solution, and then obtain the optimal interface layout solution for the financial risk assessment report.

[0103] Obviously, by combining the objective cognitive laws with the user's subjective cognition, we obtain the interface layout principles, and transform the interface layout principles into a computable interface layout multi-objective optimization function. Then, we obtain the optimal interface layout by calculating the objective optimization function, which can effectively accommodate the user's subjective feelings and thus effectively improve the user experience.

[0104] Based on the same inventive concept, the present application embodiment provides a risk assessment and analysis system based on big data, such as Figure 2 As shown, including:

[0105] Collection module 1, used to collect financial risk assessment related data according to the preset data collection strategy, and build a financial big data architecture based on Hadoop;

[0106] Calculation module 2 is used to group the financial big data by factor analysis to obtain a number of factors, and obtain the financial risk assessment value according to the factors and their corresponding variance contribution rates;

[0107] Report module 3, used for automatically generating a financial risk assessment report based on the financial risk assessment value and using a sequence-to-sequence method;

[0108] The display module 4 is used to visualize the financial risk assessment report in combination with the preset interface layout principles to facilitate auxiliary decision-making.

[0109] Obviously, the embodiment of the present application constructs a financial big data architecture based on Hadoop, groups the financial big data through factor analysis, and then calculates the financial risk assessment value. It can not only comprehensively acquire and process the data related to the financial risk assessment, making the basic data more accurate and reliable, but also quickly obtain the financial risk data, effectively reduce the amount of calculation, save the calculation cost, and then quickly respond to the risk changes.

[0110] Figure 3 is a structural diagram of a terminal provided in an embodiment of the present application. As shown in the figure, the terminal 3 of the embodiment includes: at least one processor 30 ( Figure 3 Only one is shown in the figure), a memory 31, and a computer program 32 stored in the memory 31 and executable on the at least one processor 30, wherein the processor 30 implements the steps of any of the above-mentioned method embodiments when executing the computer program 32.

[0111] The terminal 3 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal 3 may include, but is not limited to, a processor 30 and a memory 31. Those skilled in the art will appreciate that Figure 3 It is only an example of terminal 3 and does not constitute a limitation on terminal 3. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal may also include input and output devices, network access devices, buses, etc.

[0112] The processor 30 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0113] The memory 31 may be an internal storage unit of the terminal 3, such as a hard disk or memory of the terminal 3. The memory 31 may also be an external storage device of the terminal 3, such as a plug-in hard disk, a smart memory card (SmartMediaCard, SMC), a secure digital (SecureDigital, SD) card, a flash card (FlashCard), etc. equipped on the terminal 3. Further, the memory 31 may also include both an internal storage unit of the terminal 3 and an external storage device. The memory 31 is used to store the computer program and other programs and data required by the terminal. The memory 31 may also be used to temporarily store data that has been output or is to be output.

[0114] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0115] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0116] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0117] In the embodiments provided in the present application, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0118] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0119] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0120] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0121] The present application implements all or part of the processes in the above-mentioned embodiment method, and may also be implemented through a computer program product. When the computer program product runs on a terminal, the terminal can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0122] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A risk assessment and analysis method based on big data, characterized in that: include: According to the preset data collection strategy, collect financial risk assessment related data and build a financial big data architecture based on Hadoop; The financial big data is grouped by factor analysis to obtain several factors, and the financial risk assessment value is obtained based on the factors and their corresponding variance contribution rates; According to the financial risk assessment value, the financial risk assessment report is automatically generated using the sequence-to-sequence method; The financial risk assessment report is combined with the preset interface layout principles for visual display.

2. The risk assessment and analysis method based on big data according to claim 1 is characterized in that: According to the preset data collection strategy, financial risk assessment related data is collected, and a financial big data architecture based on Hadoop is constructed, wherein the preset data collection strategy includes internal data integration, external data acquisition, and cooperative and shared data access.

3. The risk assessment and analysis method based on big data according to claim 2 is characterized in that: The external data acquisition is to screen out the target web page based on the genetic algorithm according to the relevance and importance of the web page to be crawled and the financial risk assessment theme, and crawl the financial risk assessment related data of the target web page.

4. The risk assessment and analysis method based on big data according to claim 3 is characterized in that: The method of screening out target web pages based on a genetic algorithm according to the relevance and importance of the web pages to be crawled and the financial risk assessment topic, and crawling the financial risk assessment related data of the target web pages, includes the following steps: Step 11, obtaining a queue of web page URLs to be crawled; Step 12, downloading and parsing web pages one by one, and calculating the relevance value between the web pages and the financial risk assessment topic according to the keywords and keyword weights in the web pages; Step 13, performing a selection operation based on a genetic algorithm to screen out web pages with relevance values ​​higher than a preset threshold, and forming a set A; Step 14, performing a crossover operation based on a genetic algorithm to obtain hyperlinks contained in all web pages in set A, sorting them by topic importance, and selecting the top links to update set A; Step 15, performing mutation operation based on genetic algorithm, querying and splicing subject keywords through search engine, selecting the top result links, and forming set B; Step 16, performing a merging operation based on a genetic algorithm so that the set C = A + B is inherited to the next generation; Step 17, determine whether the preset termination condition is met, if yes, output set C, if no, return to step 11; Step 18, crawl the financial risk assessment related data of the web pages in set C.

5. The risk assessment and analysis method based on big data according to claim 4 is characterized in that: The web pages are downloaded and parsed one by one, and the relevance value between the web pages and the financial risk assessment topic is calculated according to the keywords and keyword weights in the web pages, wherein the calculation formula is: In the above formula, Sim(p) represents the correlation value, η n represents the absolute frequency of the n-th dimension keyword in the web page, α n Represents the weight of the n-th dimension keyword.

6. The risk assessment and analysis method based on big data according to claim 1 is characterized in that: The financial big data is grouped by factor analysis to obtain several factors, and the financial risk assessment value is obtained according to the factors and their corresponding variance contribution rates, which specifically include: Obtain the correlation coefficient matrix based on financial big data, and remove those with correlation coefficients less than the preset value; Calculate the factor loading matrix based on the correlation coefficient matrix, and construct factor variables based on the factor loading matrix; Calculate the factor variable score for each sample, and obtain the financial risk assessment value based on the factor variable score and variance contribution rate.

7. The risk assessment and analysis method based on big data according to claim 1 is characterized in that: The method of automatically generating a financial risk assessment report based on the financial risk assessment value and using a sequence-to-sequence method is specifically as follows: Extract the features of the financial risk assessment value as an initial sequence through an extractor; After encoding and decoding the initial sequence, a financial risk assessment report is obtained.

8. The risk assessment and analysis method based on big data according to claim 1 is characterized in that: The financial risk assessment report is visualized in combination with the preset interface layout principles, including: Combining objective cognitive laws with user subjective cognition, we obtain the interface layout principles, and transform the interface layout principles into a computable multi-objective optimization function for the interface layout. The target optimization function is encoded through a preset algorithm to seek the optimal spatial solution, and then the optimal interface layout solution for the financial risk assessment report is obtained.

9. The risk assessment and analysis method based on big data according to claim 8 is characterized in that: The objective cognition law is combined with the user's subjective cognition to obtain the interface layout principle, and the interface layout principle is converted into a computable interface layout multi-objective optimization function, wherein the objective optimization function is: In the above formula, S i Indicates the order of operations, P i Indicates importance and operation frequency, O i represents the correlation principle, ω 1i and ω 2i Denotes the position control coefficient, d ia is the distance between the position coordinates of the geometric center of element i and the optimal layout point a, d a max Indicates the maximum distance between the element position point in the area to be laid out and the optimal layout point a.

10. A risk assessment and analysis system based on big data, characterized in that: include: The collection module is used to collect financial risk assessment related data according to the preset data collection strategy and build a financial big data architecture based on Hadoop; The calculation module is used to group the financial big data through factor analysis to obtain several factors, and obtain the financial risk assessment value based on the factors and their corresponding variance contribution rates; A reporting module, which is used to automatically generate a financial risk assessment report based on the financial risk assessment value and using a sequence-to-sequence method; The display module is used to visualize the financial risk assessment report in combination with the preset interface layout principles.

Images