Real estate data processing and early warning method and system based on big data

Abstract

The invention discloses a real estate data processing and early warning method and system based on big data, and belongs to the technical field of data processing.The method comprises the steps that overall extraction is conducted on stock data of a first data source and stock data of a second data source, and first main data and first additional data are obtained; performing data governance on the first main body data to obtain second main body data, and performing data simplification on the first additional data to obtain second additional data after the governance is completed; extracting a first attribute from the second main body data to generate a first form of each enterprise, and extracting a second attribute from the first form to generate a second form; updating the second form in the database every preset time interval; and setting a judgment condition, and when a second attribute in the second form meets the judgment condition, triggering risk early warning. According to the invention, the real estate condition of the enterprise can be analyzed, and the problem of incomplete analysis is solved on the basis of the prior art.

Description

Technical Field

[0001] The present invention belongs to the field of data processing technology, and specifically relates to a real estate data processing and early warning method and system based on big data. Background Art

[0002] Big data technology can acquire, store, analyze, and output large amounts of data information. With the help of big data technology, the business data of enterprises can be uniformly organized and allocated, the enterprise archives can be improved, risks can be identified for enterprises, and the financial status, credit, and operational stability of enterprises can be obtained, thereby providing assistance to the work of relevant personnel.

[0003] The prior art provides a variety of methods for analyzing the situation of enterprises in combination with big data. For example, the Chinese patent document with publication number CN116468271A discloses a method, system and medium for enterprise risk analysis based on big data. The method constructs an enterprise risk analysis model through big data, obtains multi-source business data of the enterprise, then extracts the characteristic values ​​of the multi-source business data of the enterprise, generates multi-source heterogeneous parameters of the enterprise operation, and finally inputs the multi-source heterogeneous parameters of the enterprise operation into the enterprise risk analysis model to generate risk assessment information. By comparing the risk assessment information with the preset risk standard information, the enterprise risk assessment level is generated. For another example, the Chinese patent document with publication number CN118153964A discloses a supplier enterprise risk assessment method and system based on big data technology. The method first reads the enterprise data of the supplier enterprise, then selects a corresponding summary model from a pre-trained summary model library according to the enterprise information data, obtains the enterprise operation summary based on the analysis of the enterprise information data and the enterprise operation data by the summary model, then the risk sniffing model responds to the enterprise operation data, selects a corresponding feature model from a pre-trained feature model library, obtains the data features of the enterprise operation data based on the feature model, and inputs the data features into the risk assessment model to obtain the risk assessment result of the enterprise.

[0004] However, the above method only obtains the business data of the enterprise for analysis, and does not combine it with the real estate data, so there is a problem of incomplete analysis. Summary of the invention

[0005] To solve the above problems, the present invention provides a real estate data processing and early warning method and system based on big data to solve the problems in the prior art.

[0006] In order to achieve the above-mentioned purpose of the invention, the present invention proposes a real estate data processing and early warning method based on big data, comprising: Determine a first data source and a second data source, and extract the stock data of the first data source and the second data source as a whole to obtain first main data and first additional data; Writing the first main body data and the first additional data into a database, performing data management on the first main body data in combination with the first additional data to obtain second main body data, and after the management is completed, performing data simplification on the first additional data to obtain second additional data; Setting a first attribute, extracting the first attribute from the second main body data to generate a first form for each enterprise, extracting a second attribute from the first form to generate a second form, the second attribute including real estate mortgage status, real estate seizure status, real estate valuation value, and mortgage times; extracting the first main data and the first additional data again at a preset time interval to update the second form in the database; A judgment condition is set, and when the second attribute in the second form satisfies the judgment condition, a risk warning is triggered.

[0007] Further, performing data reduction on the first additional data includes the following steps: The second data source includes a plurality of sub-data sources, locating target data of a numerical type in the sub-data sources, arranging the target data belonging to the same data name in the sub-data sources into a numerical sequence, calculating first similarities between different data names in different sub-data sources, dividing the data names into a plurality of first data groups, and the first similarities between any two data names in the first data groups being greater than a first threshold; Calculating a second similarity between different numerical sequences in the first data group, and constructing a topological graph based on the second similarity, wherein the topological graph includes a plurality of nodes, wherein the nodes represent the numerical sequences in the first data group, and the nodes are marked with optimal similarities; screening two adjacent nodes in the topological graph, determining a first sequence and a second sequence in the numerical sequences corresponding to the two nodes, and generating a mapping function based on the first sequence and the second sequence, wherein the mapping function is used to convert the first sequence into the second sequence; The database deletes the second sequence in the sub-data source, and retains the first sequence and the mapping function corresponding to the deleted second sequence, so as to complete the simplification of the first additional data.

[0008] Furthermore, constructing the topology map includes the following steps: Extract the two numerical sequences corresponding to the maximum second similarity from the first data group, define them as the first initial sequence and the second initial sequence, and define the corresponding nodes as the first initial point and the second initial point, respectively; select the numerical sequence with the maximum second similarity to the first initial sequence or the second initial sequence from the first data group as the third initial sequence, and connect it with the corresponding first initial point or the second initial point as the third initial point; repeat this step until all the numerical sequences in the first data group are extracted, and obtain the topological graph corresponding to the first data group.

[0009] Further, generating the mapping function comprises the following steps: The numerical sequences corresponding to the two nodes in adjacent positions in the topological graph are defined as a first alternative sequence and a second alternative sequence, a first function and a second function are generated based on the first alternative sequence and the second alternative sequence, the first function is used to convert the first alternative sequence into a third alternative sequence, and the second function is used to convert the second alternative sequence into a fourth alternative sequence, a first difference between the first alternative sequence and the fourth alternative sequence, and a second difference between the second alternative sequence and the third alternative sequence are calculated, and if the first difference is less than the second difference, the first alternative sequence is defined as the first sequence and the first function is defined as the mapping function; otherwise, the second alternative sequence is defined as the first sequence and the second function is defined as the mapping function.

[0010] Furthermore, after simplifying the topological graph, if there is a first retained point and a second retained point, all the nodes between the first retained point and the second retained point are deleted, and the deleted nodes are defined as target points. The first retained point and the second retained point are used as starting points respectively, and the cumulative value of the retained point is calculated in combination with the optimal similarity. If there is a retained point whose cumulative value is less than a third threshold, the retained point is restored.

[0011] Furthermore, filling missing values ​​in the first subject data includes the following steps: The first main body data that is missing is obtained, key information of the first main body data is obtained, and the first additional data is searched based on the key information to fill in the missing values.

[0012] Furthermore, the data governance includes deduplication, missing data completion, data transcoding and data desensitization.

[0013] Furthermore, when the mortgage status of the real estate changes, the seizure status of the real estate changes, the change in the real estate valuation value exceeds a fourth threshold, or the number of mortgages increases, it is determined that the second attribute meets the judgment condition.

[0014] Furthermore, the first additional data extracted again is defined as newly added data, and whenever the amount of the newly added data reaches a fifth threshold, data simplification is performed on the currently existing newly added data.

[0015] The present invention also provides a real estate data processing and early warning system based on big data, which is used to implement the above-mentioned real estate data processing and early warning method based on big data, and the system includes: A collection module is used to determine a first data source and a second data source, and the collection module is used to extract the stock data of the first data source and the second data source as a whole to obtain first main data and first additional data, and to extract the first main data and the first additional data again at a preset time interval to update the second form in the database; A storage module, writing the first main data and the first additional data into the database, performing data management on the first main data in combination with the first additional data to obtain second main data, and after the management is completed, performing data simplification on the first additional data to obtain second additional data; an extraction module, setting a first attribute, extracting the first attribute from the second main body data to generate a first form for each enterprise, extracting a second attribute from the first form to generate the second form, the second attribute including a mortgage status of the real estate, a seizure status of the real estate, a valuation value of the real estate, and a mortgage number; The early warning module is provided with a judgment condition, and when the second attribute in the second form satisfies the judgment condition, a risk early warning is triggered.

[0016] Compared with the prior art, the beneficial effects of the present invention are at least as follows: The present invention obtains the real estate data of the enterprise through multiple ways, and selects the data downloaded from the official website as the main data, namely the first main data, and performs data governance on the first main data before analysis to ensure its data quality; in the governance process, if there is data missing in the main data, the first additional data other than the first main data is used to supplement the missing values, thereby ensuring the integrity of the first main data. The present invention enables relevant personnel to quickly obtain the real estate information of the enterprise in multiple dimensions, and realizes real-time tracking and early warning of abnormal real estate information of the enterprise by setting judgment conditions.

[0017] After supplementing the missing values ​​of the first main body data by the first additional data, the present invention also simplifies the first additional data, thereby reducing the space occupied by the first additional data, accelerating the operation speed of the database, and reducing the expenditure cost of hardware. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flowchart of the steps of the real estate data processing and early warning method based on big data of the present invention; Figure 2 This is an interface diagram showing successful extraction of real estate inventory data in the present invention; Figure 3 This is an interface diagram of the real estate risk warning of the present invention; Figure 4 It is a principle schematic diagram of the topological diagram of the present invention; Figure 5 It is a structural diagram of the real estate data processing and early warning system based on big data of the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0020] It is understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, without departing from the scope of this application, a first xx script may be referred to as a second xx script, and similarly, a second xx script may be referred to as a first xx script.

[0021] like Figure 1 As shown, a real estate data processing and early warning method based on big data includes: S1: Determine a first data source and a second data source, extract the stock data of the first data source and the second data source as a whole, and obtain first main data and first additional data.

[0022] Specifically, the first data source is the e-government website, and the second data source is a third-party Internet resource. First, Kettle is used to download the real estate data of all current enterprises from the e-government website to obtain the first main data, and the extended data of the relevant enterprises is downloaded from the third-party Internet resource as the first additional data. The first additional data includes, for example, the real estate information of enterprises provided by a third-party website, as well as the business conditions, traffic conditions, building age, property information, etc. around the real estate information.

[0023] S2: writing the first main data and the first additional data into a database, performing data governance on the first main data in combination with the first additional data to obtain the second main data, and after the governance is completed, performing data simplification on the first additional data to obtain the second additional data.

[0024] The data governance in this embodiment includes deduplication, missing data filling, data transcoding and data desensitization.

[0025] The following is a detailed description taking the first data source as an example. Figure 2 As shown in the figure, during the data writing process, the data is first written into the ODS library (operational data storage) of MYSQL. Specifically, DS calls the real estate stock data kettle task to extract the stored data. After DS calls the kettle to extract the real estate stock data successfully, Figure 2 As shown, Kettle is an open source ETL tool that can realize data migration and data governance through the configuration of the graphical interface without developing code. The full name of DS is DolphinSchedule, which is a distributed and easily extensible visual workflow task scheduling platform. After the data extraction is completed, the first subject data is governed. The data quality mainly includes removing duplicate data, filling in empty data and missing data, setting special marks for data that cannot be supplemented, and for variable data or code data, parameter conversion is performed according to the code value table. Sensitive data is desensitized, such as company names, unified social credit codes, etc. In addition, when completing the data, the first additional data is searched and supplemented according to the known real estate information; after using the first additional data to complete the completion of the first subject data, the first additional data needs to be streamlined to avoid it occupying a large storage space.

[0026] S3: Set the first attribute, extract the first attribute from the second main data to generate a first form for each enterprise, extract the second attribute from the first form to generate a second form, the second attribute includes the real estate mortgage status, the real estate seizure status, the real estate valuation value and the number of mortgages.

[0027] In this embodiment, the first attribute specifically includes the name of the right holder, the right holder's certificate type, the right holder's certificate number, the real estate certificate number, the real estate unit number, the location, the region, the nature of the rights, the purpose, the building area, the registration date (issuance date), the measured floor area, the measured apportioned area, the nature of the house, the real estate seizure status, the seizure period, the name of the executor, the name of the person to be executed, the real estate mortgage status, the mortgage amount, the mortgage period, the number of mortgages, the name of the house owner (mortgagor), the house owner (mortgagor) certificate type, the house owner (mortgagor) certificate number, the real estate valuation, the update time and other parameters; then the second attribute is extracted from the above-mentioned first attribute to generate a second form, and the second attribute includes the real estate mortgage status, the real estate seizure status, the real estate valuation value and the number of mortgages.

[0028] S4: extracting the first main data and the first additional data again at every preset time interval to update the second form in the database.

[0029] Before updating, first set the data update task and the corresponding execution time. Specifically, use DS to schedule Kettle's update task on a daily basis and set up error email reminders. After building the extraction and update mechanism for each data source, apply kettle's "Insert Update Component" to perform a full-field comparison of the source data with the ODS stock data, thereby inserting the new data into ODS and completing the update of the second form.

[0030] S5: Setting a judgment condition, when the second attribute in the second form satisfies the judgment condition, triggering a risk warning.

[0031] In this embodiment, when the mortgage status of the real estate changes, the seizure status of the real estate changes, the change value of the real estate valuation exceeds the fourth threshold, and the number of mortgages increases, it is determined that the second attribute meets the judgment condition.

[0032] When the second attribute shows that the mortgage status of the real estate has changed, the seizure status of the real estate has changed, the change in the real estate valuation value exceeds the fourth threshold, or the number of mortgages increases, it is judged that the second attribute meets the judgment condition. For example, the fourth threshold is set to 10000, and the following will be generated: Figure 3 The risk warning diagram shown is used to remind relevant personnel to pay attention to enterprise risks.

[0033] The present invention obtains the real estate data of the enterprise through multiple ways, and selects the data downloaded from the official website as the main data, namely the first main data, and performs data governance on the first main data before analysis to ensure its data quality; in the governance process, if there is data missing in the main data, the first additional data other than the first main data is used to supplement the missing values, thereby ensuring the integrity of the first main data. The present invention enables relevant personnel to quickly obtain the real estate information of the enterprise in multiple dimensions, and realizes real-time tracking and early warning of abnormal real estate information of the enterprise by setting judgment conditions.

[0034] After supplementing the missing values ​​of the first main body data by the first additional data, the present invention also simplifies the first additional data, thereby reducing the space occupied by the first additional data, accelerating the operation speed of the database, and reducing the expenditure cost of hardware.

[0035] It is particularly noteworthy that the present invention can be used to analyze the real estate situation of an enterprise, thereby compensating for the problem of incomplete analysis on the basis of the prior art.

[0036] The present invention performs data simplification on the first additional data, comprising the following steps: The second data source includes multiple sub-data sources, locates target data of numerical type in the sub-data sources, organizes the target data belonging to the same data name in the sub-data sources into a numerical sequence, calculates first similarities between different data names in different sub-data sources, divides the data names into multiple first data groups, and the first similarities between any two data names in the first data groups are greater than a first threshold.

[0037] The sub-data sources are different third-party websites. After crawling data from different third-party websites, the target data of numerical type are located, such as real estate valuation, real estate transaction price, reference average price per square meter, etc. Then, various values ​​belonging to the same name will be sorted into a numerical sequence about time, such as sorting the real estate valuation of each day on website A into a numerical sequence.

[0038] The purpose of this step is to merge data of the same type from different third-party websites, such as real estate valuation data. Specifically, if the data name of real estate valuation in website A is "estimated total price of the house", and the data name of real estate valuation in website B is "estimated selling price of the house", then compare the first similarity of "estimated total price of the house" and "estimated selling price of the house". In this example, the number of identical characters in the two names is used as the first similarity. For example, in the above two names, each name has 6 characters, 5 of which are the same, then the first similarity is 5 / 6=0.83, and the first threshold is 0.5. The first similarity here is greater than the first threshold.

[0039] A second similarity between different numerical sequences in the first data group is calculated, and a topological graph is constructed based on the second similarity. The topological graph includes a plurality of nodes, each node represents a numerical sequence in the first data group, and the nodes are marked with an optimal similarity.

[0040] Due to the existence of multiple sub-data sources, there may be more than two numerical sequences in each first data group. For example, for real estate valuation, the numerical sequences A, B, and C are from websites A, B, and C respectively, and they are all in one first data group. Then, the first similarities between numerical sequences A and B, B and C, and A and C are all greater than the first threshold. The difference between the corresponding data at the same time point in the two numerical sequences can be calculated, and the corresponding variance can be calculated based on all the calculated differences, and the variance can be used as the second similarity of the two numerical sequences. Through this step, sequences with similar names and similar data can be divided into one data group. The established topology diagram is as follows: Figure 4 As shown, the method for constructing the topological graph will be introduced later. In the topological graph, nodes A, B, and C correspond to numerical sequences A, B, and C respectively.

[0041] Two adjacent nodes in the topology graph are screened, a first sequence and a second sequence are determined in the numerical sequences corresponding to the two nodes, and a mapping function is generated based on the first sequence and the second sequence, where the mapping function is used to convert the first sequence into the second sequence.

[0042] The database deletes the second sequence in the sub-data source, and retains the first sequence and the mapping function corresponding to the deleted second sequence, so as to complete the simplification of the first additional data.

[0043] Determine which of the two numerical sequences corresponding to nodes A and B is used as the first sequence and which is used as the second sequence, and then generate a mapping function based on the first sequence and the second sequence. The mapping function here is a linear function, such as y=ax+b, where a∈E, E is a set of polynomial coefficients of the mapping function, b is a constant, x is the first sequence, and y is the second sequence. Existing linear regression tools can be used to cut off E and b to determine the mapping function of the first sequence and the second sequence. During the specific conversion, the second sequence is obtained by inputting the numerical value in the first sequence into the mapping function.

[0044] Through the previous screening, two sequences with very similar numerical distributions can be obtained. In particular, the present embodiment uses the variance as the second similarity, so that the distributions of the two numerical sequences after screening are closer, so that even if the restored second sequence is different from the original second sequence, the difference will be very small; on this basis, the data is simplified by deleting the second sequence. When the first additional data is needed for other data analysis later, it can be restored with the help of the first sequence and the mapping function without having to obtain it from the website again.

[0045] This embodiment of constructing a topology map includes the following steps: Two numerical sequences corresponding to the maximum second similarity are extracted from the first data group, defined as the first initial sequence and the second initial sequence, and the corresponding nodes are defined as the first initial point and the second initial point, respectively. The numerical sequence with the maximum second similarity to the first initial sequence or the second initial sequence is selected from the first data group as the third initial sequence, and is connected to the corresponding first initial point or the second initial point as the third initial point. This step is repeated until all numerical sequences in the first data group are extracted, and a topological map corresponding to the first data group is obtained.

[0046] For example, the first data group includes numerical sequences A, B, C, and D, among which there are 6 second similarities. The second similarity with the largest value is obtained. For example, the second similarity between numerical sequences A and numerical sequence C is the largest, and the corresponding ones are defined as the first initial point and the second initial point respectively; then the second similarities between numerical sequences B and D and numerical sequence A, and the second similarities between numerical sequences B and D and numerical sequence C are obtained. After comparison, the second similarity between numerical sequence B and A is greater than the second similarity between numerical sequence B and C, and both are greater than the second similarity between numerical sequence D and A or C. Then, numerical sequence B is used as the third initial sequence, and the third initial sequence is connected to the first initial point as the third initial point in the topological graph. Finally, numerical sequence C is used as the target to analyze the numerical sequence with the largest second similarity among numerical sequences A, B, and D. The topological graph obtained by this method has the largest second similarity between adjacent nodes, and its purpose is to reduce the error value of the subsequent restoration of the second sequence from the first sequence, and further improve the accuracy of data restoration.

[0047] This embodiment generates a mapping function including the following steps: The numerical sequences corresponding to two adjacent nodes in the topological graph are defined as a first alternative sequence and a second alternative sequence, a first function and a second function are generated based on the first alternative sequence and the second alternative sequence, the first function is used to convert the first alternative sequence into a third alternative sequence, and the second function is used to convert the second alternative sequence into a fourth alternative sequence, and a first difference between the first alternative sequence and the fourth alternative sequence and a second difference between the second alternative sequence and the third alternative sequence are calculated. If the first difference is less than the second difference, the first alternative sequence is defined as a first sequence and the first function is defined as a mapping function; otherwise, the second alternative sequence is defined as a first sequence and the second function is defined as a mapping function.

[0048] Continue to refer to Figure 4, this embodiment determines the first sequence, the second sequence and the mapping function through the following steps: first select node A and node B, corresponding to the first candidate sequence and the second candidate sequence respectively, generate a first function for converting the first candidate sequence into the second candidate sequence, and a second function for converting the second candidate sequence into the first candidate sequence. It is particularly noted that in linear regression analysis, the mutual exchange of dependent variables and independent variables will affect the subsequent generated mapping function, and the change of the mapping function will affect the restoration accuracy. Based on this, after generating the first function and the second function, the first candidate sequence is converted into the third candidate sequence using the first function, and the first difference between the third candidate sequence and the second candidate sequence is calculated. The second function is used to convert the second candidate sequence into the fourth candidate sequence, and the second difference between the fourth candidate sequence and the first candidate sequence is calculated. The sum of the absolute values ​​of the numerical differences at the corresponding positions in the two numerical sequences can be used as the difference value. For example, if the first candidate sequence is 1, 2, and 3, and the third candidate sequence is 2, 1, and 3, the difference is |1-2|+|2-1|+|3-3|=2. Finally, the function corresponding to the smaller difference is selected as the mapping function, the candidate sequence used for conversion is the first sequence, and the candidate sequence after conversion is the second sequence. Then, node B and node C are selected again for analysis to determine the first sequence and the second sequence.

[0049] In this embodiment, after simplifying the topological graph, if there is a first retention point and a second retention point, all nodes between the first retention point and the second retention point are deleted, and the deleted nodes are defined as target points. The first retention point and the second retention point are used as starting points respectively, and the cumulative value of the retention point is calculated in combination with the optimal similarity. If there is a retention point whose cumulative value is less than the third threshold, the retention point is restored.

[0050] Continue to refer to Figure 4 When deleting, in nodes A and B, A is the first sequence, and the second sequence corresponding to node B will be deleted. In nodes B and C, node B is the first sequence, and the second sequence corresponding to node C will be deleted. In nodes C and D, node C is the first sequence, and the second sequence corresponding to node D will be deleted. In nodes D and E, node E is the first sequence, and the second sequence corresponding to node D will be deleted. Eventually, nodes B, C, and D will be deleted. Correspondingly, nodes A and node B will become the first and second retention points, and nodes B, C, and D will be located as target points.

[0051] If the best similarity between node A and node B is 0.92, the best similarity between node B and node C is 0.8, the best similarity between node C and node D is 0.85, the best similarity between node D and node E, and the best similarity between node A and node B is 0.92, for node C, the cumulative value from node A to node C is 0.92 0.8=0.736, the cumulative value from node E to node C is 0.92 0.85=0.782, assuming that the third threshold is 0.75, node C is restored.

[0052] Since there will be a certain error when restoring node A to node B, restoring node B to node C will increase the existing error, which will cause the accuracy of subsequent restorations to become increasingly poor. Therefore, the purpose of this step is to avoid the restoration chain being too long, which will cause the error value to be continuously amplified.

[0053] In this embodiment, filling missing values ​​of the first subject data includes the following steps: Obtain missing first subject data, obtain key information of the first subject data, and search the first additional data based on the key information to fill in the missing values.

[0054] Specifically, for example, if there are missing real estate valuation values ​​in the first main data, key information such as the location of the real estate, the region it belongs to, the name of the right holder, etc. is obtained, and these key information are used to search in the first additional data. If a real estate valuation value that matches the key information is obtained in the first additional data, it is imported into the first main data to complete the missing value filling.

[0055] In this embodiment, the first additional data extracted again is defined as the newly added data. Whenever the amount of the newly added data reaches a fifth threshold, data simplification is performed on the currently existing newly added data.

[0056] like Figure 5 As shown, the present invention also provides a real estate data processing and early warning system based on big data, which is used to implement the above-mentioned real estate data processing and early warning method based on big data, and the system includes: A collection module is used to determine the first data source and the second data source, and the collection module is used to extract the stock data of the first data source and the second data source as a whole to obtain the first main data and the first additional data, and to extract the first main data and the first additional data again at a preset time interval to update the second form in the database; A storage module writes the first main body data and the first additional data into a database, performs data management on the first main body data in combination with the first additional data to obtain the second main body data, and after the management is completed, performs data simplification on the first additional data to obtain the second additional data; An extraction module sets a first attribute, extracts the first attribute from the second main body data to generate a first form for each enterprise, extracts a second attribute from the first form to generate a second form, and the second attribute includes a mortgage status of the real estate, a seizure status of the real estate, a valuation value of the real estate, and a mortgage number; The early warning module is provided with a judgment condition, and when the second attribute in the second form satisfies the judgment condition, a risk early warning is triggered.

[0057] It should be understood that the various technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the various technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0058] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A real estate data processing and early warning method based on big data, characterized in that: include: Determine a first data source and a second data source, and extract the stock data of the first data source and the second data source as a whole to obtain first main data and first additional data; Writing the first main body data and the first additional data into a database, performing data management on the first main body data in combination with the first additional data to obtain second main body data, and after the management is completed, performing data simplification on the first additional data to obtain second additional data; Setting a first attribute, extracting the first attribute from the second main body data to generate a first form for each enterprise, extracting a second attribute from the first form to generate a second form, the second attribute including real estate mortgage status, real estate seizure status, real estate valuation value, and mortgage times; extracting the first main data and the first additional data again at a preset time interval to update the second form in the database; A judgment condition is set, and when the second attribute in the second form satisfies the judgment condition, a risk warning is triggered.

2. The method according to claim 1, characterized in that The step of simplifying the first additional data comprises the following steps: The second data source includes a plurality of sub-data sources, locating target data of a numerical type in the sub-data sources, arranging the target data belonging to the same data name in the sub-data sources into a numerical sequence, calculating first similarities between different data names in different sub-data sources, dividing the data names into a plurality of first data groups, and the first similarities between any two data names in the first data groups being greater than a first threshold; Calculating a second similarity between different numerical sequences in the first data group, and constructing a topological graph based on the second similarity, wherein the topological graph includes a plurality of nodes, wherein the nodes represent the numerical sequences in the first data group, and the nodes are marked with optimal similarities; screening two adjacent nodes in the topological graph, determining a first sequence and a second sequence in the numerical sequences corresponding to the two nodes, and generating a mapping function based on the first sequence and the second sequence, wherein the mapping function is used to convert the first sequence into the second sequence; The database deletes the second sequence in the sub-data source, and retains the first sequence and the mapping function corresponding to the deleted second sequence, so as to complete the simplification of the first additional data.

3. The method according to claim 2, characterized in that Constructing the topology diagram includes the following steps: Extract the two numerical sequences corresponding to the maximum second similarity from the first data group, define them as the first initial sequence and the second initial sequence, and define the corresponding nodes as the first initial point and the second initial point, respectively; select the numerical sequence with the maximum second similarity to the first initial sequence or the second initial sequence from the first data group as the third initial sequence, and connect it with the corresponding first initial point or the second initial point as the third initial point; repeat this step until all the numerical sequences in the first data group are extracted, and obtain the topological graph corresponding to the first data group.

4. The method according to claim 2, characterized in that: Generating the mapping function comprises the following steps: The numerical sequences corresponding to the two nodes in adjacent positions in the topological graph are defined as a first alternative sequence and a second alternative sequence, a first function and a second function are generated based on the first alternative sequence and the second alternative sequence, the first function is used to convert the first alternative sequence into a third alternative sequence, and the second function is used to convert the second alternative sequence into a fourth alternative sequence, a first difference between the first alternative sequence and the fourth alternative sequence, and a second difference between the second alternative sequence and the third alternative sequence are calculated, and if the first difference is less than the second difference, the first alternative sequence is defined as the first sequence and the first function is defined as the mapping function; otherwise, the second alternative sequence is defined as the first sequence and the second function is defined as the mapping function.

5. The method according to claim 4, characterized in that After simplifying the topological graph, if there is a first retained point and a second retained point, all the nodes between the first retained point and the second retained point are deleted, and the deleted nodes are defined as target points. The first retained point and the second retained point are used as starting points respectively, and the cumulative value of the retained point is calculated in combination with the optimal similarity. If there is a retained point where the cumulative value is less than a third threshold, the retained point is restored.

6. The method according to claim 1, characterized in that Filling missing values ​​for the first subject data includes the following steps: The first main body data that is missing is obtained, key information of the first main body data is obtained, and the first additional data is searched based on the key information to fill in the missing values.

7. The method according to claim 6, characterized in that The data governance includes data deduplication, missing data completion, data transcoding and data desensitization.

8. The method according to claim 1, characterized in that When the mortgage status of the real estate changes, the seizure status of the real estate changes, the change in the real estate valuation value exceeds the fourth threshold, or the number of mortgages increases, it is determined that the second attribute meets the judgment condition.

9. The method according to claim 1, characterized in that: The first additional data extracted again is defined as newly added data, and whenever the amount of the newly added data reaches a fifth threshold, data simplification is performed on the currently existing newly added data.

10. A real estate data processing and early warning system based on big data, used to implement the method according to any one of claims 1 to 9, characterized in that: include: A collection module is used to determine a first data source and a second data source, and the collection module is used to extract the stock data of the first data source and the second data source as a whole to obtain first main data and first additional data, and to extract the first main data and the first additional data again at a preset time interval to update the second form in the database; A storage module, writing the first main data and the first additional data into the database, performing data management on the first main data in combination with the first additional data to obtain second main data, and after the management is completed, performing data simplification on the first additional data to obtain second additional data; an extraction module, setting a first attribute, extracting the first attribute from the second main body data to generate a first form for each enterprise, extracting a second attribute from the first form to generate a second form, the second attribute including a mortgage status of the real estate, a seizure status of the real estate, an estimated value of the real estate, and a mortgage number; The early warning module is provided with a judgment condition, and when the second attribute in the second form satisfies the judgment condition, a risk early warning is triggered.

Images