Collapse hidden danger early warning method and system based on big data

Abstract

The invention provides a collapse hidden danger early warning method and system based on big data, and relates to the technical field of hidden danger early warning. Knowledge fragment extraction processing is performed on example collapse hidden danger data to obtain secondary knowledge fragments and important knowledge fragments; according to the secondary knowledge fragments corresponding to the multiple pieces of example collapse hidden danger data, establishing secondary description labels, and meanwhile, according to the important knowledge fragments corresponding to the multiple pieces of example collapse hidden danger data, establishing important description labels. According to the method, similar collapse hidden danger data analysis aiming at the to-be-processed collapse hidden danger data is realized in the secondary branch and the important branch, knowledge fragments of the collapse hidden danger data in different aspects can be concerned at the same time, and then the analysis precision can be improved, so that the collapse hidden danger can be accurately obtained, and early warning can be performed in advance. And the harm caused by collapse can be reduced as much as possible.

Description

Technical Field

[0001] The present application relates to the technical field of hidden danger warning, and specifically, to a method and system for early warning of collapse hidden dangers based on big data. Background Art

[0002] In geological disasters, collapse is a very short-term event, which only takes a moment from its occurrence to its end. Therefore, monitoring and early warning using traditional technologies are useless. After the signal is transmitted, the collapse is over, so monitoring and early warning are ineffective.

[0003] Therefore, it is necessary to analyze and evaluate the hidden danger points to achieve early warning of collapse. This technology is obviously different from the traditional technology. The traditional technology has been improved by analyzing and evaluating the hidden danger points and then warning of the possibility and time of collapse to improve the problems existing in the traditional technology. However, how to improve the accuracy of the early warning is a technical problem that is difficult to solve at present. Summary of the invention

[0004] In order to improve the technical problems existing in the relevant technologies, the present application provides a collapse hazard early warning method and system based on big data.

[0005] In a first aspect, a method for early warning of potential collapse hazards based on big data is provided, the method comprising: Performing knowledge segment extraction processing on the example collapse hazard data to obtain secondary knowledge segments and important knowledge segments of the example collapse hazard data; Building secondary description tags according to secondary knowledge fragments corresponding to several of the exemplary collapse hazard data, and building important description tags according to important knowledge fragments corresponding to several of the exemplary collapse hazard data; Performing knowledge segment extraction processing on the collapse hazard data to be processed to obtain secondary knowledge segments and important knowledge segments of the collapse hazard data to be processed; Performing similarity analysis on the secondary description tags according to the secondary knowledge fragments of the to-be-processed collapse hazard data to obtain similar secondary description tags, and performing similarity analysis on the important description tags according to the important knowledge fragments of the to-be-processed collapse hazard data to obtain similar important description tags; The example collapse hazard data corresponding to the similar minor description tags and at least part of the example collapse hazard data corresponding to the similar major description tags are determined as collapse hazard warning information of the collapse hazard data to be processed.

[0006] In the present application, before the knowledge fragment extraction process is performed on the example collapse hazard data, the method further includes: Performing knowledge fragment extraction processing on the configuration collapse hidden danger data in the configuration collapse hidden danger data set through the collapse hidden danger data processing thread to obtain secondary knowledge fragments and important knowledge fragments of the configuration collapse hidden danger data, and performing important content classification processing on the important knowledge fragments of the configuration collapse hidden danger data to obtain predicted classification knowledge fragments; Performing cost index calculation processing on the secondary knowledge fragments, the important knowledge fragments and the predicted classification knowledge fragments of the configuration collapse hidden danger data respectively, to obtain the secondary knowledge fragment cost index value, the important knowledge fragment cost index value and the important content classification cost index value corresponding to the configuration collapse hidden danger data set; Performing splicing processing on the secondary knowledge fragment cost index value, the important knowledge fragment cost index value and the important content classification cost index value of the configuration collapse potential risk data set, and configuring the collapse potential risk data processing thread according to the obtained splicing cost index value; The configured collapse potential risk data processing thread is used to perform knowledge segment extraction processing on the example collapse potential risk data and the to-be-processed collapse potential risk data.

[0007] In the present application, the collapse hidden danger data processing thread includes an original knowledge fragment extraction model, a secondary knowledge fragment extraction model, an important knowledge fragment extraction model and an important content classification model; the collapse hidden danger data processing thread performs knowledge fragment extraction processing on the configuration collapse hidden danger data in the configuration collapse hidden danger data set to obtain the secondary knowledge fragments and important knowledge fragments of the configuration collapse hidden danger data, including: Performing original knowledge fragment extraction processing on the configuration collapse hidden danger data by using the original knowledge fragment extraction model to obtain original knowledge fragments of the configuration collapse hidden danger data; Performing secondary knowledge segment extraction processing on the original knowledge segments of the configuration collapse hidden danger data by using the secondary knowledge segment extraction model to obtain the secondary knowledge segments of the configuration collapse hidden danger data; Performing important knowledge fragment extraction processing on the original knowledge fragments of the configuration collapse hidden danger data by using the important knowledge fragment extraction model to obtain the important knowledge fragments of the configuration collapse hidden danger data; The important knowledge fragments of the configuration collapse hidden danger data are classified into important contents to obtain predicted classified knowledge fragments, including: The important content classification model is used to perform important content classification processing on the important knowledge fragments of the configuration collapse hidden danger data to obtain the predicted classification knowledge fragments of the configuration collapse hidden danger data.

[0008] In the present application, configuring the collapse potential data processing thread according to the obtained splicing cost index value includes: Perform one of the following random operations: configuring the minor knowledge fragment extraction model, the important knowledge fragment extraction model and the important content classification model according to the splicing cost index value; configuring the original knowledge fragment extraction model, the secondary knowledge fragment extraction model, the important knowledge fragment extraction model and the important content classification model according to the splicing cost index value; Among them, the configuration efficiency of the important content classification model is greater than the model in the collapse potential data processing thread that is different from the important content classification model.

[0009] In the present application, the configuration collapse hazard data set includes first configuration collapse hazard data, second configuration collapse hazard data similar to the first configuration collapse hazard data, and third configuration collapse hazard data not similar to the first configuration collapse hazard data; the cost index calculation processing is performed on the secondary knowledge fragments, the important knowledge fragments and the predicted classification knowledge fragments of the configuration collapse hazard data respectively to obtain the secondary knowledge fragment cost index value, the important knowledge fragment cost index value and the important content classification cost index value corresponding to the configuration collapse hazard data set, including: Determining a secondary knowledge segment cost index value of the configuration collapse hazard data set according to a secondary difference between the secondary knowledge segment of the first configuration collapse hazard data and the secondary knowledge segment of the second configuration collapse hazard data, and a secondary difference between the secondary knowledge segment of the first configuration collapse hazard data and the secondary knowledge segment of the third configuration collapse hazard data; Determine the important knowledge fragment cost index value of the configuration collapse hazard data set according to the important difference between the important knowledge fragment of the first configuration collapse hazard data and the important knowledge fragment of the second configuration collapse hazard data, and the important difference between the important knowledge fragment of the first configuration collapse hazard data and the important knowledge fragment of the third configuration collapse hazard data; According to the labeled classification knowledge fragment and the predicted classification knowledge fragment of a random configuration collapse hazard data in the configuration collapse hazard data set, the important content classification cost index value of the random configuration collapse hazard data is determined, and the important content classification cost index values ​​corresponding to the first configuration collapse hazard data, the second configuration collapse hazard data and the third configuration collapse hazard data are respectively concatenated to obtain the important content classification cost index value of the configuration collapse hazard data set.

[0010] In the present application, before determining the important content classification cost index value of a random configuration collapse hazard data according to the labeled classification knowledge fragment and the predicted classification knowledge fragment of the random configuration collapse hazard data in the configuration collapse hazard data set, the method further includes: Perform one of the following random operations: Obtain a labeled classification knowledge fragment for the random configuration collapse hazard data setting; Performing important content classification processing on the random configuration collapse potential risk data through an important content classification thread to obtain a labeled classification knowledge fragment; Among them, the important content classification thread is different from the collapse potential data processing thread.

[0011] In the present application, before performing knowledge fragment extraction processing on the configuration collapse potential risk data in the configuration collapse potential risk data set by the collapse potential risk data processing thread, the method further includes: Performing clustering processing on each of the plurality of configuration collapse potential risk data sets to obtain a plurality of cluster directories respectively; wherein the cluster directories include a plurality of similar configuration collapse potential risk data; Performing mining processing on the plurality of cluster directories to obtain a plurality of configuration collapse hidden danger data sets; The configuration collapse potential risk data set includes at least one of the following: a plurality of configuration collapse potential risk data belonging to the same cluster directory; or a plurality of configuration collapse potential risk data belonging to different configuration collapse potential risk data sets.

[0012] In the present application, the example collapse hazard data is obtained by mining the example collapse hazard data set, and the to-be-processed collapse hazard data is obtained by mining the to-be-processed collapse hazard data set; after determining the example collapse hazard data corresponding to the similar secondary description tags and at least part of the example collapse hazard data corresponding to the similar important description tags as collapse hazard warning information of the to-be-processed collapse hazard data, the method further includes: The example collapse hazard data set where the collapse hazard warning information is located is determined as a similar example collapse hazard data set to the collapse hazard data set to be processed.

[0013] In the present application, the method further comprises: According to the correspondence between the example collapse hazard data and the target description tag, a correspondence between the example collapse hazard data set where the example collapse hazard data is located and the target description tag is established; wherein the target description tag includes a random one of the secondary description tag and the important description tag; Establishing a correspondence between the to-be-processed collapse hazard data set and the similar target description label obtained by the similarity analysis process; wherein the similar target description label includes a random one of the similar secondary description label and the similar important description label; According to the correspondence between the similar example collapse hazard data sets and the target description labels, and the correspondence between the to-be-processed collapse hazard data sets and the similar target description labels, a plurality of the similar example collapse hazard data sets are distributedly processed.

[0014] In the present application, the distributed processing of a plurality of similar example collapse hazard data sets according to the correspondence between the similar example collapse hazard data sets and the target description labels, and the correspondence between the to-be-processed collapse hazard data sets and the similar target description labels, includes: Perform one of the following random operations: According to the intersection-and-union ratio between the important description labels corresponding to the similar example collapse hazard data sets and the similar important description labels corresponding to the collapse hazard data sets to be processed, performing distribution processing on a plurality of the similar example collapse hazard data sets; The intersection-and-union ratios of the secondary description labels corresponding to the similar example collapse hazard dataset and the similar secondary description labels corresponding to the collapse hazard dataset to be processed, as well as the intersection-and-union ratios of the important description labels corresponding to the similar example collapse hazard dataset and the similar important description labels corresponding to the collapse hazard dataset to be processed are spliced, and based on the obtained spliced ​​intersection-and-union comparisons, several different similar example collapse hazard datasets are distributed.

[0015] In the present application, the secondary description tags are constructed based on the secondary knowledge fragments corresponding to the several example collapse hazard data, and the important description tags are constructed based on the important knowledge fragments corresponding to the several example collapse hazard data, including: Clustering the secondary knowledge fragments corresponding to the plurality of example collapse hazard data, and determining the obtained plurality of secondary cluster centers as secondary description labels; Clustering is performed on the important knowledge fragments corresponding to the several example collapse hazard data, and the several obtained important cluster centers are determined as important description labels; The method further comprises: Determine the secondary differences between the secondary knowledge segments of the example collapse hazard data and a plurality of the secondary description tags, respectively, and establish a corresponding relationship between the example collapse hazard data and the secondary description tags that meet the secondary difference requirements; The important differences between the important knowledge fragments of the example collapse hazard data and a plurality of the important description tags are respectively determined, and a corresponding relationship between the example collapse hazard data and the important description tags that meet the requirements of the important differences is established.

[0016] In the present application, the similarity analysis processing is performed on the secondary description tags according to the secondary knowledge fragments of the collapse hazard data to be processed to obtain similar secondary description tags, including: Determine the secondary differences between the secondary knowledge fragments of the to-be-processed collapse hazard data and a plurality of the secondary description tags respectively, and determine the secondary description tags that meet the secondary difference requirements as similar secondary description tags; The performing similarity analysis on the important description tags according to the important knowledge fragments of the collapse hazard data to be processed to obtain similar important description tags includes: The important differences between the important knowledge fragments of the to-be-processed collapse hazard data and a plurality of the important description tags are respectively determined, and the important description tags meeting the important difference requirements are determined as similar important description tags.

[0017] In a second aspect, a collapse hazard warning system based on big data is provided, comprising a processor and a memory communicating with each other, wherein the processor is used to read a computer program from the memory and execute it to implement the above method.

[0018] The embodiment of the present application provides a collapse hazard early warning method and system based on big data, which extracts knowledge fragments from example collapse hazard data to obtain secondary knowledge fragments and important knowledge fragments, builds secondary description tags according to the secondary knowledge fragments corresponding to several example collapse hazard data, and builds important description tags according to the important knowledge fragments corresponding to several example collapse hazard data. In this way, similar collapse hazard data analysis for the collapse hazard data to be processed is implemented in the secondary and important branches, which can simultaneously focus on knowledge fragments of the collapse hazard data in different aspects, thereby improving the analysis accuracy, so that the collapse hazard can be accurately obtained and early warning can be given. In this way, the harm caused by the collapse can be reduced as much as possible. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1A flowchart of a collapse hazard early warning method based on big data provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to better understand the above technical scheme, the technical scheme of the present application is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical scheme of the present application, rather than limitations on the technical scheme of the present application. In the absence of conflict, the embodiments of the present application and the technical features in the embodiments can be combined with each other.

[0022] See also Figure 1 , shows a collapse hazard early warning method based on big data, which may include the technical solutions described in the following steps 101 to 105.

[0023] In step 101, knowledge fragments are extracted from the example collapse hazard data to obtain secondary knowledge fragments and important knowledge fragments of the example collapse hazard data.

[0024] Exemplarily, a knowledge segment can be understood as a feature.

[0025] Here, several example collapse hazard data are obtained, and knowledge fragment extraction processing is performed on each example collapse hazard data. The knowledge fragment extraction processing includes two aspects. The first aspect is to extract the secondary knowledge fragments of the example collapse hazard data, and the second aspect is to extract the important knowledge fragments of the example collapse hazard data.

[0026] The embodiments of the present application do not limit the form of the secondary knowledge fragments and the form of the important knowledge fragments, for example, they can both be in vector form; the implementation method of the knowledge fragment extraction and processing is also not limited, for example, it can be implemented through a specific collapse risk data processing thread.

[0027] In step 102, secondary description tags are constructed according to secondary knowledge fragments corresponding to a number of example collapse hazard data, and important description tags are constructed according to important knowledge fragments corresponding to a number of example collapse hazard data.

[0028] Exemplarily, several example collapse hazard data may be understood as sample data in a database.

[0029] Here, the minor description tag may be a minor knowledge segment corresponding to at least part of the example collapse hazard data, and the important description tag may be an important knowledge segment corresponding to at least part of the example collapse hazard data.

[0030] In some possible implementations, the above-mentioned construction of secondary description tags based on the secondary knowledge fragments corresponding to several example collapse hazard data and the construction of important description tags based on the important knowledge fragments corresponding to several example collapse hazard data can be achieved in the following way: the secondary knowledge fragments corresponding to all example collapse hazard data are determined as secondary description tags, and the important knowledge fragments corresponding to all example collapse hazard data are determined as important description tags. Under this premise, there is a one-to-one correspondence between the example collapse hazard data and the secondary description tags, and between the example collapse hazard data and the important description tags.

[0031] In some possible embodiments, the above-mentioned construction of secondary description tags according to secondary knowledge fragments corresponding to several example collapse hazard data, and construction of important description tags according to important knowledge fragments corresponding to several example collapse hazard data can be achieved in the following manner: clustering the secondary knowledge fragments corresponding to several example collapse hazard data, and determining the obtained several secondary clustering centers as secondary description tags; clustering the important knowledge fragments corresponding to several example collapse hazard data, and determining the obtained several important clustering centers as important description tags.

[0032] Here, in order to avoid the excessive amount of subsequent calculations caused by too many description tags, the secondary knowledge fragments corresponding to all the example collapse hazard data can also be clustered, and the obtained several cluster centers (named as secondary cluster centers for easy distinction) are all determined as secondary description tags; at the same time, the important knowledge fragments corresponding to all the example collapse hazard data are clustered, and the obtained several cluster centers (named as important cluster centers for easy distinction) are all determined as important description tags. The embodiment of the present application does not limit the method of clustering processing. For example, clustering processing can be achieved by algorithms such as Kmeans algorithm, hierarchical clustering algorithm or K-means++ algorithm.

[0033] Through the above method, it can be ensured that the number of secondary description tags and the number of important description tags will not be too large, and the determined secondary description tags and important description tags are representative, which is helpful to improve the subsequent similarity analysis effect.

[0034] In some possible embodiments, after step 102, it also includes: determining the secondary differences between the secondary knowledge segments of the example collapse hazard data and several secondary description tags, and establishing a correspondence between the example collapse hazard data and the secondary description tags that meet the secondary difference requirements; determining the important differences between the important knowledge segments of the example collapse hazard data and several important description tags, and establishing a correspondence between the example collapse hazard data and the important description tags that meet the important difference requirements.

[0035] In the embodiment of the present application, the correspondence between the example collapse hazard data and the secondary description tags, and the correspondence between the example collapse hazard data and the important description tags can also be intelligently established.

[0036] Through the above method, it can be ensured that each example collapse hazard data is in correspondence with at least one secondary description tag, and at the same time in correspondence with at least one important description tag, and the accuracy of the established correspondence can be guaranteed.

[0037] In step 103, knowledge fragments are extracted from the collapse hazard data to be processed, so as to obtain secondary knowledge fragments and important knowledge fragments of the collapse hazard data to be processed.

[0038] For example, the minor knowledge fragment can be understood as a feature with a small coefficient of collapse hazard, and the important knowledge fragment can be understood as a feature with a large coefficient of collapse hazard. The coefficient range is between 0 and 1, and a value less than or equal to 0.49 is understood as a small coefficient, and a value greater than 0.49 is understood as a large coefficient.

[0039] Here, the method of extracting knowledge segments from the collapse hazard data to be processed may be the same as step 101. After the knowledge segment extraction, the secondary knowledge segments and the important knowledge segments of the collapse hazard data to be processed are obtained.

[0040] In step 104, similarity analysis is performed on the secondary description tags according to the secondary knowledge fragments of the collapse hazard data to be processed to obtain similar secondary description tags, and similarity analysis is performed on the important description tags according to the important knowledge fragments of the collapse hazard data to be processed to obtain similar important description tags.

[0041] For example, similarity analysis can understand similar data in the analysis data.

[0042] For example, based on the minor differences between the minor knowledge fragments of the collapse hazard data to be processed and each minor description tag, at least some of the minor description tags can be selected to be determined as similar minor description tags; based on the major differences between the important knowledge fragments of the collapse hazard data to be processed and each important description tag, at least some of the important description tags can be selected to be determined as similar important description tags. The embodiment of the present application does not limit the number of similar minor description tags determined, for example, it can be at least one, and the same applies to similar important description tags.

[0043] In some possible embodiments, the above-mentioned similarity analysis processing of secondary description tags according to secondary knowledge fragments of the collapse hazard data to be processed to obtain similar secondary description tags, and similarity analysis processing of important description tags according to important knowledge fragments of the collapse hazard data to be processed to obtain similar important description tags can be achieved in the following manner: the secondary differences between the secondary knowledge fragments of the collapse hazard data to be processed and several secondary description tags are respectively determined, and the secondary description tags that meet the secondary difference requirements are determined as similar secondary description tags; the important differences between the important knowledge fragments of the collapse hazard data to be processed and several important description tags are respectively determined, and the important description tags that meet the important difference requirements are determined as similar important description tags.

[0044] Here, the minor differences between the minor knowledge fragments of the collapse hazard data to be processed and all the minor description tags can be determined respectively, and the minor description tags that meet the minor difference requirements can be determined as similar minor description tags; the important differences between the important knowledge fragments of the collapse hazard data to be processed and all the important description tags can be determined respectively, and the important description tags that meet the important difference requirements can be determined as similar important description tags.

[0045] In step 105, the example collapse hazard data corresponding to similar minor description tags and at least part of the example collapse hazard data corresponding to similar major description tags are determined as collapse hazard warning information of the collapse hazard data to be processed.

[0046] For example, the collapse hazard warning information may include the possibility of collapse, the probability of collapse occurring within a certain time period, the probability of collapse occurring within a certain range, and the like.

[0047] Here, the example collapse hazard data corresponding to similar minor description tags and the example collapse hazard data corresponding to similar major description tags may be determined as the collapse hazard warning information (similar collapse hazard data) of the collapse hazard data to be processed.

[0048] In some possible embodiments, the example collapse hazard data can also be further selected, for example, all example collapse hazard data corresponding to similar minor description tags are determined, and the minor differences between the minor knowledge segments of these example collapse hazard data and the minor knowledge segments of the collapse hazard data to be processed are determined, and the example collapse hazard data that meets the minor difference requirements are determined as collapse hazard warning information; all example collapse hazard data corresponding to similar important description tags are determined, and the important differences between the important knowledge segments of these example collapse hazard data and the important knowledge segments of the collapse hazard data to be processed are determined, and the example collapse hazard data that meets the important difference requirements are determined as collapse hazard warning information. In this way, the analysis accuracy can be further improved.

[0049] In some possible embodiments, the collapse hazard warning method based on big data provided in the embodiments of the present application may, before step 101, also perform knowledge fragment extraction processing on the configuration collapse hazard data in the configuration collapse hazard data set in step 201 through the collapse hazard data processing thread to obtain secondary knowledge fragments and important knowledge fragments of the configuration collapse hazard data, and perform important content classification processing on the important knowledge fragments of the configuration collapse hazard data to obtain predictive classification knowledge fragments.

[0050] In the embodiment of the present application, the collapse hazard data processing thread can be used to implement knowledge fragment extraction and processing of the example collapse hazard data and the collapse hazard data to be processed. In order to ensure the accuracy of the knowledge fragment extraction and processing, the collapse hazard data processing thread can be pre-configured.

[0051] During the configuration process, for each configuration collapse hazard data in the configuration collapse hazard data set, forward decision processing is performed through the collapse hazard data processing thread, that is, knowledge fragment extraction processing is performed on the configuration collapse hazard data to obtain secondary knowledge fragments and important knowledge fragments of the configuration collapse hazard data, and important content classification processing is performed on the important knowledge fragments of the configuration collapse hazard data to obtain predicted classification knowledge fragments. Among them, the configuration collapse hazard data set includes a number of configuration collapse hazard data, and the configuration collapse hazard data set can be generated manually or automatically; the predicted classification knowledge fragment is used to represent the predicted important content classification of the configuration collapse hazard data. For example, the predicted classification knowledge fragment can be a predicted probability vector, in which the important content classification corresponding to the largest value is the predicted important content classification.

[0052] In step 202, cost index calculation is performed respectively according to the secondary knowledge fragments, important knowledge fragments and predicted classification knowledge fragments of the configuration collapse hazard data to obtain the secondary knowledge fragment cost index value, important knowledge fragment cost index value and important content classification cost index value of the corresponding configuration collapse hazard data set.

[0053] Exemplarily, the cost indicator value can be understood as a loss value.

[0054] Here, corresponding cost index strategies can be set in advance for minor knowledge fragments, important knowledge fragments and predicted classification knowledge fragments respectively. In this way, cost index calculation and processing can be performed based on the minor knowledge fragments of the configuration collapse hazard data to obtain the cost index values ​​of the minor knowledge fragments of the configuration collapse hazard data set; cost index calculation and processing can be performed based on the important knowledge fragments of the configuration collapse hazard data to obtain the cost index values ​​of the important knowledge fragments of the configuration collapse hazard data set; cost index calculation and processing can be performed based on the predicted classification knowledge fragments of the configuration collapse hazard data to obtain the important content classification cost index values ​​of the configuration collapse hazard data set.

[0055] In some possible embodiments, a configuration collapse hazard data set includes first configuration collapse hazard data, second configuration collapse hazard data similar to the first configuration collapse hazard data, and third configuration collapse hazard data not similar to the first configuration collapse hazard data; the above-mentioned cost index calculation processing is performed according to the secondary knowledge fragments, important knowledge fragments and predicted classification knowledge fragments of the configuration collapse hazard data in the following manner to obtain the secondary knowledge fragment cost index value, important knowledge fragment cost index value and important content classification cost index value of the corresponding configuration collapse hazard data set: according to the secondary differences between the secondary knowledge fragments of the first configuration collapse hazard data and the secondary knowledge fragments of the second configuration collapse hazard data, and the secondary differences between the secondary knowledge fragments of the first configuration collapse hazard data and the secondary knowledge fragments of the third configuration collapse hazard data, determine Determine the secondary knowledge fragment cost index value of the configuration collapse hazard data set; determine the important knowledge fragment cost index value of the configuration collapse hazard data set according to the important differences between the important knowledge fragments of the first configuration collapse hazard data and the important knowledge fragments of the second configuration collapse hazard data, and the important differences between the important knowledge fragments of the first configuration collapse hazard data and the important knowledge fragments of the third configuration collapse hazard data; determine the important content classification cost index value of a random configuration collapse hazard data according to the labeled classification knowledge fragment and the predicted classification knowledge fragment of a random configuration collapse hazard data in the configuration collapse hazard data set, and concatenate the important content classification cost index values ​​corresponding to the first configuration collapse hazard data, the second configuration collapse hazard data and the third configuration collapse hazard data, respectively, to obtain the important content classification cost index value of the configuration collapse hazard data set.

[0056] In an embodiment of the present application, the configuration collapse hazard data set may be a triplet, i.e., including first configuration collapse hazard data, second configuration collapse hazard data similar to the first configuration collapse hazard data, and third configuration collapse hazard data dissimilar to the first configuration collapse hazard data. Under this premise, the secondary knowledge segment cost index value of the configuration collapse hazard data set may be determined based on the secondary difference between the secondary knowledge segment of the first configuration collapse hazard data and the secondary knowledge segment of the second configuration collapse hazard data, and the secondary difference between the secondary knowledge segment of the first configuration collapse hazard data and the secondary knowledge segment of the third configuration collapse hazard data. The secondary knowledge segment cost index value is used to expand the difference (i.e., difference) between the first configuration collapse hazard data and the third configuration collapse hazard data in the secondary knowledge segment, while reducing the difference between the first configuration collapse hazard data and the second configuration collapse hazard data in the secondary knowledge segment, wherein the secondary knowledge segment cost index value may be obtained by triplet loss calculation, but is not limited thereto.

[0057] Similarly, the important knowledge fragment cost index value of the configuration collapse hazard data set can be determined based on the important differences between the important knowledge fragments of the first configuration collapse hazard data and the important knowledge fragments of the second configuration collapse hazard data, and the important differences between the important knowledge fragments of the first configuration collapse hazard data and the important knowledge fragments of the third configuration collapse hazard data.

[0058] As for the predicted classification knowledge fragment, for each configuration collapse hazard data in the configuration collapse hazard data set, the important content classification cost index value of the configuration collapse hazard data can be determined according to the labeled classification knowledge fragment and the predicted classification knowledge fragment of the configuration collapse hazard data, wherein the important content classification cost index value can be calculated by the relative entropy cost index strategy, but is not limited to this. Then, the important content classification cost index values ​​corresponding to the first configuration collapse hazard data, the second configuration collapse hazard data, and the third configuration collapse hazard data in the configuration collapse hazard data set are spliced ​​to obtain the important content classification cost index value of the configuration collapse hazard data set. In the embodiment of the present application, the splicing process can be achieved by direct summation or weighted summation.

[0059] For the case where the configuration collapse risk data set is a triple, the secondary knowledge fragment cost index value, the important knowledge fragment cost index value and the important content classification cost index value of the triple can be quickly and accurately determined through the above method.

[0060] In some possible embodiments, before determining the important content classification cost index value of a random piece of configuration collapse hazard data based on the labeled classification knowledge fragment and the predicted classification knowledge fragment of a random piece of configuration collapse hazard data in the configuration collapse hazard data set, it also includes: executing the following random one of the processing: obtaining a labeled classification knowledge fragment set for a random piece of configuration collapse hazard data; performing important content classification processing on a random piece of configuration collapse hazard data through an important content classification thread to obtain a labeled classification knowledge fragment; wherein the important content classification thread is different from the collapse hazard data processing thread.

[0061] Here, for the configuration collapse hazard data in the configuration collapse hazard data set, the labeled classification knowledge fragments can be obtained in the following two ways.

[0062] The embodiment of the present application does not limit the type of the important content classification thread, for example, it can be a neural model thread, etc. Before the important content classification thread performs important content classification processing on the configuration collapse risk data, the important content classification thread can also be pre-configured.

[0063] In step 203, the secondary knowledge fragment cost index values, the important knowledge fragment cost index values ​​and the important content classification cost index values ​​of the configuration collapse potential data set are spliced, and the collapse potential data processing thread is configured according to the obtained splicing cost index values.

[0064] Here, the secondary knowledge fragment cost index values, important knowledge fragment cost index values ​​and important content classification cost index values ​​of the configured collapse potential risk data set are spliced, and reverse decision processing is performed in the collapse potential risk data processing thread based on the obtained splicing cost index values ​​to realize the configuration of the collapse potential risk data processing thread.

[0065] It should be understood that the configuration process of the collapse potential risk data processing thread is essentially a process of updating the weight parameters in the collapse potential risk data processing thread.

[0066] Step 101 may be updated to step 204. In step 204, the configured collapse hazard data processing thread performs knowledge segment extraction processing on the example collapse hazard data to obtain secondary knowledge segments and important knowledge segments of the example collapse hazard data.

[0067] Step 103 may be updated to step 205. In step 205, the configured collapse hazard data processing thread performs knowledge segment extraction processing on the collapse hazard data to be processed to obtain secondary knowledge segments and important knowledge segments of the collapse hazard data to be processed.

[0068] When the configuration of the collapse potential danger data processing thread is completed, the knowledge fragment extraction and processing of the example collapse potential danger data and the collapse potential danger data to be processed can be realized through the configured collapse potential danger data processing thread.

[0069] The embodiment of the present application configures the collapse hazard data processing thread based on the principle of artificial intelligence, and performs knowledge fragment extraction processing through the configured collapse hazard data processing thread, which can effectively improve the accuracy of knowledge fragment extraction processing.

[0070] In step 301, the original knowledge fragment extraction model is used to extract the original knowledge fragments of the configuration collapse potential hazard data to obtain the original knowledge fragments of the configuration collapse potential hazard data.

[0071] In an embodiment of the present application, the collapse hazard data processing thread can mainly include three parts: an original knowledge fragment extraction model (also called a basic knowledge fragment extraction model), a secondary branch model and an important branch model, wherein the secondary branch model and the important branch model share the original knowledge fragment extraction model; the secondary branch model includes the secondary knowledge fragment extraction model; the important branch model includes the important knowledge fragment extraction model and the important content classification model.

[0072] In the process of forward decision processing of configuration collapse hazard data through collapse hazard data processing thread, the configuration collapse hazard data is firstly processed by original knowledge fragment extraction model to obtain original knowledge fragments of configuration collapse hazard data.

[0073] It should be understood that, before step 301, the original knowledge fragment extraction model may be configured separately, such as by configuring through an open source collapse hazard data set, thereby improving the overall configuration effect of the collapse hazard data processing thread.

[0074] In step 302, secondary knowledge fragment extraction processing is performed on the original knowledge fragments of the configuration collapse hazard data through the secondary knowledge fragment extraction model to obtain secondary knowledge fragments of the configuration collapse hazard data.

[0075] Here, the output result of the original knowledge fragment extraction model (ie, the original knowledge fragment) is subjected to secondary knowledge fragment extraction processing by the secondary knowledge fragment extraction model to obtain secondary knowledge fragments for configuring collapse hazard data.

[0076] In step 303, the important knowledge fragment extraction model is used to extract the important knowledge fragments from the original knowledge fragments of the configuration collapse hazard data to obtain the important knowledge fragments of the configuration collapse hazard data.

[0077] Here, the important knowledge fragment extraction model is used to perform important knowledge fragment extraction processing on the output results of the original knowledge fragment extraction model to obtain important knowledge fragments for configuring the collapse hazard data.

[0078] In step 304, important content classification processing is performed on the important knowledge fragments of the configuration collapse hazard data by using an important content classification model to obtain predicted classification knowledge fragments of the configuration collapse hazard data.

[0079] Here, the output results of the important knowledge fragment extraction model (ie, the important knowledge fragments) are subjected to important content classification processing through the important content classification model to obtain the predicted classified knowledge fragments of the configuration collapse hazard data.

[0080] In step 305, the secondary knowledge fragment cost index values, the important knowledge fragment cost index values ​​and the important content classification cost index values ​​of the configuration collapse hazard data set are spliced ​​to obtain a spliced ​​cost index value.

[0081] In step 306, a secondary knowledge segment extraction model, an important knowledge segment extraction model and an important content classification model are configured according to the splicing cost index value.

[0082] The embodiment of the present application provides two configuration methods. The first configuration method is to determine the secondary knowledge fragment extraction model, the important knowledge fragment extraction model and the important content classification model as models that need to be configured, and configure them according to the splicing cost index value. This method has a high configuration efficiency.

[0083] In step 307, the original knowledge fragment extraction model, the secondary knowledge fragment extraction model, the important knowledge fragment extraction model and the important content classification model are configured according to the splicing cost index value.

[0084] The second configuration method is to determine the original knowledge fragment extraction model, the secondary knowledge fragment extraction model, the important knowledge fragment extraction model and the important content classification model as the models that need to be configured, and configure them according to the splicing cost index value. This method has a better configuration effect, but it may take longer configuration time than the first configuration method.

[0085] In some possible implementations, in the process of configuring the collapse potential data processing thread, the configuration efficiency of the important content classification model is greater than the model in the collapse potential data processing thread that is different from the important content classification model.

[0086] In the configuration process of the collapse hazard data processing thread, compared with other models, the important content classification model is more likely to be over-integrated (because the task of important content classification is to output the same prediction category for two collapse hazard data with the same important category, that is, to integrate the same learning goal, and the predictions classified under over-integration are very similar to the prediction categories of multiple dissimilar collapse hazard data), and this important over-integration will make other models more likely to be over-integrated, that is, the knowledge fragments (referring to the secondary knowledge fragments / important knowledge fragments) of two collapse hazard data with the same important category are too similar (or even the same), resulting in no distinction between the knowledge fragments of different collapse hazard data. Therefore, in an embodiment of the present application, an asynchronous configuration efficiency is adopted, that is, the configuration efficiency of the important content classification model is greater than that of other models that need to be configured in the collapse hazard data processing thread, wherein the configuration efficiency is the update efficiency of the weight parameters, also known as the learning rate. In this way, it can be ensured that the weight parameters of other models are updated more slowly than the weight parameters of the important content classification model, effectively avoiding the occurrence of over-integration, thereby improving the configuration effect of the collapse hazard data processing thread.

[0087] In some possible embodiments, before step 201, in step 401, each of the plurality of configuration collapse potential risk data sets may be clustered to obtain a plurality of cluster directories respectively; wherein the cluster directory includes a plurality of similar configuration collapse potential risk data.

[0088] For each of the plurality of configuration collapse potential risk data sets, clustering is performed on the configuration collapse potential risk data sets to obtain a plurality of cluster directories, wherein the cluster directories include a plurality of similar configuration collapse potential risk data, and the clustering process can be implemented with the aid of open source tools. After obtaining the plurality of cluster directories, the plurality of cluster directories can be further selected.

[0089] In step 402, several cluster directories are mined to obtain several configuration collapse hazard data sets; wherein the configuration collapse hazard data sets include at least one of the following: several configuration collapse hazard data belonging to the same cluster directory; several configuration collapse hazard data belonging to different configuration collapse hazard data sets.

[0090] For example, mining processing (configuration collapse hidden danger data mining processing) is performed in all cluster directories obtained through step 401 to obtain several configuration collapse hidden danger data sets, wherein each configuration collapse hidden danger data set includes at least one of the following: several configuration collapse hidden danger data belonging to the same cluster directory (the similarity relationship between the several configuration collapse hidden danger data is similar); several configuration collapse hidden danger data belonging to different configuration collapse hidden danger data sets respectively (the similarity relationship between the several configuration collapse hidden danger data is dissimilar).

[0091] The mining processing strategy in step 402 may be adjusted accordingly according to the number of configuration collapse hazard data required to be included in the configuration collapse hazard data set and the similarity relationship between the configuration collapse hazard data. For example, the following situations may exist.

[0092] 1) The configuration collapse hidden danger data set is a binary group, which includes two types. One is that the two configuration collapse hidden danger data in the binary group are similar. For this, two configuration collapse hidden danger data can be randomly mined from the same cluster directory of a certain configuration collapse hidden danger data set to build a binary group; the second is that the two configuration collapse hidden danger data in the binary group are not similar. For this, one configuration collapse hidden danger data can be randomly mined from two cluster directories from different configuration collapse hidden danger data sets to build a binary group. It should be understood that the cost indicator strategy for metric learning corresponding to the binary group can be a contrastive loss indicator.

[0093] 2) The configuration collapse hazard data set is a triplet, which includes a first configuration collapse hazard data, a second configuration collapse hazard data similar to the first configuration collapse hazard data, and a third configuration collapse hazard data that is not similar to the first configuration collapse hazard data. In view of this, two configuration collapse hazard data can be randomly mined from the same cluster directory of a certain configuration collapse hazard data set, and respectively determined as the first configuration collapse hazard data and the second configuration collapse hazard data; a configuration collapse hazard data can be randomly mined from a random cluster directory of another configuration collapse hazard data set to be determined as the third configuration collapse hazard data.

[0094] 3) The configuration collapse hazard data set is a quaternary group, which includes the first configuration collapse hazard data, the second configuration collapse hazard data similar to the first configuration collapse hazard data, the third configuration collapse hazard data dissimilar to the first configuration collapse hazard data, and the fourth configuration collapse hazard data dissimilar to the first configuration collapse hazard data. In view of this, two configuration collapse hazard data can be randomly mined from the same cluster directory of a certain configuration collapse hazard data set, and respectively determined as the first configuration collapse hazard data and the second configuration collapse hazard data; a configuration collapse hazard data can be randomly mined from a random cluster directory of another configuration collapse hazard data set to be determined as the third configuration collapse hazard data; a configuration collapse hazard data can be randomly mined from a random cluster directory of another configuration collapse hazard data set to be determined as the fourth configuration collapse hazard data. Among them, the first configuration collapse hazard data, the third configuration collapse hazard data and the fourth configuration collapse hazard data correspond to different configuration collapse hazard data sets. It should be understood that the cost indicator strategy corresponding to the quadruplet for metric learning may be a quadruplet cost indicator (QuadrupletLoss).

[0095] In some possible embodiments, in step 501, a correspondence between a sample collapse hazard data set where the sample collapse hazard data is located and a target description tag can be established based on the correspondence between the sample collapse hazard data and the target description tag; wherein the target description tag includes a random one of a secondary description tag and an important description tag.

[0096] In addition to supporting the analysis of similar collapse hazard data, the embodiments of the present application can also support the analysis of similar collapse hazard data sets. For example, the example collapse hazard data is obtained by mining the example collapse hazard data set, and the to-be-processed collapse hazard data is obtained by mining the to-be-processed collapse hazard data set. Under this premise, for each example collapse hazard data, the correspondence between the example collapse hazard data set where the example collapse hazard data is located and the target description label can be established based on the correspondence between the example collapse hazard data set and the target description label; wherein the target description label includes a random one of a secondary description label and an important description label. In this way, the correspondence between the example collapse hazard data set and the secondary description label, as well as the correspondence between the example collapse hazard data set and the important description label can be obtained.

[0097] After step 105, in step 502, the example collapse hazard data set where the collapse hazard warning information is located may be determined as a similar example collapse hazard data set to the collapse hazard data set to be processed where the collapse hazard data to be processed is located.

[0098] Here, after obtaining the collapse hazard warning information of the collapse hazard data to be processed, the example collapse hazard dataset where the collapse hazard warning information is located can be determined as a similar example collapse hazard dataset of the collapse hazard dataset to be processed where the collapse hazard data to be processed is located, that is, similar collapse hazard dataset analysis is realized.

[0099] In step 503, a correspondence is established between the collapse hazard data set to be processed and the similar target description tags obtained by similarity analysis processing; wherein the similar target description tags include a random one of similar secondary description tags and similar important description tags.

[0100] Here, a correspondence between the to-be-processed collapse hazard dataset and similar secondary description tags obtained through similarity analysis and a correspondence between the to-be-processed collapse hazard dataset and similar important description tags obtained through similarity analysis are established.

[0101] In step 504, a plurality of similar example collapse hazard data sets are distributedly processed according to the correspondence between the similar example collapse hazard data sets and the target description labels, and the correspondence between the to-be-processed collapse hazard data sets and the similar target description labels.

[0102] For each similar example collapse hazard dataset, the distribution score of the similar example collapse hazard dataset is determined based on the correspondence between the similar example collapse hazard dataset and the target description label, and the correspondence between the collapse hazard dataset to be processed and the similar target description label. Then, several similar example collapse hazard datasets are distributed and processed according to the distribution score.

[0103] In some possible embodiments, the above-mentioned distribution processing of several similar example collapse hazard datasets based on the correspondence between similar example collapse hazard datasets and target description labels, and the correspondence between the collapse hazard dataset to be processed and similar target description labels can be achieved in the following way: performing one of the following random processing: distributing several similar example collapse hazard datasets based on the intersection-and-union ratio between important description labels corresponding to similar example collapse hazard datasets and similar important description labels corresponding to the collapse hazard dataset to be processed; performing splicing processing on the intersection-and-union ratio between secondary description labels corresponding to similar example collapse hazard datasets and similar secondary description labels corresponding to the collapse hazard dataset to be processed, and the intersection-and-union ratio between important description labels corresponding to similar example collapse hazard datasets and similar important description labels corresponding to the collapse hazard dataset to be processed, and performing distribution processing on several similar example collapse hazard datasets based on the obtained spliced ​​intersection-and-union comparison.

[0104] In an embodiment of the present application, distribution processing can be implemented based on important description tags. For example, the intersection-and-union ratio between the important description tags corresponding to the similar example collapse hazard dataset and the similar important description tags corresponding to the collapse hazard dataset to be processed is determined, the intersection-and-union ratio is determined as the important distribution score, and distribution processing is performed on several similar example collapse hazard datasets based on the important distribution score. The larger the important distribution score, the more similar the corresponding similar example collapse hazard dataset is to the collapse hazard dataset to be processed in terms of importance, and therefore the distribution of the similar example collapse hazard dataset is closer to the front.

[0105] In an embodiment of the present application, distribution processing can be implemented in combination with secondary description tags and important description tags. For example, the intersection-and-union ratio between the secondary description tags corresponding to the similar example collapse hazard data set and the similar secondary description tags corresponding to the collapse hazard data set to be processed, as well as the intersection-and-union ratio between the important description tags corresponding to the similar example collapse hazard data set and the similar important description tags corresponding to the collapse hazard data set to be processed are spliced, and the obtained spliced ​​intersection-and-union ratio is determined as a splicing distribution score, and a number of similar example collapse hazard data sets are distributed according to the splicing distribution score. The larger the splicing distribution score, the more similar the corresponding similar example collapse hazard data set is to the collapse hazard data set to be processed in terms of comprehensive degree (comprehensive secondary and important), and therefore the distribution of the similar example collapse hazard data set is closer.

[0106] For the collapse hazard data to be processed, knowledge fragments are similarly extracted to obtain collapse hazard data knowledge fragments, and then the collapse hazard data knowledge fragments of the collapse hazard data to be processed are compared with several cluster centers, and collapse hazard warning information of the collapse hazard data to be processed is selected from the collapse hazard database according to the comparison results. For example, the example collapse hazard data corresponding to the cluster center that is most different from the collapse hazard data knowledge fragment of the collapse hazard data to be processed can be determined as the collapse hazard warning information. On this basis, the example collapse hazard data set where the collapse hazard warning information is located can also be determined as a similar example collapse hazard data set of the collapse hazard data set to be processed where the collapse hazard data to be processed is located.

[0107] It should be understood that when determining the cluster center that is closest to the collapse hazard data knowledge segment difference of the collapse hazard data to be processed, the collapse hazard data knowledge segment of the example collapse hazard data corresponding to the cluster center can also be compared with the collapse hazard data knowledge segment of the collapse hazard data to be processed. For example, when the difference between the collapse hazard data knowledge segment of a random example collapse hazard data corresponding to the cluster center and the collapse hazard data knowledge segment of the collapse hazard data to be processed is less than the set difference threshold, the random example collapse hazard data is determined as collapse hazard warning information. In this way, further selection of example collapse hazard data can be achieved.

[0108] In addition, after determining the example collapse hazard data set where the collapse hazard warning information is located, the collapse hazard data knowledge segments of the example collapse hazard data set can also be compared with the collapse hazard data set to be processed one by one. For example, determine the difference between the collapse hazard data knowledge segment of the first collapse hazard data of the example collapse hazard data set and the collapse hazard data knowledge segment of the first collapse hazard data of the collapse hazard data set to be processed, and the difference between the collapse hazard data knowledge segment of the second collapse hazard data of the example collapse hazard data set and the collapse hazard data knowledge segment of the second collapse hazard data of the collapse hazard data set to be processed... When the difference between the example collapse hazard data set and the collapse hazard data set to be processed after comparing each collapse hazard data knowledge segment is less than the difference threshold, the example collapse hazard data set is determined as a similar example collapse hazard data set, so that further selection of the example collapse hazard data set can be achieved.

[0109] For ease of understanding, the following text will take the scenario of analyzing a similar collapse hazard data set as an example to illustrate the collapse hazard warning method based on big data provided by the embodiment of the present application in the form of steps.

[0110] Configure the collapse potential data processing thread. A collapse potential data processing thread is provided. The collapse potential data processing thread mainly includes three units, namely the original knowledge fragment extraction model, the secondary branch model and the important branch model.

[0111] On the basis of the above, a collapse hazard early warning device based on big data is provided, the device comprising: A knowledge fragment extraction module is used to perform knowledge fragment extraction processing on the example collapse hidden danger data to obtain secondary knowledge fragments and important knowledge fragments of the example collapse hidden danger data; A label building module, used to build secondary description labels according to the secondary knowledge fragments corresponding to the several example collapse hazard data, and to build important description labels according to the important knowledge fragments corresponding to the several example collapse hazard data; A knowledge fragment obtaining module is used to extract knowledge fragments from the collapse hazard data to be processed, and obtain secondary knowledge fragments and important knowledge fragments of the collapse hazard data to be processed; A tag analysis module, used for performing similarity analysis processing on the secondary description tags according to the secondary knowledge fragments of the collapse hazard data to be processed to obtain similar secondary description tags, and performing similarity analysis processing on the important description tags according to the important knowledge fragments of the collapse hazard data to be processed to obtain similar important description tags; The information module is used to determine the example collapse hazard data corresponding to the similar secondary description tags and at least part of the example collapse hazard data corresponding to the similar important description tags as the collapse hazard warning information of the collapse hazard data to be processed.

[0112] On the basis of the above, a collapse hazard warning system based on big data is shown, which includes a processor and a memory that communicate with each other, and the processor is used to read and execute a computer program from the memory to implement the above method.

[0113] Based on the above, a computer-readable storage medium is also provided, on which a computer program stored implements the above method when running.

[0114] In summary, based on the above scheme, the example collapse hazard data is processed by knowledge fragment extraction to obtain secondary knowledge fragments and important knowledge fragments, and secondary description labels are built according to the secondary knowledge fragments corresponding to several example collapse hazard data. At the same time, important description labels are built according to the important knowledge fragments corresponding to several example collapse hazard data. In this way, similar collapse hazard data analysis is implemented for the collapse hazard data to be processed in the secondary and important branches, and attention can be paid to knowledge fragments of collapse hazard data in different aspects at the same time, thereby improving the analysis accuracy, so that the collapse hazard can be accurately obtained and early warning can be given. In this way, the harm caused by the collapse can be reduced as much as possible.

[0115] It should be understood that the system and its modules shown above can be implemented in various ways. For example, in some embodiments, the system and its modules can be implemented by hardware, software, or a combination of software and hardware. Among them, the hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or a dedicated design hardware. Those skilled in the art will understand that the above methods and systems can be implemented using computer executable instructions and / or included in a processor control code, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The system and its modules of the present application can not only be implemented by hardware circuits such as ultra-large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., but can also be implemented by software such as executed by various types of processors, and can also be implemented by a combination of the above hardware circuits and software (e.g., firmware).

[0116] It should be noted that different embodiments may produce different beneficial effects. In different embodiments, the beneficial effects that may be produced may be any one or a combination of the above, or any other beneficial effects that may be obtained.

Claims

1. A collapse hazard early warning method based on big data, characterized in that: The method comprises: Performing knowledge segment extraction processing on the example collapse hazard data to obtain secondary knowledge segments and important knowledge segments of the example collapse hazard data; Building secondary description tags according to secondary knowledge fragments corresponding to several of the exemplary collapse hazard data, and building important description tags according to important knowledge fragments corresponding to several of the exemplary collapse hazard data; Performing knowledge segment extraction processing on the collapse hazard data to be processed to obtain secondary knowledge segments and important knowledge segments of the collapse hazard data to be processed; Performing similarity analysis on the secondary description tags according to the secondary knowledge fragments of the to-be-processed collapse hazard data to obtain similar secondary description tags, and performing similarity analysis on the important description tags according to the important knowledge fragments of the to-be-processed collapse hazard data to obtain similar important description tags; The example collapse hazard data corresponding to the similar minor description tags and at least part of the example collapse hazard data corresponding to the similar major description tags are determined as collapse hazard warning information of the collapse hazard data to be processed.

2. The method according to claim 1, characterized in that Before the knowledge fragment extraction process is performed on the example collapse hazard data, the method further includes: Performing knowledge fragment extraction processing on the configuration collapse hidden danger data in the configuration collapse hidden danger data set through the collapse hidden danger data processing thread to obtain secondary knowledge fragments and important knowledge fragments of the configuration collapse hidden danger data, and performing important content classification processing on the important knowledge fragments of the configuration collapse hidden danger data to obtain predicted classification knowledge fragments; Performing cost index calculation processing on the secondary knowledge fragments, the important knowledge fragments and the predicted classification knowledge fragments of the configuration collapse hidden danger data respectively, to obtain the secondary knowledge fragment cost index value, the important knowledge fragment cost index value and the important content classification cost index value corresponding to the configuration collapse hidden danger data set; Performing splicing processing on the secondary knowledge fragment cost index value, the important knowledge fragment cost index value and the important content classification cost index value of the configuration collapse potential risk data set, and configuring the collapse potential risk data processing thread according to the obtained splicing cost index value; The configured collapse potential risk data processing thread is used to perform knowledge segment extraction processing on the example collapse potential risk data and the to-be-processed collapse potential risk data.

3. The method according to claim 2, characterized in that The collapse hidden danger data processing thread includes an original knowledge fragment extraction model, a secondary knowledge fragment extraction model, an important knowledge fragment extraction model and an important content classification model; the collapse hidden danger data processing thread performs knowledge fragment extraction processing on the configuration collapse hidden danger data in the configuration collapse hidden danger data set to obtain the secondary knowledge fragments and important knowledge fragments of the configuration collapse hidden danger data, including: Performing original knowledge fragment extraction processing on the configuration collapse hidden danger data by using the original knowledge fragment extraction model to obtain original knowledge fragments of the configuration collapse hidden danger data; Performing secondary knowledge segment extraction processing on the original knowledge segments of the configuration collapse hidden danger data by using the secondary knowledge segment extraction model to obtain the secondary knowledge segments of the configuration collapse hidden danger data; Performing important knowledge fragment extraction processing on the original knowledge fragments of the configuration collapse hidden danger data by using the important knowledge fragment extraction model to obtain the important knowledge fragments of the configuration collapse hidden danger data; The important knowledge fragments of the configuration collapse hidden danger data are classified into important contents to obtain predicted classified knowledge fragments, including: The important content classification model is used to perform important content classification processing on the important knowledge fragments of the configuration collapse hidden danger data to obtain the predicted classification knowledge fragments of the configuration collapse hidden danger data.

4. The method according to claim 3, characterized in that The configuring the collapse potential data processing thread according to the obtained splicing cost index value includes: Perform one of the following random operations: configuring the minor knowledge fragment extraction model, the important knowledge fragment extraction model and the important content classification model according to the splicing cost index value; configuring the original knowledge fragment extraction model, the secondary knowledge fragment extraction model, the important knowledge fragment extraction model and the important content classification model according to the splicing cost index value; Among them, the configuration efficiency of the important content classification model is greater than the model in the collapse potential data processing thread that is different from the important content classification model.

5. The method according to claim 2, characterized in that: The configuration collapse hazard data set includes first configuration collapse hazard data, second configuration collapse hazard data similar to the first configuration collapse hazard data, and third configuration collapse hazard data not similar to the first configuration collapse hazard data; the cost index calculation processing is performed on the secondary knowledge fragments, the important knowledge fragments and the predicted classification knowledge fragments of the configuration collapse hazard data respectively to obtain the secondary knowledge fragment cost index value, the important knowledge fragment cost index value and the important content classification cost index value corresponding to the configuration collapse hazard data set, including: Determining a secondary knowledge segment cost index value of the configuration collapse hazard data set according to a secondary difference between the secondary knowledge segment of the first configuration collapse hazard data and the secondary knowledge segment of the second configuration collapse hazard data, and a secondary difference between the secondary knowledge segment of the first configuration collapse hazard data and the secondary knowledge segment of the third configuration collapse hazard data; Determine the important knowledge fragment cost index value of the configuration collapse hazard data set according to the important difference between the important knowledge fragment of the first configuration collapse hazard data and the important knowledge fragment of the second configuration collapse hazard data, and the important difference between the important knowledge fragment of the first configuration collapse hazard data and the important knowledge fragment of the third configuration collapse hazard data; Determine the important content classification cost index value of a random configuration collapse hazard data in the configuration collapse hazard data set according to the labeled classification knowledge fragment and the predicted classification knowledge fragment of the random configuration collapse hazard data, and perform splicing processing on the important content classification cost index values ​​corresponding to the first configuration collapse hazard data, the second configuration collapse hazard data and the third configuration collapse hazard data, respectively, to obtain the important content classification cost index value of the configuration collapse hazard data set; Wherein, before determining the important content classification cost index value of a random configuration collapse hazard data according to the labeled classification knowledge fragment and the predicted classification knowledge fragment of the random configuration collapse hazard data in the configuration collapse hazard data set, the method further includes: Perform one of the following random operations: Obtain a labeled classification knowledge fragment for the random configuration collapse hazard data setting; Performing important content classification processing on the random configuration collapse potential risk data through an important content classification thread to obtain a labeled classification knowledge fragment; The important content classification thread is different from the collapse potential data processing thread.

6. The method according to claim 2, characterized in that Before performing knowledge segment extraction processing on the configuration collapse potential risk data in the configuration collapse potential risk data set by the collapse potential risk data processing thread, the method further comprises: Performing clustering processing on each of the plurality of configuration collapse potential risk data sets to obtain a plurality of cluster directories respectively; wherein the cluster directories include a plurality of similar configuration collapse potential risk data; Performing mining processing on the plurality of cluster directories to obtain a plurality of configuration collapse hidden danger data sets; The configuration collapse potential risk data set includes at least one of the following: a plurality of configuration collapse potential risk data belonging to the same cluster directory; or a plurality of configuration collapse potential risk data belonging to different configuration collapse potential risk data sets.

7. The method according to any one of claims 1 to 6, characterized in that: The example collapse hazard data is obtained by mining the example collapse hazard data set, and the to-be-processed collapse hazard data is obtained by mining the to-be-processed collapse hazard data set; after determining the example collapse hazard data corresponding to the similar minor description tags and at least part of the example collapse hazard data corresponding to the similar major description tags as collapse hazard warning information of the to-be-processed collapse hazard data, the method further includes: Determine the example collapse hazard data set where the collapse hazard warning information is located as a similar example collapse hazard data set to the collapse hazard data set to be processed; Wherein, the method further comprises: According to the correspondence between the example collapse hazard data and the target description tag, a correspondence between the example collapse hazard data set where the example collapse hazard data is located and the target description tag is established; wherein the target description tag includes a random one of the secondary description tag and the important description tag; Establishing a correspondence between the to-be-processed collapse hazard data set and the similar target description label obtained by the similarity analysis process; wherein the similar target description label includes a random one of the similar secondary description label and the similar important description label; According to the correspondence between the similar example collapse hazard data set and the target description label, and the correspondence between the to-be-processed collapse hazard data set and the similar target description label, a plurality of the similar example collapse hazard data sets are distributedly processed; The method of performing distributed processing on a plurality of similar example collapse hazard data sets according to the correspondence between the similar example collapse hazard data sets and the target description labels, and the correspondence between the to-be-processed collapse hazard data sets and the similar target description labels, comprises: Perform one of the following random operations: According to the intersection-and-union ratio between the important description labels corresponding to the similar example collapse hazard data sets and the similar important description labels corresponding to the collapse hazard data sets to be processed, performing distribution processing on a plurality of the similar example collapse hazard data sets; The intersection-and-union ratios of the secondary description labels corresponding to the similar example collapse hazard dataset and the similar secondary description labels corresponding to the collapse hazard dataset to be processed, as well as the intersection-and-union ratios of the important description labels corresponding to the similar example collapse hazard dataset and the similar important description labels corresponding to the collapse hazard dataset to be processed are spliced, and based on the obtained spliced ​​intersection-and-union comparisons, several different similar example collapse hazard datasets are distributed.

8. The method according to any one of claims 1 to 6, characterized in that: The method of constructing secondary description tags based on secondary knowledge fragments corresponding to a plurality of the exemplary collapse hazard data, and constructing important description tags based on important knowledge fragments corresponding to a plurality of the exemplary collapse hazard data, includes: Clustering the secondary knowledge fragments corresponding to the plurality of example collapse hazard data, and determining the obtained plurality of secondary cluster centers as secondary description labels; Clustering is performed on the important knowledge fragments corresponding to the several example collapse hazard data, and the several obtained important cluster centers are determined as important description labels; The method further comprises: Determine the secondary differences between the secondary knowledge segments of the example collapse hazard data and a plurality of the secondary description tags, respectively, and establish a corresponding relationship between the example collapse hazard data and the secondary description tags that meet the secondary difference requirements; The important differences between the important knowledge fragments of the example collapse hazard data and a plurality of the important description tags are respectively determined, and a corresponding relationship between the example collapse hazard data and the important description tags that meet the requirements of the important differences is established.

9. The method according to any one of claims 1 to 6, characterized in that: The performing similarity analysis on the secondary description tags according to the secondary knowledge fragments of the collapse hazard data to be processed to obtain similar secondary description tags includes: Determine the secondary differences between the secondary knowledge fragments of the to-be-processed collapse hazard data and a plurality of the secondary description tags respectively, and determine the secondary description tags that meet the secondary difference requirements as similar secondary description tags; The performing similarity analysis on the important description tags according to the important knowledge fragments of the collapse hazard data to be processed to obtain similar important description tags includes: The important differences between the important knowledge fragments of the to-be-processed collapse hazard data and a plurality of the important description tags are respectively determined, and the important description tags meeting the important difference requirements are determined as similar important description tags.

10. A collapse hazard early warning system based on big data, characterized in that: The invention comprises a processor and a memory communicating with each other, wherein the processor is used to read a computer program from the memory and execute the computer program to implement the method according to any one of claims 1 to 9.

Images