An open source information collection and integration system and method based on geographic grid coding

Abstract

The application discloses the technical field of information collection and discloses an open-source information collection and integration system and method based on geographic grid coding, comprising: obtaining a geographic area to be divided and information to be determined; after the geographic area to be divided is divided according to a geographic spatial grid hierarchy system, geographic grid coding is performed on each grid unit; after all information elements in the grid are stored based on the geographic grid coding, entity relationship extraction is performed to generate a triple knowledge graph; after the geographic grid whose triple knowledge graph is generated is subjected to multi-source heterogeneous data cleaning and disambiguation and event time mapping; based on the geographic grid after the event time mapping is completed, the information to be determined is subjected to association relationship determination and credibility determination. The application converts any geographic area into a calculable grid set by using global split grid coding, crawls multi-source and multi-modal open-source information in the area, and realizes rapid convergence, cross-verification and credible presentation of information.

Description

TECHNICAL FIELD

[0001] The present application relates to an open source information collection and integration system and method based on geographic grid coding, belonging to the technical field of information collection. BACKGROUND

[0002] With the rapid development of Internet technology, the information on the network is growing explosively, and open source information has important value for decision making, risk assessment, situation awareness, etc. in various fields. However, the current open source information collection method often lacks precise focus on a specific spatial range. When collecting information in a certain area, it is difficult to quickly and accurately define the boundary of the search, resulting in too large or too small a range of collected information, which cannot meet the actual needs. For example, when collecting commercial information at the city level, a large amount of irrelevant information from surrounding cities or rural areas may be mixed in, or some key information from the edge areas of the city may be missed.

[0003] The existing open source information collection method often lacks precise focus on a specific spatial range, and it is difficult to quickly and accurately define the boundary of the search in a complex spatial range when collecting information in a certain area. SUMMARY

[0004] The purpose of the present application is to overcome the deficiencies in the prior art, and to provide an open source information collection and integration system and method based on geographic grid coding, which collects relevant information in the region based on the spatial range defined by the geographic coding grid, and summarizes and refines it to improve the accuracy and speed of open source information search.

[0005] To achieve the above purpose, the present application adopts the following technical solutions:

[0006] In a first aspect, the present application provides an open source information collection and integration method based on geographic grid coding, comprising:

[0007] Obtaining a geographic area to be divided and information to be determined;

[0008] After dividing the geographic area to be divided into a geographic spatial grid hierarchy system, performing geographic grid coding on each grid unit;

[0009] After storing all information elements in the grid based on geographic grid coding, performing entity relationship extraction to generate a triple knowledge graph;

[0010] After generating the triple knowledge graph, performing multi-source heterogeneous data cleaning and disambiguation and event time mapping on the geographic grid;

[0011] Based on the geographic grid after the event time mapping has been completed, determining the association relationship and the credibility of the information to be determined, and completing the open source information collection and integration based on the determination result.

[0012] Further, after the geographical region to be divided is divided by the geographical spatial grid hierarchy system, each grid unit is geographically grid coded, including: the earth surface is divided from high to low into multiple levels, wherein the grid longitude span and latitude span of the kth level are:

[0013] Δλ k =360 / (2^k×m)

[0014] Δφ k =180 / (2^k×n)

[0015] In the formula: Δλ k is the longitude span, Δφ k is the latitude span, k is the level number of the grid, and m and n are constants related to the level;

[0016] For each grid unit, the level information, longitude direction index and latitude direction index are coded, and the expression is:

[0017] ID=k×10^(2L)+i×10^L+j

[0018] i=floor((λ-λ0) / Δλ k )

[0019] j=floor((φ-φ0) / Δφ k )

[0020] In the formula: ID is the unique code of the grid, L is the maximum number of indices i and j, i is the index of the longitude direction, j is the index of the latitude direction, λ is the central longitude of the grid unit, φ is the central latitude of the grid unit, λ0 is the starting longitude, φ0 is the starting latitude, and floor is the floor function.

[0021] Further, after storing all information elements in the grid based on the geographical grid coding, entity relationship extraction is performed to generate a triple knowledge graph, including:

[0022] Taking the grid ID as the key, all information elements in the grid are stored, and the correspondence between the grid and the information elements is established, and the information elements include domain name, IP, social account and event report;

[0023] The entity relationship extraction model based on BERT is used to extract the entity relationship of the text / picture, and for the extracted results, a triple knowledge graph containing grid ID, theme word and confidence is generated, and the calculation formula of the confidence is:

[0024] C=α×P+β×R

[0025] Wherein, C is the confidence, P is the accuracy of entity relation extraction, R is the ratio of the frequency of the information element appearing in the grid to the total frequency, and a and b are weight coefficients and a+b=1.

[0026] Further, the grid after generating the triple knowledge graph is subjected to multi-source heterogeneous data cleaning and disambiguation and event time mapping, including:

[0027] Fusion of Gaode, Baidu, Google geocoding API and self-built postcode / POI knowledge graph, for the address information to be processed, each API is called for geocoding to obtain multiple candidate coordinates, the similarity of each candidate coordinate and the corresponding address coordinate in the self-built knowledge graph is calculated, and the highest similarity coordinate is selected as the final coordinate of the address, a time convolution network is used to predict the real occurrence time interval of the event, the input of the time convolution network is a feature vector containing the publishing time and the time description in the event related text, and the output is the time interval of the event occurrence, and the predicted event time interval is mapped to the corresponding grid, and the mapping rule is that if there is an effective time coverage of the grid in the event occurrence time interval, the event is mapped to the grid;

[0028] The similarity calculation formula is:

[0029] S=1-d / D max

[0030] In the formula: S is the similarity, d is the distance between the two coordinates, D max is the maximum distance threshold.

[0031] Further, based on the geographical grid after the event time mapping is completed, the association relationship of the to-be-judged information is determined and the credibility is determined, including:

[0032] Based on the geographical grid after the event time mapping is completed, the time similarity and the geographical similarity of the to-be-judged information are calculated;

[0033] Based on the time similarity and the geographical similarity of the to-be-judged information, the spatio-temporal similarity of the to-be-judged information is calculated;

[0034] Based on the spatio-temporal similarity of the to-be-judged information, the information association degree of the to-be-judged information is calculated;

[0035] The information association degree of the to-be-judged information is compared with the set threshold, and the information association degree is greater than the set threshold, and it is determined that the information has an association relationship, otherwise it is determined that there is no association;

[0036] The credibility total score of the to-be-judged information is calculated through a three-level verification mechanism, and the credibility of the to-be-judged information is determined.

[0037] Further, the calculation formula of the information association degree is:

[0038] R= ×Sst+(1- )×Ss

[0039] Ss=V A V B / (||V A ||×||V B ||)

[0040] Sst=γ×St+(1-γ)×Sg

[0041] St=1-|t A -t B | / T max

[0042] Sg=1-d AB / G max

[0043] In the formula: R is the information correlation degree, is the weight coefficient of space-time similarity, Sst is the space-time similarity, Ss is the semantic similarity, V A is the semantic feature vector of information A, V B is the semantic feature vector of information B, γ is the weight coefficient of time similarity, St is the time similarity, t A is the time stamp of information A, t B is the time stamp of information B, T max is the set maximum time difference threshold, Sg is the geographical similarity, d AB is the distance between the geographical coordinates of information A and information B, G max is the set maximum geographical distance threshold.

[0044] Further, the three-level verification mechanism includes primary verification, secondary verification and tertiary verification, wherein:

[0045] The primary verification includes checking whether the information appears on at least two different types of open source platforms, and if so, it is recorded as passing the primary verification and a primary verification score is given;

[0046] The secondary verification includes analyzing whether the key elements in the information are consistent on the basis of passing the primary verification, calculating the consistency score C key of the key elements, and when C key ≥ a preset threshold b, it is recorded as passing the secondary verification, a secondary verification score is given and the primary verification score is added up;

[0047] The third-level authentication includes judging whether the information is in line with logic and common sense on the basis of the secondary authentication, combining historical data and domain knowledge, calculating the degree of compliance F through logical reasoning and knowledge matching, and marking as passing the third-level authentication when F is greater than or equal to a preset threshold d, and assigning a third-level authentication score and adding the first-level and second-level authentication scores as a total score;

[0048] When the total score is greater than or equal to a preset total score threshold f, it is marked as 'high credibility'; when the preset total score threshold g is less than or equal to the total score and the total score is less than the preset total score threshold f, it is marked as'medium credibility'; and when the total score is less than the preset total score threshold g, it is marked as 'low credibility', wherein the preset total score threshold g is less than the preset total score threshold f.

[0049] In a second aspect, the present application provides an open source information collection and integration system based on geographic grid coding, comprising:

[0050] An information acquisition module: acquiring a geographic area to be divided and information to be determined;

[0051] A grid coding module: after dividing the geographic area to be divided into a geographic spatial grid hierarchy system, performing geographic grid coding on each grid unit;

[0052] A graph generation module: after storing all information elements in the grid based on geographic grid coding, performing entity relationship extraction to generate a triple knowledge graph;

[0053] A data mapping module: performing multi-source heterogeneous data cleaning and disambiguation and event time mapping on the geographic grid after generating the triple knowledge graph;

[0054] A determination and integration module: based on the geographic grid after the event time mapping has been completed, performing association relationship determination and credibility determination on the information to be determined, and completing open source information collection and integration based on the determination result.

[0055] In a third aspect, the present application provides an open source information collection and integration device based on geographic grid coding, comprising a processor and a storage medium;

[0056] The storage medium is used to store instructions;

[0057] The processor is used to operate according to the instructions to perform the steps of the method according to any one of the above.

[0058] In a fourth aspect, the present application provides a computer readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method according to any one of the above.

[0059] Compared with the prior art, the present application has the following beneficial effects:

[0060] The scheme provides a unified "space ruler" for all open source information of mixed sources and different standards through mandatory gridding processing, and fundamentally solves the fusion problem of multi-source heterogeneous spatial data. Meanwhile, the geographic grid code is used as a first-level index to convert complex spatial range queries into efficient one-dimensional code prefix matching, which is particularly suitable for fast screening and aggregation of massive data, which is a significant advantage. More importantly, the structured and standardized grid data is an ideal input for advanced data mining algorithms such as machine learning and deep learning, paving the way for more intelligent application scenarios such as spatio-temporal behavior prediction and anomaly pattern detection. BRIEF DESCRIPTION OF DRAWINGS

[0061] The drawings constituting a part of this application provide further understanding of the application, and the illustrative embodiments of the application and their descriptions serve to explain the application, and do not constitute improper limitations on the application. In the drawings:

[0062] Figure 1 A flowchart of an open source information collection and integration method based on geographic grid coding provided for embodiment one of the application. DETAILED DESCRIPTION

[0063] The application will be described in detail below with reference to the drawings and in combination with the embodiments. It should be noted that the embodiments in the application and the features in the embodiments can be combined with each other without conflict.

[0064] The following detailed description is exemplary and is intended to provide further detailed description of the application. Unless otherwise specified, all technical terms used in the application have the same meaning as generally understood by those skilled in the art to which the application belongs. The terms used in the application are only for the purpose of describing the specific embodiments and are not intended to limit the exemplary embodiments according to the application.

[0065] Embodiment one:

[0066] Please refer to Figure 1 In view of the problems of inaccurate geographical space definition, one-sided information collection, weak integration capability and low credibility in the prior art, the scheme designs an open source information collection and integration method based on geographic grid coding. The scheme takes geographic coding grid as a spatial reference framework, combines intelligent keyword generation and multi-source crawling technology to collect information comprehensively, and then completes information integration and credibility verification through preprocessing, correlation modeling and interactive verification, and finally forms high-quality open source information. The overall technical route is described as follows:

[0067] (1) First, based on the geographic grid coding rules (for example, the Earth Space Grid Coding Rules (GB / T 40087-2021)), determine the geographic space grid hierarchy system division and coding method, and generate a unique code for each grid cell.

[0068] (2) Then, build an open-source information "grid-semantic" double index. With grid ID as the key, store all information elements (domain name, IP, social account, event report, etc.) within the grid, and perform entity relationship extraction on text / pictures to generate (grid ID, topic word, confidence) triplets, realizing "search by grid, find by event".

[0069] (3) Second, multi-source heterogeneous data cleaning and disambiguation. Integrate Gaode / Baidu / Google geocoding API + self-built postcode / POI knowledge graph to solve the problem of same name different address and different name same address. And separate "publishing time" and "event time", predict the real occurrence period of the event through time sequence convolution network (TCN), and map it to the corresponding grid.

[0070] (4) Again, build an information correlation model, build an information correlation model based on spatiotemporal features and semantic features. Spatiotemporal features include information occurrence timestamp and geographic coordinates; semantic features include event theme, keywords, sentiment orientation, etc. By calculating the spatiotemporal similarity and semantic similarity between different information, the correlation between information is established.

[0071] (5) Finally, multi-source information interaction verification. Classify information sources, build a confidence evaluation system, verify the authenticity of information, and set up a three-level verification mechanism. First-level verification: check if the information appears on at least two different types of open-source platforms; second-level verification: analyze whether the key elements (such as time, place, and characters) in the information are consistent; third-level verification: combine historical data and domain knowledge to judge whether the information is logical and reasonable. Information marked as "high credibility" through three-level verification, information marked as "medium credibility" or "low credibility" through one or two-level verification.

[0072] The following further describes the present scheme in detail:

[0073] Aiming at the current open source information collection with keyword or administrative region as the retrieval unit, lack of unified spatial reference, leading to cross-source, cross-scale data difficult to align and open data format, coordinate system, semantic inconsistency, lack of automatic conflict detection and confidence evaluation mechanism, etc. The scheme adopts an open source information collection and integration method based on geographic grid coding. Through geographic grid coding, multi-source heterogeneous data is aggregated, managed, analyzed and evaluated. "Any public information" is converted into "event record with Cell-ID"; through "grid-semantic-time" three-dimensional cubic model, large-scale open source information collection, fusion, retrieval and distribution are realized. Based on the above analysis, the specific steps are as follows:

[0074] S1: Global grid subdivision and coding construction. Based on user demand or existing national standard, the earth surface or a specific region is divided into cells with equal size or different levels. Each cell is assigned a global or regional unique code or identifier.

[0075] S1-1: Division of geographic spatial grid level system. The earth surface is divided into n levels from high to low. The higher the level, the higher the grid precision. The longitude span of the kth level (1≤k≤n) grid is Δλ k And the latitude span Δφ k Is determined according to the following formula:

[0076] Δλ k =360 / (2^k×m)

[0077] Δφ k =180 / (2^k×n)

[0078] Where m and n are constants related to the level, which are set according to actual demand and meet the minimum required precision of the grid at the highest level.

[0079] S1-2: Geographic grid coding. For each grid cell, its code consists of level information, longitude direction index and latitude direction index. Suppose a grid cell is located at the kth level, its index in the longitude direction is i, and its index in the latitude direction is j. Then the unique code ID of the grid can be calculated by the following formula:

[0080] ID=k×10^(2L)+i×10^L+j

[0081] Where L is the maximum number of bits of index i and j, which ensures that i and j can be completely represented. The calculation method of i and j is:

[0082] i=floor((λ-λ0) / Δλ k )

[0083] j=floor((φ-φ0) / Δφ k)

[0084] where λ is the longitude of the center of the grid cell, φ is the latitude of the center of the grid cell, λ0 is the starting longitude (such as -180°), φ0 is the starting latitude (such as -90°), and floor is the floor function.

[0085] S2: Construction of open source information "grid-semantic" double index

[0086] S2-1: Store all information elements in the grid, such as domain name, IP, social account, event report, etc., with the grid ID as the key, and establish the correspondence between the grid and the information elements.

[0087] S2-2: Perform entity relationship extraction on text / picture, using a BERT-based entity recognition model and relationship extraction model. For the extracted results, generate a (grid ID, topic word, confidence) triple knowledge graph, and the confidence C is calculated as follows:

[0088] C = α × P + β × R

[0089] where P is the accuracy of entity relationship extraction, R is the ratio of the frequency of the information element appearing in the grid to the total frequency, α and β are weight coefficients, and α + β = 1. The actual data distribution and business requirements can be adjusted, and the triple realizes the function of "finding things by grid and finding grid by things".

[0090] S3: Multi-source heterogeneous data cleaning and disambiguation and event time mapping

[0091] S3-1: Multi-source heterogeneous data cleaning and disambiguation. Fusion of Gaode / Baidu / Google geocoding API and self-built postcode / POI knowledge graph, for the address information to be processed, call each API for geocoding to get multiple candidate coordinates. Calculate the similarity S between each candidate coordinate and the corresponding address coordinate in the self-built knowledge graph, and the similarity S is calculated as follows:

[0092] S = 1 - d / D max

[0093] where d is the distance between the two coordinates, which can be calculated by latitude and longitude, and D max is the maximum distance threshold. Select the highest similarity coordinate as the final coordinate of the address to solve the problem of same name different address and different name same address.

[0094] S3-2: Event time processing. Separate the "publish time" from the "event time", and use a time convolution network (TCN) to predict the event real occurrence period. The input of TCN is a feature vector containing the publish time, time description in event related text, etc. The output is the time interval of event occurrence. Map the predicted event time interval to the corresponding grid. The mapping rule is: if there is a valid time coverage of the grid in the event occurrence time interval, then map the event to the grid.

[0095] S4: Information correlation model construction

[0096] S4-1: Spatiotemporal similarity calculation

[0097] Let the timestamp of information A be t A , and the geographic coordinates be (λ A , φ A ); the timestamp of information B be t B , and the geographic coordinates be (λ B , φ B ). The calculation formula of time similarity St is:

[0098] St=1-|t A -t B | / T max

[0099] Where T max is the set maximum time difference threshold.

[0100] The calculation formula of geographic similarity Sg is:

[0101] Sg=1-d AB / G max

[0102] Where d AB is the distance between the geographic coordinates of information A and information B, and G max is the set maximum geographic distance threshold.

[0103] Spatiotemporal similarity Sst=γ×St+(1-γ)×Sg, where γ is the weight coefficient of time similarity, 0<γ<1.

[0104] S4-2: Semantic similarity calculation

[0105] The cosine similarity of word vectors is used to calculate the similarity of the theme and keywords of the event, and the sentiment similarity is calculated by the difference of sentiment scores. Let the semantic feature vector of information A be V A , and the semantic feature vector of information B be V B . The calculation formula of semantic similarity Ss is:

[0106] Ss=VA V B / (||V A ||×||V B ||)

[0107] wherein, is the vector dot product, and is the norm of the vector.

[0108] S4-3: Information correlation degree R calculation

[0109] R= x Sst+ (1-5) x Ss, wherein is the weight coefficient of spatiotemporal similarity, 0 <1. When R is greater than a set threshold value, it is considered that the two information have a correlation.

[0110] S5: Multi-source information interaction verification

[0111] S5-1: Information source classification. The information sources are divided into official media, authoritative institutions, ordinary user-generated content, and other different types, and different initial credibility values are assigned to each type.

[0112] S5-2: Confidence evaluation system and three-level verification mechanism.

[0113] First-level verification: Check whether the information appears on at least two different types of open source platforms. If it does, it is considered to pass the first-level verification and is assigned a score a.

[0114] Second-level verification: On the basis of passing the first-level verification, analyze whether the key elements (such as time, place, and person) in the information are consistent. Calculate the consistency score C key of the key elements, and preset a threshold value b. When C key is greater than or equal to the threshold value b, it is considered to pass the second-level verification, and the total score becomes a+c (c is the second-level verification score).

[0115] Third-level verification: On the basis of passing the second-level verification, combine historical data and domain knowledge to judge whether the information is consistent with logic and common sense. Calculate the degree of consistency F through logical reasoning and knowledge matching, and preset a threshold value d. When F is greater than or equal to the threshold value d, it is considered to pass the third-level verification, and the total score becomes a+c+e (e is the third-level verification score).

[0116] S5-3: Credibility calculation: Set total score thresholds f and g. When the total score is greater than or equal to the threshold value f, it is marked as "high credibility"; when the threshold value g is less than or equal to the total score and less than the threshold value f, it is marked as "medium credibility"; and when the total score is less than the threshold value g, it is marked as "low credibility", where g

[0117] Example Two:

[0118] An open source information collection and integration system based on geographic grid coding, which can implement the open source information collection and integration method based on geographic grid coding in embodiment one, comprises:

[0119] An information acquisition module: acquire a geographic area to be divided and to-be-judgment information;

[0120] A grid coding module: after the geographic area to be divided is divided into a geographic spatial grid hierarchy system, the geographic grid coding is performed on each grid unit;

[0121] A graph generation module: after all information elements in the grid are stored based on the geographic grid coding, entity relationship extraction is performed to generate a triple knowledge graph;

[0122] A data mapping module: after the triple knowledge graph is generated, the geographic grid is subjected to multi-source heterogeneous data cleaning and disambiguation and event time mapping;

[0123] A judgment integration module: based on the geographic grid after the event time mapping is completed, the to-be-judgment information is subjected to association relationship judgment and credibility judgment, and the open source information collection and integration are completed based on the judgment result.

[0124] Embodiment three:

[0125] The embodiment of the application further provides an open source information collection and integration device based on geographic grid coding, which can implement the open source information collection and integration method based on geographic grid coding in embodiment one, and comprises a processor and a storage medium.

[0126] The storage medium is used for storing instructions;

[0127] The processor is used for operating according to the instructions to perform the steps of the following method:

[0128] Acquire a geographic area to be divided and to-be-judgment information;

[0129] After the geographic area to be divided is divided into a geographic spatial grid hierarchy system, the geographic grid coding is performed on each grid unit;

[0130] After all information elements in the grid are stored based on the geographic grid coding, entity relationship extraction is performed to generate a triple knowledge graph;

[0131] After the triple knowledge graph is generated, the geographic grid is subjected to multi-source heterogeneous data cleaning and disambiguation and event time mapping;

[0132] Based on the geographic grid after the event time mapping is completed, the to-be-judgment information is subjected to association relationship judgment and credibility judgment, and the open source information collection and integration are completed based on the judgment result.

[0133] Embodiment four:

[0134] The embodiment of the present application also provides a computer readable storage medium, which can realize the open source information collection and integration method based on geographic grid coding according to the method in the embodiment one, and has a computer program stored thereon, which realizes the steps of the method when executed by a processor.

[0135] Obtaining a geographic region to be divided and to-be-judged information;

[0136] After the geographic region to be divided is divided according to a geographic spatial grid hierarchy system, geographic grid coding is performed on each grid unit;

[0137] After all information elements in the grid are stored based on the geographic grid coding, entity relationship extraction is performed to generate a triple knowledge graph;

[0138] After the geographic grid whose triple knowledge graph is generated is processed, multi-source heterogeneous data cleaning and disambiguation and event time mapping are performed;

[0139] Based on the geographic grid after the event time mapping is completed, to-be-judged information is associated and judged in terms of credibility, and open source information collection and integration are completed based on the judgment result.

[0140] As known from common general knowledge, the present application can be realized by other embodiments which do not depart from the spirit or essential characteristics thereof. Therefore, the above-described embodiments are merely illustrative in all aspects and are not the only ones. All changes within the scope of the present application or within the equivalent scope of the present application are intended to be embraced by the present application.

[0141] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0142] The present application is described with reference to flowcharts and / or block diagrams of the method, device (system) and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be realized by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing devices to produce a machine, so that the instructions executed by the computer or other programmable data processing devices produce a device for realizing the functions specified in the flowcharts and / or block diagrams. Figure 1one or more processes and / or blocks Figure 1 an apparatus for performing the functions specified in the flowchart or multiple flows and / or blocks.

[0143] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the flowchart or multiple flows and / or blocks. Figure 1 one or more processes and / or blocks Figure 1 an apparatus for performing the functions specified in the flowchart or multiple flows and / or blocks.

[0144] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flowchart or multiple flows and / or blocks. Figure 1 one or more processes and / or blocks Figure 1 an apparatus for performing the functions specified in the flowchart or multiple flows and / or blocks.

[0145] Finally, it should be noted that the above-mentioned embodiments are merely used to illustrate the technical solutions of the present application, rather than limiting them. Although the present application has been described in detail with reference to the above-mentioned embodiments, those skilled in the art should understand that the specific embodiments of the present application can be modified or replaced by equivalents without departing from the spirit and scope of the present application, and any modification or replacement without departing from the spirit and scope of the present application should be covered in the protection scope of the claims of the present application.

Claims

1. An open source information gathering integration method based on geographic grid encoding, characterized in that, Comprise: Acquire the geographical area to be divided and the information to be judged; After the geographical space grid hierarchy division of the geographical area to be divided, the geographical grid coding of each grid unit is carried out; After storing all information elements in the grid based on the geographical grid coding, entity relationship extraction is carried out to generate a triple knowledge graph, and the information elements include event reports; After the geographical grid generating a triple knowledge graph, multi-source heterogeneous data cleaning and disambiguation and event time mapping are carried out, and the event time mapping includes using a time sequence convolution network, inputting a feature vector containing the publishing time and the time description in the event related text, mapping the predicted event time interval to the corresponding grid, and the mapping rule is: if there is an effective time coverage of the grid in the event occurrence time interval, the event is mapped to the grid; Based on the geographical grid after the event time mapping is completed, the association relationship judgment and the credibility judgment of the information to be judged are carried out, and the open source information collection and integration are completed based on the judgment result.

2. The open source information harvesting integration method based on geographic grid coding according to claim 1, characterized in that, After the geographical space grid hierarchy division of the geographical area to be divided, the geographical grid coding of each grid unit is carried out, including: the earth's surface is divided from high to low into multiple levels, and the grid longitude span and latitude span of the kth level are: Δλ k = 360 / (2^k x m) Δφ k = 180 / (2^k x n) where Δλ k is the longitude span, Δφ k is the latitude span, k is the level number of the grid, and m and n are constants related to the level. For each grid unit, the level information, longitude direction index and latitude direction index are coded, and the expression is: ID=kx10^(2L)+ix10^L+j i = floor((l - l0) / Dl k ) j = floor((φ - φ0) / Δφ k ) In the formula: ID is the unique code of the grid, L is the maximum number of indexes i and j, i is the index of the longitude direction, j is the index of the latitude direction, λ is the central longitude of the grid unit, φ is the central latitude of the grid unit, λ0 is the starting longitude, φ0 is the starting latitude, and floor is the floor function.

3. The open source information harvesting integration method based on geographic grid coding according to claim 2, characterized in that, After storing all information elements in the grid based on the geographical grid coding, entity relationship extraction is carried out to generate a triple knowledge graph, including: Taking the grid ID as the key, all information elements in the grid are stored, and the correspondence between the grid and the information elements is established, and the information elements include domain name, IP, social account and event report; The entity recognition model and relationship extraction model based on BERT are used for entity relationship extraction of text / picture, and for the extracted results, a triple knowledge graph containing grid ID, theme word and confidence is generated, and the calculation formula of the confidence is: C=α×P+β×R Wherein, C is the confidence, P is the accuracy of entity relationship extraction, R is the ratio of the frequency of the information element appearing in the grid to the total frequency, and α and β are weight coefficients and α+β=1.

4. The open source information harvesting integration method based on geographic grid coding according to claim 3, characterized in that, After the grid generating a triple knowledge graph, multi-source heterogeneous data cleaning and disambiguation and event time mapping are carried out, including: The address information to be processed is subjected to geographic coding by calling the geographic coding APIs of Gaode, Baidu and Google and a self-built postal code / POI knowledge graph, a plurality of candidate coordinates are obtained, the similarity between each candidate coordinate and the coordinate of the corresponding address in the self-built knowledge graph is calculated, the coordinate with the highest similarity is selected as the final coordinate of the address, a time sequence convolution network is used to predict the real occurrence time interval of an event, the input of the time sequence convolution network is a feature vector containing the publishing time and the time description in the event-related text, and the output is the time interval of the event occurrence, and the predicted event time interval is mapped to a corresponding grid, and the mapping rule is that if there is an effective time coverage of the grid in the event occurrence time interval, the event is mapped to the grid; The similarity calculation formula is: S = 1 - d / D max where S is the similarity, d is the distance between two coordinates, and D max is the set maximum distance threshold.

5. The open source information harvesting integration method based on geographic grid coding according to claim 4, characterized in that, Based on the geographic grid after the completion of the event time mapping, the correlation relationship and the credibility of the to-be-judged information are determined, including: Based on the geographic grid after the completion of the event time mapping, the time similarity and the geographic similarity of the to-be-judged information are calculated; Based on the time similarity and the geographic similarity of the to-be-judged information, the spatio-temporal similarity of the to-be-judged information is calculated; Based on the spatio-temporal similarity of the to-be-judged information, the information correlation degree of the to-be-judged information is calculated; The information correlation degree of the to-be-judged information is compared with a set threshold, and the information is determined to have a correlation relationship in response to the information correlation degree being greater than the set threshold, otherwise it is determined to have no correlation; The credibility total score of the to-be-judged information is calculated through a three-level verification mechanism to determine the credibility of the to-be-judged information.

6. The open source information harvesting integration method based on geographic grid coding according to claim 5, characterized in that, The calculation formula of the information correlation degree is: R=δ×Sst+(1-δ)×Ss Ss=V A V B / (||V A ||×||V B ||) Sst=γ×St+(1-γ)×Sg St = 1 - |t A -t B | / T max Sg = 1 - d AB / G max wherein: R is the information correlation degree, δ is the weight coefficient of the spatiotemporal similarity, Sst is the spatiotemporal similarity, Ss is the semantic similarity, V A is the semantic feature vector of information A, V B is the semantic feature vector of information B, γ is the weight coefficient of the temporal similarity, St is the temporal similarity, t A is the timestamp of information A, t B is the timestamp of information B, T max is the set maximum time difference threshold, Sg is the geographic similarity, d AB is the distance between the geographic coordinates of information A and information B, G max is the set maximum geographic distance threshold.

7. The open source information harvesting integration method based on geographic grid encoding according to claim 6, characterized in that, The three-level verification mechanism includes a first-level verification, a second-level verification and a third-level verification, wherein: The first-level verification includes checking whether the information appears on at least two different types of open source platforms, and if so, it is recorded as passing the first-level verification and a first-level verification score is given; The secondary corroboration includes analyzing whether the key elements in the information are consistent on the basis of the primary corroboration, and calculating a consistency score C of the key elements key When C key is equal to or greater than a preset threshold value b, it is recorded as passing the secondary corroboration, a secondary corroboration score is given, and the primary corroboration score is accumulated. The third-level verification includes, on the basis of passing the second-level verification, combining historical data and domain knowledge to determine whether the information conforms to logic and common sense, calculating the degree of conformity F through logical reasoning and knowledge matching, and recording as passing the third-level verification when F≥a preset threshold c, giving a third-level verification score and adding the first-level and second-level verification scores as the total score; When the total score is greater than a preset total score threshold f, it is marked as "high credibility"; when the preset total score threshold g is less than the total score and the total score is less than the preset total score threshold f, it is marked as "medium credibility"; and when the total score is less than the preset total score threshold g, it is marked as "low credibility", wherein the preset total score threshold g is less than the preset total score threshold f.

8. An open source information gathering integration system based on geographic grid encoding, characterized in that, It includes: An information acquisition module: acquiring a to-be-divided geographic area and to-be-judged information; A grid coding module: dividing the to-be-divided geographic area into a geographic spatial grid hierarchy system, and then coding each grid unit; A graph generation module: storing all information elements in the grid based on the geographic grid coding, and then performing entity relationship extraction to generate a triple knowledge graph, wherein the information elements include event reports. The data mapping module is configured to perform the following operations: performing multi-source heterogeneous data cleaning and disambiguation on the geographical grid after generating the triple knowledge graph, and performing event time mapping on the geographical grid, wherein the event time mapping comprises: inputting a feature vector containing a publishing time and a time description in event-related text into a time series convolution network, and mapping a predicted event time interval to a corresponding grid according to a mapping rule, wherein the mapping rule is: if there is an effective time coverage of the grid in the event time interval, the event is mapped to the grid. The judgment integration module is configured to perform the following operations: performing association relationship judgment and credibility judgment on the to-be-judged information based on the geographical grid after the event time mapping is completed, and performing open source information collection and integration based on the judgment result.

9. An open source information gathering integration device based on geographic grid encoding, characterized by, The device comprises a processor and a storage medium. The storage medium is configured to store instructions. The processor is configured to operate according to the instructions to perform the steps of the method according to any one of claims 1-7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the steps of the method according to any one of claims 1-7.

Images