Government affair digital resource duplicate checking method and device, medium and program product

Abstract

The embodiment of the invention provides a duplicate checking method and device for government affair digital resources, a medium and a program product. The method comprises the steps that digital resources to be subjected to duplicate checking are determined, the feature vector space of the digital resources to be subjected to duplicate checking is extracted, and the feature vector space comprises metadata features and semantic features; based on a feature database and the metadata features and semantic features of the digital resources to be subjected to duplicate checking, duplicate checking processing is conducted on the digital resources to be subjected to duplicate checking, a duplicate checking result is obtained, and the feature database is constructed based on multiple government affair digital resources, multiple text data and regional features of an area where the digital resources to be subjected to duplicate checking are located. According to the method, a reference basis is provided for digital resource construction, and the quality of digital resource construction is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a method, device, medium and program product for checking duplicate government digital resources. Background Art

[0002] As the construction of digital departments gradually advances, the scale of digital resources is becoming increasingly large. In order to better preserve and manage digital resources, it is necessary to check for duplicate content in digital resources. The accuracy of duplicate content checking directly affects the quality and effectiveness of digital resource construction.

[0003] At present, common duplicate checking methods usually integrate metadata features and digital content semantics, combine with databases to perform similarity analysis, and comprehensively determine similarity.

[0004] However, for newly released strategies that have not yet been implemented, existing technical solutions face cold start problems when checking for duplicates due to the lack of historical data support; in addition, different regions may have different implementation practices for the same strategy, so regional characteristics have an impact on digital resources. Summary of the invention

[0005] The embodiments of the present application provide a method, device, medium and program product for checking duplicate government digital resources. The method provides a reference for the construction of digital resources and improves the quality of digital resource construction.

[0006] In a first aspect, an embodiment of the present application provides a method for checking duplicate government digital resources, comprising: determining a digital resource to be checked for duplicates, and extracting a feature vector space of the digital resource to be checked for duplicates, wherein the feature vector space includes metadata features and semantic features;

[0007] Based on a feature database, metadata features and semantic features of the digital resources to be checked for duplicates, the digital resources to be checked for duplicates are checked for duplicates to obtain duplicate checking results. The feature database is constructed based on multiple government digital resources, multiple text data and the regional characteristics of the area where the digital resources to be checked for duplicates are located.

[0008] In a possible implementation, before performing a duplicate check on the digital resource to be checked for duplicates based on the feature database, the metadata features and the semantic features of the digital resource to be checked for duplicates and obtaining the duplicate check result, the method further includes:

[0009] Determine a feature set according to multiple government digital resources and a feature vector space corresponding to each government digital resource;

[0010] Performing mining processing on the plurality of text data, and determining an expansion set based on the plurality of text data after mining processing and the regional characteristics of the region where the digital resources to be checked for duplicates are located;

[0011] Based on the expansion set and feature set, the feature database is constructed.

[0012] In a possible implementation, the above-mentioned mining process is performed on the plurality of text data, and the expansion set is determined based on the plurality of text data after the mining process and the regional characteristics of the region where the digital resources to be checked for duplicates are located, including:

[0013] Performing mining processing on the plurality of text data to obtain semantic association relationships between the plurality of text data;

[0014] Determine an initial expansion set based on the semantic association relationship between the multiple text data and the regional characteristics of the area where the digital resources to be checked for duplicates are located;

[0015] Based on the multiple text data, corresponding interpretation information is determined, and based on the interpretation information, the initial expansion set is expanded to obtain an expansion set.

[0016] In a possible implementation, the metadata features include business metadata and technical metadata, and the digital resources to be checked for duplicates include text content. The above-mentioned determination of the digital resources to be checked for duplicates and extraction of the feature vector space of the digital resources to be checked for duplicates include:

[0017] Determine the digital resources to be checked for duplicates, extract the technical metadata and business metadata of the digital resources to be checked for duplicates, perform supplementary processing on the business metadata based on standard parameters, and determine metadata features;

[0018] According to the text content, the digital resource to be checked for duplicates is segmented to obtain multiple segmented words corresponding to the digital resource to be checked for duplicates;

[0019] Based on the multiple participles, determine the importance of each participle, and based on the importance of each participle, determine the semantic feature corresponding to the digital resource to be checked for duplicates;

[0020] A feature vector space of the digital resource to be checked for duplicates is obtained according to the metadata features and the semantic features.

[0021] In a possible implementation, the feature database further includes a vocabulary library, and the feature database, metadata features and semantic features of the digital resource to be checked for duplicates are used to perform a duplicate checking process on the digital resource to be checked for duplicates, to obtain a duplicate checking result, including:

[0022] Based on the vocabulary library, multiple word segments corresponding to the digital resource to be checked for duplicates are matched to obtain a tag set of the digital resource to be checked for duplicates, wherein the tag set is used to indicate guidance information of the digital resource to be checked for duplicates;

[0023] According to the tag set, determining a duplicate checking range from the feature database;

[0024] Based on the duplication checking scope, metadata features, and semantic features, the digital resource to be checked for duplication is checked for duplication to obtain a duplication checking result.

[0025] In a possible implementation, the duplicate checking result includes a plurality of reference digital resources, and the method further includes:

[0026] Based on the multiple reference digital resources and the digital resource to be checked for duplicates, determining the similarity between each reference digital resource and the digital resource to be checked for duplicates;

[0027] Based on the multiple similarities, determining a reference judgment result, the reference judgment result is used to indicate whether the to-be-checked duplicate digital resource is duplicated;

[0028] If among the multiple similarities, a preset number of similarities reach the similarity threshold, it indicates that the digital resource to be checked for duplicates is duplicated;

[0029] If, among the multiple similarities, no preset number of similarities reaches the similarity threshold, it indicates that the digital resource to be checked for duplicates is not duplicated.

[0030] In a possible implementation, the method further includes:

[0031] In the case that the digital resource to be checked for duplicates is not duplicated, adding the digital resource to be checked for duplicates and the metadata features and semantic features of the digital resource to be checked for duplicates to the feature set;

[0032] Based on the added feature set, the feature database is updated.

[0033] In a second aspect, an embodiment of the present application provides a device for checking duplicate government digital resources, including:

[0034] An extraction module, used for determining the digital resource to be checked for duplicates, and extracting the feature vector space of the digital resource to be checked for duplicates, wherein the feature vector space includes metadata features and semantic features;

[0035] A processing module is used to perform a duplication check on the digital resources to be checked for duplication based on a feature database, metadata features and semantic features of the digital resources to be checked for duplication, and obtain a duplication check result. The feature database is constructed based on multiple government digital resources, multiple text data and the regional characteristics of the area where the digital resources to be checked for duplication are located.

[0036] In a possible implementation, the processing module is also used to determine a feature set based on multiple government digital resources and the feature vector space corresponding to each government digital resource; perform mining processing on the multiple text data, and determine an extension set based on the multiple text data after mining processing and the regional characteristics of the area where the digital resources to be checked for duplicates are located; and construct the feature database based on the extension set and the feature set.

[0037] In a possible implementation, the processing module is also used to perform mining processing on the multiple text data to obtain semantic association relationships between the multiple text data; determine an initial expansion set based on the semantic association relationships between the multiple text data and the regional characteristics of the area where the digital resources to be checked for duplicates are located; determine corresponding interpretation information based on the multiple text data, and based on the interpretation information, perform expansion processing on the initial expansion set to obtain an expansion set.

[0038] In a possible implementation, the metadata features include business metadata and technical metadata, and the digital resources to be checked for duplicates include text content. The extraction module is also used to determine the digital resources to be checked for duplicates, extract the technical metadata and business metadata of the digital resources to be checked for duplicates, supplement the business metadata based on standard parameters, and determine the metadata features; perform word segmentation on the digital resources to be checked for duplicates according to the text content to obtain multiple word segments corresponding to the digital resources to be checked for duplicates; determine the importance of each word segment based on the multiple word segments, and determine the semantic features corresponding to the digital resources to be checked for duplicates based on the importance of each word segment; and obtain the feature vector space of the digital resources to be checked for duplicates according to the metadata features and the semantic features.

[0039] In a possible implementation, the feature database also includes a vocabulary library, and the processing module is further used to perform segmentation matching on multiple segmentations corresponding to the digital resource to be checked for duplicates based on the vocabulary library to obtain a tag set of the digital resource to be checked for duplicates, and the tag set is used to indicate guidance information of the digital resource to be checked for duplicates; according to the tag set, determine the duplication checking scope from the feature database; based on the duplication checking scope, metadata features, and semantic features, perform duplication checking processing on the digital resource to be checked for duplicates to obtain a duplication checking result.

[0040] In one possible implementation, the duplicate checking result includes multiple reference digital resources, and the processing module is also used to determine the similarity between each reference digital resource and the digital resource to be checked for duplicates based on the multiple reference digital resources and the digital resource to be checked for duplicates; based on the multiple similarities, determine a reference judgment result, and the reference judgment result is used to indicate whether the digital resource to be checked for duplicates is repeated; if among the multiple similarities, there are a preset number of similarities that reach a similarity threshold, it indicates that the digital resource to be checked for duplicates is repeated; if among the multiple similarities, there is no preset number of similarities that reach the similarity threshold, it indicates that the digital resource to be checked for duplicates is not repeated.

[0041] In a possible implementation, the processing module is also used to add the digital resource to be checked for duplicates and the metadata features and semantic features of the digital resource to be checked for duplicates to the feature set when the digital resource to be checked for duplicates is not duplicated; and update the feature database based on the added feature set.

[0042] In a third aspect, an embodiment of the present application provides a government digital resource duplicate checking device, including: a memory, a processor;

[0043] The memory stores computer-executable instructions;

[0044] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.

[0045] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect above and / or various possible implementations of the first aspect.

[0046] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementation methods of the first aspect.

[0047] The government digital resource duplication checking method, device, medium and program product provided in the embodiments of the present application enhance the regional characteristics of digital resources by constructing a feature database based on multiple government digital resources, multiple text data and the regional characteristics of the area where the digital resources to be checked for duplication are located; based on the text data of strategies that have not yet been put into practice, the existing digital resources are expanded and revised, thereby improving the timeliness of the feature database; based on the metadata characteristics, semantic characteristics and feature database of the digital resources to be checked for duplication, the duplication checking results obtained by processing the resources to be checked for duplication provide a reference basis for digital resource construction, thereby improving the quality of digital resource construction. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0049] Figure 1 Schematic diagram of the process of checking duplicate government digital resources provided in the embodiment of this application Figure 1 ;

[0050] Figure 2 Schematic diagram of the process of checking duplicate government digital resources provided in the embodiment of this application Figure 2 ;

[0051] Figure 3 Schematic diagram of the process of checking duplicate government digital resources provided in the embodiment of this application Figure 3 ;

[0052] Figure 4 A schematic diagram of the structure of the government digital resource duplicate checking device provided for this application;

[0053] Figure 5 A schematic diagram of the structure of the government digital resource duplication checking equipment provided for this application.

[0054] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0055] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0056] "Multiple" means two or more, and other quantifiers are similar. "And / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0057] It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0058] As the construction of digital departments gradually advances, a certain scale of digital resources has been formed in infrastructure, public digital, public support, security facilities, business applications, etc., and the scale of digital resources is huge. In order to better preserve and manage digital resources, it is necessary to check for duplicates. The accuracy of duplicate checking of digital resources directly affects the quality and effectiveness of digital resource construction.

[0059] At present, common duplicate checking methods usually integrate metadata features and digital content semantics, combine with databases to perform similarity analysis, and comprehensively determine similarity.

[0060] However, for the relevant text data of newly released policies that have not yet been implemented, the existing technical solutions face the problem of cold start due to the lack of historical digital support; in addition, different regions may have different implementation practices for the same policy, and the construction of government digital resources is basically driven by pure technology in terms of resource feature engineering, which makes the digital resources in different regions highly overlap in collection scope, and ultimately leads to the convergence of the results of digital feature mining of digital resources.

[0061] In response to the above problems, the present application provides a method for checking duplicate government digital resources. The method solves the problem of convergence of digital feature mining results of digital resources in different regions by enhancing the regional characteristics of digital resources. The method solves the cold start problem by mining and analyzing relevant text data of strategies that have not yet been put into practice, expanding and correcting existing digital resources.

[0062] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0063] Figure 1 Schematic diagram of the process of checking duplicate government digital resources provided in the embodiment of this application Figure 1 ,like Figure 1 As shown, the method includes:

[0064] S101, determining a digital resource to be checked for duplicates, and extracting a feature vector space of the digital resource to be checked for duplicates, wherein the feature vector space includes metadata features and semantic features;

[0065] Among them, digital resources are information resources that are published, accessed, and used in digital form, for example, they can be formed through the integration of computer technology, communication technology, and multimedia technology; the feature vector space is a set of multiple features, including metadata features and semantic features; semantic features are used to indicate the content or meaning of digital resources.

[0066] Determine the information resources that are published, stored, and used in digital form and need to be checked for duplicates, i.e., the resources to be checked for duplicates, and extract metadata features and semantic features of the digital resources to be checked for duplicates, wherein the metadata features include business metadata and technical metadata.

[0067] By extracting the metadata features and semantic features of the digital resources to be checked for duplicate content, the complex digital resources to be checked for duplicate content are converted into an easy-to-process vector form, which facilitates subsequent duplicate content checking, reduces the time for duplicate content checking, and improves the efficiency of duplicate content checking.

[0068] S102, based on the feature database, the metadata features and the semantic features of the digital resources to be checked for duplication, the digital resources to be checked for duplication are checked for duplication to obtain a duplication checking result.

[0069] Among them, the feature database is constructed based on multiple government digital resources, multiple text data and the regional characteristics of the area where the digital resources to be checked for duplicates are located; government digital resources are used to indicate digital resources supported by historical data, and to extract feature vectors of digital resources supported by historical data, including metadata features and semantic features; text data is used to indicate relevant information on strategies that have not yet been implemented; in the construction of digital resources, the factors used for development in different regions and the factors of each place are different. The same strategy may have different implementation practices in different regions. Regional characteristics are used to indicate the focus or bias of different regions; the duplicate checking results can be, for example, multiple government digital resources whose similarity with the digital resources to be checked for duplicates reaches a preset threshold, or a specified number of digital government resources with the highest similarity.

[0070] For example, the K-Nearest Neighbor (KNN) algorithm can be used to perform a duplicate check on the digital resources to be checked for duplicates based on the metadata features and semantic features of the digital resources to be checked for duplicates, from a feature database constructed based on multiple government digital resources, multiple text data, and the regional characteristics of the area where the digital resources to be checked for duplicates are located, to obtain a duplicate check result.

[0071] In a possible implementation, the duplicate checking result includes a plurality of reference digital resources, and the similarities between the plurality of reference digital resources in the duplicate checking result and the digital resource to be checked for duplicates can also be determined to judge whether the digital resource to be checked for duplicates is repeated. The specific process is as follows: based on the plurality of reference digital resources and the digital resource to be checked for duplicates, the similarities between each reference digital resource and the digital resource to be checked for duplicates are determined; based on the plurality of similarities, a reference judgment result is determined;

[0072] Among them, the similarity is used to indicate the degree of similarity between two government digital resources; the reference judgment result is used to indicate whether the digital resources to be checked for duplicates are duplicates.

[0073] Based on multiple reference digital resources in the duplicate checking results, the degree of similarity between each reference digital resource and the digital resource to be checked for duplicates is determined. For example, the similarity of metadata features and semantic features between each reference digital resource and the digital resource to be checked for duplicates can be judged, and based on the similarity corresponding to each reference digital resource, a reference judgment result indicating whether the digital resource to be checked for duplicates is a duplicate can be given.

[0074] If among the multiple similarities, there are a preset number of similarities that reach the similarity threshold, it indicates that the digital resource to be checked for duplicates is repeated; if among the multiple similarities, there is no preset number of similarities that reach the similarity threshold, it indicates that the digital resource to be checked for duplicates is not repeated.

[0075] According to the similarity between each reference digital resource and the digital resource to be checked for duplicates, the number of reference digital resources that reach the similarity threshold is determined. If there are a preset number of similarities that reach the similarity threshold, it indicates that the digital resource to be checked for duplicates is duplicated; if there are no preset number of similarities that reach the similarity threshold, it indicates that the digital resource to be checked for duplicates is not duplicated. For example: when the preset number is 1, when the similarity between the reference digital resource and the digital resource to be checked for duplicates reaches the preset threshold of 90%, it indicates that the digital resource to be checked for duplicates. Combining metadata features and semantic features for similarity judgment improves the accuracy of the reference judgment results.

[0076] In a possible implementation, when it is determined that the digital resource to be checked for duplicates is a non-duplicate digital resource, the digital resource to be checked for duplicates and the metadata features and semantic features of the digital resource to be checked for duplicates can also be added to the feature set; based on the added feature set, the feature database is updated.

[0077] When it is determined that the digital resource to be checked for duplicates is a non-duplicate digital resource, it means that it is necessary to construct government digital resources based on the digital resource to be checked for duplicates. Therefore, the digital resource to be checked for duplicates and its metadata features and semantic features are added to the feature set; and based on the added feature set, the feature database is updated, thereby improving the timeliness and accuracy of the construction of government digital resources.

[0078] The method for checking duplicate government resources provided in the embodiment of the present application enhances the regional characteristics of digital resources by constructing a feature database based on multiple government digital resources, multiple text data and the regional characteristics of the area where the digital resources to be checked for duplicate are located; based on the text data of the strategy that has not yet been put into practice, the existing digital resources are expanded and revised, thereby improving the timeliness of the feature database; based on the metadata characteristics, semantic characteristics and feature database of the digital resources to be checked for duplicate, the duplicate checking results obtained by processing the resources to be checked for duplicate are provided, which provide a reference basis for the construction of digital resources and improve the quality of the construction of digital resources.

[0079] Figure 2 Schematic diagram of the process of checking duplicate government digital resources provided in the embodiment of this application Figure 2 In this embodiment, the feature database also includes a vocabulary library, the metadata features include technical metadata and business metadata, and the digital resources to be checked for duplicate content include text content, such as Figure 2 As shown, in this embodiment Figure 1 Based on the embodiment, the specific process of the method for checking duplicate government digital resources is described in detail, and the method includes:

[0080] S201, determining a digital resource to be checked for duplicates, extracting technical metadata and business metadata of the digital resource to be checked for duplicates, supplementing the business metadata based on standard parameters, and determining metadata features;

[0081] Metadata features include technical metadata and business metadata. Technical metadata is used to indicate information about the characteristics and attributes of data at the technical level, such as data structure, including field type, data type, etc., or feature descriptions of data processing, etc. Business metadata is used to indicate information about the characteristics and attributes of data at the business level, such as relevant concepts, relationships, and rules in the business field, etc. Standard parameters are standard parameters in the relevant field of the digital resource to be checked for duplicates.

[0082] Identify the digital resources to be checked for duplicates, extract the characteristics and attribute information of the digital resources to be checked for duplicates at the technical and business levels, and standardize the parameters to supplement the business metadata, associate multiple government digital resources with strategies in a timely manner, and determine the metadata characteristics based on technical metadata and business metadata.

[0083] S202, performing word segmentation processing on the digital resource to be checked for duplicates according to the text content, and obtaining a plurality of word segments corresponding to the digital resource to be checked for duplicates;

[0084] The text content of the digital resource to be checked for duplicates is segmented by a word segmentation tool, and multiple word segments corresponding to the digital resource to be checked for duplicates are obtained.

[0085] S203, based on the multiple participles, determining the importance of each participle, and based on the importance of each participle, determining the semantic feature corresponding to the to-be-checked duplicate digital resource;

[0086] A Vector Space Model (VSM) model is constructed according to the feature weight calculation method, and the term frequency-inverse document frequency (TF-IDF) of each word segment is calculated to determine the importance of each word segment in the digital resource to be checked for duplicates, and at least one word segment with a high TF-IDF value is extracted as at least one feature label of the digital resource to be checked for duplicates, and the at least one feature label is used as the semantic feature of the digital resource to be checked for duplicates.

[0087] S204, obtaining a feature vector space of the digital resource to be checked for duplicate content according to the metadata feature and the semantic feature;

[0088] Comprehensive metadata features and semantic features as the feature vector space of the digital resources to be checked for duplicates

[0089] S205, based on the vocabulary library, performing word segmentation matching on a plurality of word segments corresponding to the digital resource to be checked for duplicates, to obtain a tag set of the digital resource to be checked for duplicates;

[0090] The vocabulary library includes multiple words and labels corresponding to each word;

[0091] Based on the vocabulary library, multiple word segments of the digital resource to be checked for duplicates are matched. If a single word segment of the digital resource to be checked for duplicates is the same as any one of the multiple words in the vocabulary library, the label corresponding to the word is used as the label of the digital resource to be checked for duplicates, and a label set of the digital resource to be checked for duplicates is obtained based on the multiple labels. If a single word segment of the digital resource to be checked for duplicates is different from the multiple words in the vocabulary library, similar words in the vocabulary library can be obtained based on the semantic features corresponding to the word segment, and the label of the similar word is used as the label of the digital resource to be checked for duplicates.

[0092] S206, determining a duplicate checking range from the feature database according to the tag set;

[0093] According to the tag set of the digital resource to be checked for duplicates, the scope of the check is determined from the feature database, the dimension of the feature term vector is reduced, and the calculation speed and accuracy are improved.

[0094] S207, based on the duplication checking range, metadata features, and semantic features, the digital resource to be checked for duplication is checked for duplication to obtain a duplication checking result.

[0095] From the corresponding duplication checking range of the feature database, based on the metadata features and semantic features of the digital resources to be checked for duplication, the digital resources to be checked for duplication are checked for duplication to obtain a duplication checking result. The specific process of checking for duplication is similar to step S102 and will not be repeated here.

[0096] In one possible implementation, multiple reference digital resources in the duplicate checking results can also be visualized, showing multiple reference digital resources and the meta-digital information, content information, and label sets of each reference digital resource. Staff can make manual judgments and corrections based on the meta-digital information, content information, and label sets of each reference digital resource to improve the accuracy of the duplicate checking.

[0097] The method for checking duplicate government digital resources provided in the embodiment of the present application introduces standard parameters to add business attributes to technical metadata, so that digital resources present dual metadata characteristics of technology + business; by extracting the semantic features of digital resources, the information of digital resources is described more completely and accurately; based on the metadata features, semantic features, and feature database of the digital resources to be checked for duplicates, the tag set of the digital resources to be checked for duplicates is determined, the dimension of the feature term vector is reduced, and the calculation speed and accuracy are improved; the duplicate checking results obtained by checking the resources to be checked for duplicates provide a reference basis for the construction of digital resources, thereby improving the quality of digital resource construction.

[0098] Figure 3 Schematic diagram of the process of checking duplicate government digital resources provided in the embodiment of this application Figure 3 ,like Figure 3 As shown, in this embodiment Figure 1 Example or Figure 2 Based on this, the specific process of constructing a feature database is described in detail. The method includes:

[0099] S301, determining a feature set according to multiple government digital resources and a feature vector space corresponding to each government digital resource;

[0100] Each feature vector space includes metadata features and semantic features. A feature set is determined based on multiple government digital resources and the feature vector space corresponding to each government digital resource.

[0101] S302, performing mining processing on a plurality of text data to obtain semantic association relationships between the plurality of text data;

[0102] Text data is used to indicate relevant information of strategies that have not yet been put into practice and lacks the support of historical data. Text data can be, for example, work reports, meetings, etc. Therefore, it is necessary to perform word segmentation and semantic association expansion on the titles and contents of multiple text data to obtain the semantic association relationship between multiple text data.

[0103] S303, determining an initial expansion set based on the semantic association relationship between the plurality of text data and the regional characteristics of the region where the to-be-checked duplicate digital resources are located;

[0104] Among them, the extension set is used to expand and modify existing digital resources.

[0105] Based on the semantic association between multiple text data and the regional characteristics of the area where the digital resources to be checked for duplicates are located, that is, combined with the actual construction situation of digital resources, the initial expansion set is determined.

[0106] S304, determining corresponding interpretation information based on the multiple text data, and performing expansion processing on the initial expansion set based on the interpretation information to obtain an expansion set;

[0107] Among them, the interpretation of information is an example of experts' analysis of text data.

[0108] Based on the interpretation information obtained by experts from the analysis of multiple text data, the initial expansion set is expanded to obtain an expansion set, which solves the cold start problem and analyzes multiple text data to provide strong support for strategies that have not yet been put into practice.

[0109] S305: construct the feature database based on the extension set and feature set.

[0110] Based on the expansion set and feature set, a feature database is constructed. This feature database not only combines the construction status of actual digital resources, but also solves the cold start problem.

[0111] The method for checking duplicate government digital resources provided in the embodiment of the present application enhances the regional characteristics of digital resources by constructing a feature database based on multiple government digital resources, multiple text data and the regional characteristics of the area where the digital resources to be checked for duplicate government resources are located; based on text data of strategies that have not yet been implemented, the existing digital resources are expanded and revised, thereby improving the timeliness of the feature database.

[0112] Figure 4 The structural diagram of the government digital resource duplicate checking device provided for this application is as follows: Figure 4 As shown, the government digital resource duplicate checking device 40 provided in this embodiment includes:

[0113] Extraction module 401, used to determine the digital resource to be checked for duplicates, and extract the feature vector space of the digital resource to be checked for duplicates, the feature vector space including metadata features and semantic features;

[0114] Processing module 402 is used to perform duplication checking on the digital resources to be checked for duplication based on the feature database, metadata features and semantic features of the digital resources to be checked for duplication, and obtain duplication checking results. The feature database is constructed based on multiple government digital resources, multiple text data and the regional characteristics of the area where the digital resources to be checked for duplication are located.

[0115] In one possible implementation, the processing module 402 is also used to determine a feature set based on multiple government digital resources and the feature vector space corresponding to each government digital resource; perform mining processing on multiple text data, and determine an extension set based on the multiple text data after mining processing and the regional characteristics of the area where the digital resources to be checked for duplicates are located; and construct a feature database based on the extension set and the feature set.

[0116] In one possible implementation, the processing module 402 is also used to mine multiple text data to obtain semantic associations between the multiple text data; determine an initial extension set based on the semantic associations between the multiple text data and the regional characteristics of the area where the digital resources to be checked for duplicates are located; determine corresponding interpretation information based on the multiple text data, and based on the interpretation information, expand the initial extension set to obtain an extension set.

[0117] In one possible implementation, metadata features include business metadata and technical metadata, and the digital resources to be checked for duplicates include text content. The extraction module 401 is also used to determine the digital resources to be checked for duplicates, extract the technical metadata and business metadata of the digital resources to be checked for duplicates, supplement the business metadata based on standard parameters, and determine the metadata features; perform word segmentation on the digital resources to be checked for duplicates according to the text content to obtain multiple word segments corresponding to the digital resources to be checked for duplicates; determine the importance of each word segment based on the multiple word segments, and determine the semantic features corresponding to the digital resources to be checked for duplicates based on the importance of each word segment; and obtain the feature vector space of the digital resources to be checked for duplicates according to the metadata features and the semantic features.

[0118] In a possible implementation, the feature database also includes a vocabulary library, and the processing module 402 is also used to perform word segmentation matching on multiple word segments corresponding to the digital resources to be checked for duplicates based on the vocabulary library to obtain a tag set of the digital resources to be checked for duplicates, and the tag set is used to indicate guidance information of the digital resources to be checked for duplicates; according to the tag set, determine the scope of the duplicate check from the feature database; based on the duplicate check scope, metadata features, and semantic features, perform duplicate check processing on the digital resources to be checked for duplicates to obtain duplicate check results.

[0119] In one possible implementation, the duplicate checking result includes multiple reference digital resources, and the processing module 402 is also used to determine the similarity between each reference digital resource and the digital resource to be checked for duplicates based on the multiple reference digital resources and the digital resource to be checked for duplicates; based on the multiple similarities, determine a reference judgment result, and the reference judgment result is used to indicate whether the digital resource to be checked for duplicates is repeated; if among the multiple similarities, there are a preset number of similarities that reach the similarity threshold, it indicates that the digital resource to be checked for duplicates is repeated; if among the multiple similarities, there is no preset number of similarities that reach the similarity threshold, it indicates that the digital resource to be checked for duplicates is not repeated.

[0120] In one possible implementation, the processing module 402 is also used to add the digital resources to be checked for duplicates and the metadata features and semantic features of the digital resources to be checked for duplicates to the feature set when the digital resources to be checked for duplicates are not duplicated; and update the feature database based on the added feature set.

[0121] The government digital resource duplication checking device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and this embodiment will not be repeated here.

[0122] Figure 5 The schematic diagram of the structure of the government digital resource duplicate checking device provided for this application. Figure 5 As shown, the electronic device 50 provided in this embodiment includes: at least one processor 501 and a memory 502. Optionally, the device 50 also includes a communication component 503. The processor 501, the memory 502 and the communication component 503 are connected via a bus 504.

[0123] In a specific implementation process, at least one processor 501 executes the computer-executable instructions stored in the memory 502, so that at least one processor 501 executes the above method.

[0124] The specific implementation process of the processor 501 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0125] In the above embodiments, it should be understood that the processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the invention can be directly implemented as a hardware processor, or can be implemented by a combination of hardware and software modules in the processor.

[0126] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (NVM), such as at least one disk storage.

[0127] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.

[0128] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0129] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0130] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0131] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0132] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0133] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0134] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0135] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0136] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0137] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A method for checking duplicate government digital resources, characterized in that: include: Determine the digital resource to be checked for duplicates, and extract the feature vector space of the digital resource to be checked for duplicates, wherein the feature vector space includes metadata features and semantic features; Based on a feature database, metadata features and semantic features of the digital resources to be checked for duplicates, the digital resources to be checked for duplicates are checked for duplicates to obtain duplicate checking results. The feature database is constructed based on multiple government digital resources, multiple text data and the regional characteristics of the area where the digital resources to be checked for duplicates are located.

2. The method according to claim 1, characterized in that The method further comprises: performing a duplicate check on the digital resource to be checked for duplicates based on the feature database, the metadata features and the semantic features of the digital resource to be checked for duplicates, and obtaining the duplicate check result. Determine a feature set according to multiple government digital resources and a feature vector space corresponding to each government digital resource; Performing mining processing on the plurality of text data, and determining an expansion set based on the plurality of text data after mining processing and the regional characteristics of the region where the digital resources to be checked for duplicates are located; Based on the expansion set and feature set, the feature database is constructed.

3. The method according to claim 2, characterized in that The mining process of the plurality of text data and determining the expansion set based on the plurality of text data after the mining process and the regional characteristics of the region where the to-be-checked duplicate digital resources are located include: Performing mining processing on the plurality of text data to obtain semantic association relationships between the plurality of text data; Determine an initial expansion set based on the semantic association relationship between the multiple text data and the regional characteristics of the area where the digital resources to be checked for duplicates are located; Based on the multiple text data, corresponding interpretation information is determined, and based on the interpretation information, the initial expansion set is expanded to obtain an expansion set.

4. The method according to claim 1, characterized in that The metadata features include business metadata and technical metadata, the digital resources to be checked for duplicates include text content, and the steps of determining the digital resources to be checked for duplicates and extracting the feature vector space of the digital resources to be checked for duplicates include: Determine the digital resources to be checked for duplicates, extract the technical metadata and business metadata of the digital resources to be checked for duplicates, perform supplementary processing on the business metadata based on standard parameters, and determine metadata features; According to the text content, the digital resource to be checked for duplicates is segmented to obtain multiple segmented words corresponding to the digital resource to be checked for duplicates; Based on the multiple participles, determine the importance of each participle, and based on the importance of each participle, determine the semantic feature corresponding to the digital resource to be checked for duplicates; A feature vector space of the digital resource to be checked for duplicates is obtained according to the metadata features and the semantic features.

5. The method according to claim 1, characterized in that The feature database also includes a vocabulary library. Based on the feature database, the metadata features and semantic features of the digital resource to be checked for duplicates, the digital resource to be checked for duplicates is checked for duplicates to obtain a duplicate checking result, including: Based on the vocabulary library, multiple word segments corresponding to the digital resource to be checked for duplicates are matched to obtain a tag set of the digital resource to be checked for duplicates, wherein the tag set is used to indicate guidance information of the digital resource to be checked for duplicates; According to the tag set, determining a duplicate checking range from the feature database; Based on the duplication checking scope, metadata features, and semantic features, the digital resource to be checked for duplication is checked for duplication to obtain a duplication checking result.

6. The method according to claim 1, characterized in that The duplicate checking result includes a plurality of reference digital resources, and the method further includes: Based on the multiple reference digital resources and the digital resource to be checked for duplicates, determining the similarity between each reference digital resource and the digital resource to be checked for duplicates; Based on the multiple similarities, determining a reference judgment result, the reference judgment result is used to indicate whether the to-be-checked duplicate digital resource is duplicated; If among the multiple similarities, a preset number of similarities reach the similarity threshold, it indicates that the digital resource to be checked for duplicates is duplicated; If, among the multiple similarities, no preset number of similarities reaches the similarity threshold, it indicates that the digital resource to be checked for duplicates is not duplicated.

7. The method according to claim 6, characterized in that The method further comprises: In the case that the digital resource to be checked for duplicates is not duplicated, adding the digital resource to be checked for duplicates and the metadata features and semantic features of the digital resource to be checked for duplicates to the feature set; Based on the added feature set, the feature database is updated.

8. A device for checking duplicate government digital resources, characterized in that: include: An extraction module, used for determining the digital resource to be checked for duplicates, and extracting the feature vector space of the digital resource to be checked for duplicates, wherein the feature vector space includes metadata features and semantic features; A processing module is used to perform a duplication check on the digital resources to be checked for duplication based on a feature database, metadata features and semantic features of the digital resources to be checked for duplication, and obtain a duplication check result. The feature database is constructed based on multiple government digital resources, multiple text data and the regional characteristics of the area where the digital resources to be checked for duplication are located.

9. A device for checking duplicate government digital resources, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

Images