Cross-library retrieval method, device and system and storage medium

Abstract

The invention discloses a cross-library retrieval method, device and system and a storage medium, and the method comprises the steps: when a new search request is received, analyzing the new search request into a new search field set; according to the field-category association strength of each search field in the new search field set in each database, calculating the association strength of the new search field set with each search category in each database; comparing the association strength of the new search field set with each retrieval category in each database with a preset association strength threshold, and screening to obtain a high-association retrieval category set; and retrieving the new search field set according to the high-association retrieval category set and the priority of each database to obtain a retrieval result set. The high-association retrieval category set is screened according to the fields in the search request, so that the accuracy of cross-database retrieval is improved, and in addition, retrieval is performed according to the high-association retrieval category set and the priority of each database, so that the retrieval efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data retrieval, and particularly relates to a cross-database retrieval method, device and system and a storage medium. BACKGROUND

[0002] With the continuous deepening of the process of enterprise digital transformation, business systems often need to integrate multiple heterogeneous databases to meet complex and changing business needs. However, different databases of multiple source databases have different ways of describing the same data, which seriously affects the accuracy of the search results. In addition, traditional search methods usually perform traversal search on all associated databases, which is slow in response speed and difficult to meet the strict requirements of business systems for real-time performance.

[0003] Therefore, how to provide a cross-database retrieval method and improve the accuracy and efficiency of cross-database retrieval has become a technical problem to be solved. SUMMARY

[0004] The present application provides a cross-database retrieval method, device, system and storage medium to improve the accuracy and efficiency of cross-database retrieval.

[0005] The present application provides a cross-database retrieval method, comprising: When a new search request is received, the new search request is parsed into a new search field set; According to the field-category association strength of each search field in the new search field set in each database, the association strength of the new search field set with each search category in each database is calculated; The association strength of the new search field set with each search category in each database is compared with a preset association strength threshold, and a high-association search category set is selected; According to the high-association search category set and the priority of each database, the new search field set is searched to obtain a search result set.

[0006] The present application has the beneficial effects that when a new search request is received, the new search request is parsed into a new search field set, the association strength of the new search field set with each search category in each database is calculated, the high-association search category set is selected according to the association strength, and the new search field set is searched according to the high-association search category set and the priority of each database to obtain a search result set. Since the high-association search category set is selected according to the field in the search request, the accuracy of cross-database retrieval is improved, and in addition, the search efficiency is improved according to the high-association search category set and the priority of each database.

[0007] In one embodiment, the field-category association strength is determined as follows: constructing a plurality of preset information sets and corresponding index structures according to database information and historical search information of each database, wherein the preset information sets at least include a database identification set and a category label set of search results; generating a semantic vector set according to the category label set of search results; extracting semantic vectors corresponding to category labels of each database from the semantic vector set based on the index structure; calculating cross-database category correlations between category labels of different databases according to the semantic vectors; obtaining field-category association strengths in each database according to conditional probabilities of different search fields returning each category label and the cross-database category correlations.

[0008] In one embodiment, the calculating cross-database category correlations between category labels of different databases according to the semantic vectors comprises: calculating cosine similarities of semantic vectors corresponding to category labels of different databases; determining the cosine similarities as cross-database category correlations between category labels of different databases.

[0009] In one embodiment, the obtaining field-category association strengths in each database according to conditional probabilities of different search fields returning each category label and the cross-database category correlations comprises: calculating conditional probabilities of different search fields returning each category label in each database according to the following formula; ; wherein c l is a category label, f hi is a search field, d j is a database identification, N(f hi ∩c l , d j ) is a number of records containing search field f hi and returning category label c l in database d j , and N(f hi , d j ) is a number of records containing search field f hi in database d j ; substituting conditional probabilities of different search fields returning each category label in each database and cross-database category correlations into the following formula to determine field-category association strengths in each database: ; wherein strength(f hi, c l , d j ) is the field-category association strength of the search field f j in the database d hi with the category label c l , γ is the cross-database association weight coefficient and 0 < γ < 1, represents the association of the category label c j in the database d l with the category in other databases.

[0010] In one embodiment, the calculating the association strength of the new search field set with each search category in each database according to the field-category association strength of each search field in the new search field set in each database comprises: substituting the field-category association strength of each search field in the new search field set in each database into the following formula to obtain the association strength of the new search field set with each search category in each database: ; wherein F new is the new search field set, c l is the category label, d j is the database identifier, is the search field in the new search field set, is the association strength of the search field f j in the database d l with the category label c l , and n is the number of search fields in the new search field set.

[0011] In one embodiment, the priority of each database is determined by: calculating the search success rate and field coverage of different databases according to historical search information; weighting and summing the search success rate and field coverage and performing normalization processing to obtain the database priority weight.

[0012] In one embodiment, the method further comprises: aggregating all search results and removing duplicate search results to form a total result set; calculating the sorting score of each search result in the total result set according to the database priority weight and the association strength of the search result; descendingly sorting according to the sorting score and outputting a preset number of search results.

[0013] The present application also provides a cross-database search device, comprising: a parsing module configured to parse a new search request into a new search field set when the new search request is received; a first calculating module configured to calculate an association strength of the new search field set with each search category in each database according to a field-category association strength of each search field in the new search field set in each database; a screening module configured to compare the association strength of the new search field set with each search category in each database with a preset association strength threshold, and screen a high-association search category set; a searching module configured to search the new search field set according to the high-association search category set and a priority of each database to obtain a search result set.

[0014] In one embodiment, the field-category association strength is determined as follows: constructing a plurality of preset information sets and corresponding index structures according to database information and historical search information of each database, wherein the preset information sets at least include a database identifier set and a category label set of search results; generating a semantic vector set according to the category label set of search results; extracting a semantic vector corresponding to a category label of each database from the semantic vector set based on the index structure; calculating a cross-database category association between category labels of different databases according to the semantic vector; obtaining a field-category association strength in each database according to a conditional probability of each category label returned by different search fields in each database and the cross-database category association.

[0015] In one embodiment, the calculation of the cross-database category association between category labels of different databases according to the semantic vector includes: calculating a cosine similarity of semantic vectors corresponding to category labels of different databases; determining the cosine similarity as the cross-database category association between category labels of different databases.

[0016] In one embodiment, the obtaining of the field-category association strength in each database according to the conditional probability of each category label returned by different search fields in each database and the cross-database category association includes: calculating the conditional probability of each category label returned by different search fields in each database according to the following formula; ; wherein c l is a category label, f hi is a search field, d j is a database identifier, and N(fhi ∩c l ,d j ) is the number of records in database d j containing search field f hi and returning category label c l , N(f hi , d j ) is the number of records in database d j containing search field f hi ; The conditional probability of different search fields in each database returning each category label and the cross-database category association are substituted into the following formula to determine the field-category association strength in each database: ; wherein strength(f hi , c l , d j ) is the field-category association strength of search field f j and category label c hi in database d l , γ is a cross-database association weight coefficient and 0<γ<1, represents the association of category label c j in database d l with categories in other databases.

[0017] In one embodiment, the computing module is further configured to: Substitute the field-category association strength of each search field in each database according to the new search field set into the following formula to obtain the association strength of the new search field set with each search category in each database: ; wherein F new is the new search field set, c l is a category label, d j is a database identifier, is a search field in the new search field set, is the association strength of search field j and category label c l in database d new , and n is the number of search fields in the new search field set.

[0018] In one embodiment, the priority of each database is determined in the following manner: Calculate the search success rate and field coverage of different databases according to historical search information; ​The search success rate and the field coverage are weighted and summed and normalized to obtain a database priority weight.

[0019] In one embodiment, the apparatus further comprises: a deduplication module configured to aggregate all the search results and remove duplicate search results to form a total result set; a second calculation module configured to calculate a ranking score of each search result in the total result set according to the database priority weight and the association strength of the search result; a ranking module configured to sort the search results in descending order according to the ranking scores and output a preset number of search results.

[0020] The application also provides a cross-database search system, comprising: at least one processor; and, a memory in communication connection with the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to implement the cross-database search method described in any of the above embodiments.

[0021] The application also provides a computer-readable storage medium, when the instructions in the storage medium are executed by the processor corresponding to the cross-database search system, the cross-database search system can implement the cross-database search method described in any of the above embodiments.

[0022] Other features and advantages of the application will be set forth in the specification, and in part will become apparent to those skilled in the art upon reference to the specification, or can be learned by practice of the application. The objectives and other advantages of the application can be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

[0023] The technical solutions of the application will be further described in detail below with the aid of the accompanying drawings and examples. BRIEF DESCRIPTION OF DRAWINGS

[0024] The accompanying drawings are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and serve to explain the application, and do not limit the application. In the drawings: Figure 1 A flowchart of a cross-database search method in an embodiment of the application; Figure 2 A structural schematic diagram of a cross-database search apparatus in an embodiment of the application; Figure 3 A hardware structural schematic diagram of a cross-database search system in an embodiment of the application. DETAILED DESCRIPTION

[0025] The preferred embodiments of the present application will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are merely intended to explain and illustrate the present application, and should not be used to limit the present application.

[0026] Figure 1 The flow chart of a cross-database search method in an embodiment of the present application is shown in FIG. 1, which can be implemented as the following steps S101-S104: Figure 1 In step S101, when a new search request is received, the new search request is parsed into a new search field set; In step S102, the association strength of the new search field set with each search category in each database is calculated according to the field-category association strength of each search field in the new search field set in each database; In step S103, the association strength of the new search field set with each search category in each database is compared with a preset association strength threshold, and a high-association search category set is screened out; In step S104, the new search field set is searched according to the high-association search category set and the priority of each database to obtain a search result set.

[0027] In the present application, when a new search request is received, the new search request is parsed into a new search field set. For example, the new search request is parsed, and all the search fields contained therein are grouped into a new search field set , wherein represents the i-th search field in the new search record, and n is the number of fields in the current search.

[0028] The association strength of the new search field set with each search category in each database is calculated according to the field-category association strength of each search field in the new search field set in each database. Specifically, the field-category association strength of each search field in the new search field set in each database is substituted into the following formula to obtain the association strength of the new search field set with each search category in each database: ; wherein F new is the new search field set, c l is the category label, d j is the database identifier, is the search field in the new search field set, is the association strength of the search field j in the database d with the category label c l , and n is the number of search fields in the new search field set. ​

[0029] The association strength of the new search field set in each database with each search category is compared with a preset association strength threshold, and a high-association search category set is screened. Assuming that the preset association strength threshold is θ str (0 < θ str <1), the categories satisfying the association strength greater than or equal to θ str are screened to form a high-association set .

[0030] The new search field set is searched according to the high-association search category set and the priority of each database to obtain a search result set. The databases are sorted by weight , and the first δ positions are selected as target databases D target , where δ is a preset maximum number of search databases and 1 ≤ δ ≤ m, and the data in each database is preferentially searched.

[0031] In addition, in this application, all search results can be aggregated and duplicate search results can be removed to form a total result set, for example, all search results are aggregated, duplicate records are removed based on data unique identifiers, and a total result set is formed , where r e is the e-th result in the total result set, and v is the total number of results after deduplication; then, the sorting score of each search result in the total result set is calculated according to the database priority weight and the association strength of the search result, and the sorting score of each result is calculated, and the formula is: , where μ is a weight factor and 0 < μ < 1, is the priority weight of the database d e from which the result r e comes, and strength e is the association strength of the field h e in the feedback record corresponding to the search result r e ; finally, the sorting scores are sorted in descending order, and a preset number of search results are output, sorted in descending order of score(r e ), and the first N results are output, where N is the number of returned results preset by the user.

[0032] It should be noted that in order to realize cross-database search, the application pre-constructs a plurality of preset information sets and index structures according to the database information and historical search records of a plurality of databases, and determines the field-category association strength of each search field in each database and the priority weight of each database.

[0033] (1) Constructing a plurality of preset information sets and index structures ​Multiple database information and user historical search information are acquired. The database information includes database identifiers and search result category tags, while the user historical search information includes search fields and search feedback records. Multiple preset information sets are constructed based on the database information and user historical search information. These preset information sets include a database identifier set, a search result category tag set, a full search field set, a single search field set, and a user feedback record set. The construction process of the user feedback record set is as follows: Feedback records are acquired from historical search information; search result fields are extracted from the feedback records; the field frequency of the search result fields in each database is obtained; the maximum field frequency of all search result fields in each database and the database identifier corresponding to the maximum value are determined; the maximum value is compared with a preset word frequency threshold to determine the feedback marker (i.e., positive or negative feedback) of the feedback record; the feedback marker and database identifier constitute the element corresponding to the feedback record in the user feedback record set. Then, the field frequencies of each search field and the multiple preset information sets are stored in a feature library and indexed to support efficient retrieval.

[0034] Specifically, by aggregating search fields, database identifiers, category tags, and user historical search information, a set of search fields, a set of single search fields, a set of database identifiers, a set of result category tags, and a set of user feedback records are constructed. Field frequency features are calculated and stored in a feature library, and an index structure is established. ① Collect historical search information, including user search behavior data and user feedback information, and aggregate the search fields from each search record to obtain the full set of search fields F. total ,Right now , where f h Let h represent the h-th search record, and t be the total number of search fields, where 1 ≤ h ≤ t. ② Combine the search fields included in the h-th search record to form the single search field set F for the h-th search record. h , , where f hi Let F represent the i-th search field in the h-th search record, where n is the number of search fields in that record, 1 ≤ i ≤ n; obviously F h F tota ③ Collect the database identifiers included in each search record to obtain the database identifier set D. , where d j Let represent the unique identifier of the j-th database in the set, and m be the total number of databases associated with the system, where 1 ≤ j ≤ m. ④ Collect the category tags corresponding to all search records to obtain the search result category tag set C. , where c lrepresents the l-th normalized category label in the set, k is the total number of system categories, l is the category index and 1≤1≤k. ⑤ Collecting user feedback information corresponding to each search record, and collecting all feedback information to obtain a user feedback record set R feedback , , wherein R g represents the g-th feedback record in the set, p is the total number of feedback records, g is the feedback record index and 1≤g≤p; it should be noted that R g contains a database identifier d g and a feedback result r g , wherein the r g is generated based on a word frequency comparison. Assuming that the field set of the search result in the g-th feedback record is , the word frequency of the field in all databases d j is calculated respectively, the maximum value of the word frequency of the field in all databases is taken, if , , then r g =1 is positive feedback, indicating that at least one field reaches high frequency in a certain database; otherwise r g =0 is negative feedback, indicating that all fields are low frequency in all databases. Wherein is a preset word frequency threshold, 0<θ u <1. It can be understood that the preset word frequency threshold can be obtained by statistically calculating the word frequency of all fields in the database, taking the average, median and quartile of the word frequency of all fields, etc. For this purpose, the present application is not limited. Similarly, the field frequency of each search field can be calculated, and the field frequency of the search field in the database d j is calculated, and the formula is ; wherein is any search field and ; represents the number of records in the database d j containing , represents the total number of records in the database d j .

[0035] Further, the field frequency of each search field and a plurality of preset information sets can be stored in a feature library and indexed to support efficient retrieval. For example, a multi-level index structure is established, including:‌ field index: taking the search field f∈F as the core key, associating its field frequency freq(f,d) in each database d∈D and the corresponding single search field set F h index;‌ database index: taking the database identifier d jD is a key, associated with all the field list contained in the database, corresponding category label c k The mapping relationship of C and the user feedback record r e The index of R; Category index: with category label c k D is a key, associated with all the field list contained in the database, corresponding category label c j , field-category association strength pre-computed value and historical search hit record statistics.

[0036] (2) Determine the field-category association strength of each search field in each database According to the category label set of the search result, a semantic vector set is generated. For example, a semantic vector set is obtained , wherein v l represents the semantic vector corresponding to the category label c l . Further, the semantic vector corresponding to the category label of each database can be extracted from the semantic vector set based on the index structure. According to the semantic vector, the cross-database category association between the category labels of different databases is calculated. Specifically, the cosine similarity of the semantic vectors corresponding to the category labels of different databases is calculated, and the cosine similarity is determined as the cross-database category association between the category labels of different databases. The calculation formula is as follows: , wherein, is the cross-database category association between the category label c α in the database d α and the category label c β in the database c β , v α and v β represent the semantic vectors of c α and c β respectively.

[0037] Finally, the field-category association strength in each database is obtained according to the conditional probability of each category label returned by each search field in each database and the cross-database category association. The conditional probability of each category label returned by each search field in each database is calculated according to the following formula: ; wherein, c l is a category label, f hi is a search field, d j is a database identifier, N(f hi ∩c l , d j ) is a search field f hi contained in the database d j and returning the category label c lThe number of records, N(f) hi d j ) for database d j Includes search field f hi The number of records; Substituting the conditional probabilities of different search fields returning each category label in each database and the cross-database category association into the following formula, we can determine the field-category association strength in each database: ; Among them, strength(f hi c l d j ) for database d j Search field f hi With category label c l The field-category association strength, where γ is the cross-database association weight coefficient and 0 < γ < 1. Represents database d j Category label c l Categories in other databases The correlation.

[0038] (3) Determine the priority weight of each database. First, calculate the search success rate and field coverage for different databases.

[0039] ① Calculate the search success rate Extract feedback records from each database from the user feedback record set, and calculate the search success rate of each database using the following formula: , in, For database d j Search success rate, r g For database d j Each feedback record corresponds to a feedback message in the database, where p is the database d. j The total number of records in the user feedback record set.

[0040] ② Calculate field coverage Extract feedback records from each database from the user feedback record set, and obtain the full fields of each feedback record in each database; calculate the average value of the full fields based on the field frequency and correlation of the full fields in each database, and then calculate the weighted value of the full fields. Specifically: , in, For database d j Full field f h The weighted value, For database d jThe middle full-amount field f h The field frequency, The full-amount field f h In the database d j The association strength with all categories c l , and k is the total number of categories.

[0041] The field coverage of each database is calculated according to the weighted values of all full-amount fields of each database, and the formula is ; Among them, is the field coverage of the database, is the weighted value of the full-amount field f h in the database d j , and t is the total number of full-amount fields F total .

[0042] Then, the search success rate and the field coverage are weighted and summed and normalized to obtain the database priority weight.

[0043] The database priority weight is obtained by linearly weighting the search success rate and the field coverage, and the formula is: , wherein w0 is the initial weight of the database, λ is the weight coefficient and 0≤λ≤1, and the proportion of the success rate and the coverage in the initial weight is adjusted.

[0044] The weights of all databases are normalized to obtain the weights of each database: , Among them, represents the sum of the weights of all m databases, and m is the total number of associated databases.

[0045] The final database priority weight set is obtained, and is stored in the weight configuration library.

[0046] In one embodiment, a certain enterprise business system needs to integrate three heterogeneous databases: ERP database (d1), CRM database (d2), and supply chain database (d3), and needs to realize cross-database retrieval of "customer-related data" and "transaction records" and the like.

[0047] Collect user historical search information, collect search records within 1 month per unit time, collect search fields of each search record, and collect all search fields to obtain a full-amount search field set F total , F total ​={f1=“customer name”, f2=“order number”, f3=“contact number”, f4=“transaction amount”, f5=“order date”}; for example, the 3rd search record includes the 3rd search field set F3, F3={f 31 =“customer name”, f h2 =“order number”}; collect the database identifiers included in each search record, and collect all the database identifiers to obtain the database identifier set D, D={d1=“ERP database”, d2=“CRM database”, d3=“supply chain database”}; collect the category labels corresponding to the returned results of each search record, and collect all the category labels to obtain the result category label set C, C={c1=“customer basic information”, c2=“transaction record”, c3=“order details”}; collect the user feedback information corresponding to each search record, and collect all the feedback information to obtain the user feedback record set R feedback , collect p=10 feedback records, each R g contains a database identifier d g and a feedback result r g , the 5th record g=5 searches the field “customer name”, involves databases d1, d2, calculates the word frequency max freq =0.4, the preset threshold θ u =0.3, so r5=1 is positive feedback.

[0048] For the field f1=“customer name” in the database d1, it is calculated that the number of records containing “customer name” is N(f 31, d1)=80, the total number of records is N total (d1)=200, then the field frequency is: freq(f 31, d1)=80 / 200=0.4.

[0049] F total , D, C, R feedback and the calculated field frequency are stored in the feature library, and field, database and category indexes are established to support fast query. The index structure includes taking the field f total in the search field set F h as the core key, associating its field frequency freq(f j d h, ) in the database d j and the corresponding single search field set F h index; taking d j in the database identifier set D as the key, associating all the field lists contained in the database, the mapping relationship between c l in the corresponding category label set C, and the user feedback record set R feedback involved dj Feedback record index; by category label c l Using the key, it associates all database identifiers containing this category, the correlation strength between fields and categories, and historical search hit statistics. Fast retrieval is achieved through cross-mapping of field ID, database ID, and category ID, ensuring efficient querying across the entire chain from field retrieval to database location and category matching.

[0050] The class labels C are converted into a set of semantic vectors using the pre-trained Transformer-based model BERT. Here, v1 corresponds to c1 = "Customer Basic Information", v2 corresponds to c2 = "Transaction Records", and v3 corresponds to c3 = "Order Details"; all are 768-dimensional real vectors. For example, to calculate the cosine similarity between the semantic vector v1 = [0.2, 0.5, ...] of c1 = "Customer Basic Information" and the semantic vector v2 = [0.3, 0.4, ...] of c2 = "Transaction Records": For field f in database d1 31 Calculate the conditional probability of "Customer Name" and category c1 = "Customer Basic Information": d2 contains f 31 And return the number of records in c1. d2 contains f 31 Number of records The maximum similarity of c1 to other libraries of the same category is obtained by weighting the cross-database category associations. If γ = 0.5, then the correlation strength is .

[0051] Calculate the search success rate and field coverage for each database: For database d1, the number of feedback records p=4, and the sum of positive feedback records... ,but Furthermore, for database d1, the sum of weighted values ​​of all fields. =0.4×1.2+0.3×0.9+…=2.8, therefore coverage( =2.8 / 5 = 0.56.

[0052] Database priority weights are obtained by linearly weighting the search success rate and field coverage, with λ=0.6. The initial weight of database d1 is: w(d1)=0.6×0.75+(1-0.6)×0.56=0.45+0.224=0.674; The sum of the initial weights of the three databases =0.674+0.58+0.42=1.674, then the normalized weight of d1 is: The most weight set W = {0.403, 0.347, 0.25} is stored in the weight configuration library.

[0053] Further, the user input new request is “query 2024 customer order information”, the field set is parsed, the correlation strength of the database d1 and c2 = “transaction record” is calculated: ; The correlation strength of c2 = “transaction record” is 0.93 and the correlation strength of c2 = “order details” is 0.85, both of which are greater than 0.7, forming a high correlation set .

[0054] The databases are sorted according to the weight : d1 (0.403) > d2 (0.347) > d3 (0.25), the priority search database is taken as δ = 2, and the data in the is searched preferentially.

[0055] The search results of d1 and d2 are summarized, and after deduplication based on “order number”, the total result set R total = {r 1, r 2, r 3, r4} is obtained, μ = 0.5 is taken, r1 comes from d1, = 0.403, the correlation strength strength1 = 0.93, and then ; The results are sorted in descending order according to score (r e ), and the first N = 3 results are output, and finally score (r1) = 0.6665, score (r2) = 0.62, and score (r2) = 0.58 are output.

[0056] The application has the beneficial effects that when a new search request is received, the new search request is parsed into a new search field set, the correlation strength of the new search field set in each database with each search category is calculated, a high correlation search category set is obtained according to the correlation strength, and the new search field set is searched according to the high correlation search category set and the priority of each database to obtain a search result set. Since the high correlation search category set is filtered according to the field in the search request, the accuracy of cross-database search is improved, and in addition, the search is performed according to the high correlation search category set and the priority of each database, thereby improving the search efficiency.

[0057] In one embodiment, the field-category correlation strength determination method can not be implemented as steps A1-A5:​ In step A1, a plurality of preset information sets and corresponding index structures are constructed according to database information and historical search information of each database, wherein the preset information sets at least include a database identification set and a category label set of search results; In step A2, a semantic vector set is generated according to the category label set of search results; In step A3, a semantic vector corresponding to a category label of each database is extracted from the semantic vector set based on the index structure; In step A4, cross-database category association between category labels of different databases is calculated according to the semantic vector; In step A5, field-category association strength in each database is obtained according to the conditional probability of each category label returned by different search fields in each database and the cross-database category association.

[0058] In an embodiment, the above step A4 can be implemented as steps A41-A42 as follows: In step A41, cosine similarity of semantic vectors corresponding to category labels of different databases is calculated; In step A42, the cosine similarity is determined as the cross-database category association between category labels of different databases.

[0059] In an embodiment, the above step A5 can be implemented as steps A51-A52 as follows: In step A51, the conditional probability of each category label returned by different search fields in each database is calculated according to the following formula: ; Wherein, c l is a category label, f hi is a search field, d j is a database identification, N(f hi ∩c l , d j ) is the number of records containing search field f j and returning category label c hi in database d l , N(f hi , d j ) is the number of records containing search field f j in database d hi ; In step A52, the conditional probability of each category label returned by different search fields in each database and the cross-database category association are substituted into the following formula to determine the field-category association strength in each database: ; Among them, strength(f hi c l d j ) for database d j Search field f hi With category label c l The field-category association strength, where γ is the cross-database association weight coefficient and 0 < γ < 1. Represents database d j Category label c l Categories in other databases The correlation.

[0060] In one embodiment, step S102 above can be implemented as follows: Substituting the field-category association strength of each search field in the new search field set into each database into the following formula, we can obtain the association strength between the new search field set and each retrieval category in each database: ; Among them, F new New set of search fields, c l For category labels, d j For database identification, The search fields in the new set of search fields For database d j Search fields With category label c l The correlation strength, where n is the number of search fields in the new search field set.

[0061] In one embodiment, the priority of each database can be determined by the following steps C1-C2: In step C1, the search success rate and field coverage of different databases are calculated based on historical retrieval information; In step C2, the search success rate and field coverage are weighted and summed and then normalized to obtain the database priority weight.

[0062] In one embodiment, the method may also be implemented as steps D1-D3: In step D1, all search results are summarized and duplicate search results are removed to form the total result set; In step D2, the ranking score of each search result in the total result set is calculated based on the database priority weight and the relevance strength of the search results; In step D3, the results are sorted in descending order according to the sorting score, and a preset number of search results are output.

[0063] Figure 2Fig. 1 is a schematic diagram of a cross-database searching device according to an embodiment of the present application. As shown in Fig. 1, the device comprises: Figure 2 a parsing module 201 configured to parse a new search request into a new search field set when the new search request is received; a first calculating module 202 configured to calculate the association strength of the new search field set with each search category in each database according to the field-category association strength of each search field in the new search field set in each database; a screening module 203 configured to compare the association strength of the new search field set with each search category in each database with a preset association strength threshold, and screen a high-association search category set; a searching module 204 configured to search the new search field set according to the high-association search category set and the priority of each database to obtain a search result set.

[0064] In an embodiment, the field-category association strength is determined in the following manner: constructing a plurality of preset information sets and corresponding index structures according to the database information and historical search information of each database, wherein the preset information sets at least include a database identifier set and a category label set of search results; generating a semantic vector set according to the category label set of search results; extracting the semantic vector corresponding to the category label of each database from the semantic vector set based on the index structure; calculating the cross-database category association between the category labels of different databases according to the semantic vector; obtaining the field-category association strength in each database according to the conditional probability of each category label returned by different search fields in each database and the cross-database category association.

[0065] In an embodiment, the calculation of the cross-database category association between the category labels of different databases according to the semantic vector comprises: calculating the cosine similarity of the semantic vectors corresponding to the category labels of different databases; determining the cosine similarity as the cross-database category association between the category labels of different databases.

[0066] In an embodiment, the obtaining of the field-category association strength in each database according to the conditional probability of each category label returned by different search fields in each database and the cross-database category association comprises: calculating the conditional probability of each category label returned by different search fields in each database according to the following formula: ; ​Among them, c l For category labels, f hi For the search field, d j For database identification, N(f) hi ∩c l d j ) for database d j Includes search field f hi And return category label c l The number of records, N(f) hi d j ) for database d j Includes search field f hi The number of records; Substituting the conditional probabilities of different search fields returning each category label in each database and the cross-database category association into the following formula, we can determine the field-category association strength in each database: ; Among them, strength(f hi c l d j ) for database d j Search field f hi With category label c l The field-category association strength, where γ is the cross-database association weight coefficient and 0 < γ < 1. Represents database d j Category label c l Categories in other databases The correlation.

[0067] In one embodiment, the computing module is further configured to: Substituting the field-category association strength of each search field in the new search field set into each database into the following formula, we can obtain the association strength between the new search field set and each retrieval category in each database: ; Among them, F new New set of search fields, c l For category labels, d j For database identification, The search fields in the new set of search fields For database d j Search fields With category label c l The correlation strength, where n is the number of search fields in the new search field set.

[0068] In one embodiment, the priority of each database is determined as follows: According to the historical search information, search success rates and field coverage degrees of different databases are calculated; The search success rates and the field coverage degrees are weighted and summed and normalized to obtain database priority weights.

[0069] In one embodiment, the apparatus further comprises: a deduplication module configured to aggregate all search results and remove duplicate search results to form a total result set; a second calculation module configured to calculate a ranking score of each search result in the total result set according to the database priority weights and the association strength of the search result; a sorting module configured to sort the search results in descending order according to the ranking scores and output a preset number of search results.

[0070] Figure 3 For a hardware structure diagram of a cross-database search system in an embodiment of the present application, as shown in Figure 3 The cross-database search system comprises: at least one processor 320; and a memory 304 in communication with the at least one processor 320; wherein The memory 304 stores instructions executable by the at least one processor 320, and the instructions are executed by the at least one processor 320 to implement the cross-database search method described in any of the above embodiments.

[0071] Referring to Figure 3 , the cross-database search system 300 can include one or more of the following components: a processing component 302, a memory 304, a power supply component 306, a multimedia component 308, an audio component 310, an input / output (I / O) interface 312, a sensor component 314, and a communication component 316.

[0072] The processing component 302 generally controls the overall operation of the cross-database search system 300. The processing component 302 can include one or more processors 320 to execute instructions to complete all or part of the steps of the above methods. In addition, the processing component 302 can include one or more modules to facilitate interaction between the processing component 302 and other components. For example, the processing component 302 can include a multimedia module to facilitate interaction between the multimedia component 308 and the processing component 302.

[0073] Memory 304 is configured to store various types of data to support the operation of the cross-database retrieval system 300. Examples of this data include instructions for any application or method operating on the cross-database retrieval system 300, such as text, images, videos, etc. Memory 304 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0074] Power supply component 306 provides power to various components of the cross-database retrieval system 300. Power supply component 306 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the vehicle control system 300.

[0075] Multimedia component 308 includes a screen that provides an output interface between cross-database retrieval system 300 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 308 may also include a front-facing camera and / or a rear-facing camera. When the cross-database retrieval system 300 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0076] Audio component 310 is configured to output and / or input audio signals. For example, audio component 310 includes a microphone (MIC) configured to receive external audio signals when the cross-database retrieval system 300 is in an operating mode, such as alarm mode, recording mode, voice recognition mode, and voice output mode. The received audio signals may be further stored in memory 304 or transmitted via communication component 316. In some embodiments, audio component 310 also includes a speaker for outputting audio signals.

[0077] I / O interface 312 provides an interface between processing component 302 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, start buttons, and lock buttons.

[0078] The sensor component 314 includes one or more sensors for providing status assessment of various aspects of the cross-database search system 300. For example, the sensor component 314 can include a sound sensor. In addition, the sensor component 314 can detect the open / close status of the cross-database search system 300, the relative positioning of components, such as the display and keypad of the cross-database search system 300, the sensor component 314 can also detect the operational status of the cross-database search system 300 or a component of the cross-database search system 300. The sensor component 314 can also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 314 can also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0079] The communication component 316 is configured to enable the cross-database search system 300 to provide wired or wireless communication capabilities with other devices and cloud platforms. The cross-database search system 300 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an example embodiment, the communication component 316 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In an example embodiment, the communication component 316 also includes a near field communication (NFC) module to facilitate short-range communications. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0080] In an example embodiment, the cross-database search system 300 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic elements for performing the cross-database search method as described in any of the above embodiments.

[0081] The present application also provides a computer-readable storage medium, when the instructions in the storage medium are executed by the processor corresponding to the cross-database search system, the cross-database search system can implement the cross-database search method as described in any of the above embodiments.

[0082] Those skilled in the art will appreciate that embodiments of the present application can be readily used as software, hardware, or a combination of software and hardware. In one embodiment, the present application can be implemented in software and can be stored on a computer readable medium, which can include random access memory (RAM), read only memory (ROM), magnetic disk or optical disk, or the like. The software implementation can comprise one or more computer program components embodied on one or more computer readable media. The computer readable medium can be resident within a computing device, such as an external storage device (e.g., compact disk [CD], digital versatile disk [DVD], etc.) or an internal storage device, including but not limited to a magnetic or optical disk. The computer program components can also be downloaded to a computer from an external source, via a computer network or introduced into a computer through use of a computer program product, such as a diskette or disk, USB drive, memory stick, etc. Implementations of the present application in software can be implemented in a straightforward manner using well-known or proprietary software programming tools that are available to those skilled in the art.

[0083] The present application is described in relation to flow diagrams and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present application. It is understood that each block of the flow diagrams and / or block diagrams, and combinations of blocks in the flow diagrams and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing device, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, create means for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flow diagram and / or block diagram block or blocks. Figure 1 one or more functions specified in the flow diagram and / or block diagram block or blocks.

[0084] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the flow diagrams and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flow diagram and / or block diagram block or blocks. Figure 1 one or more functions specified in the flow diagram and / or block diagram block or blocks.

[0085] These computer program instructions can also be loaded onto a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer implemented process such that the instructions that execute on the computer or other programmable device provide steps for implementing the flow diagrams and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flow diagram and / or block diagram block or blocks. Figure 1 one or more functions specified in the flow diagram and / or block diagram block or blocks.

[0086] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A cross-repository search method, characterized by, The method comprises the following steps: When a new search request is received, the new search request is parsed into a new search field set; The association strength of the new search field set with each search category in each database is calculated according to the field-category association strength of each search field in the new search field set in each database; The association strength of the new search field set with each search category in each database is compared with a preset association strength threshold, and a high-association search category set is screened out; The new search field set is searched according to the high-association search category set and the priority of each database to obtain a search result set.

2. The method of claim 1, wherein, The field-category association strength is determined in the following manner: A plurality of preset information sets and corresponding index structures are constructed according to database information and historical search information of each database, wherein the preset information sets at least include a database identifier set and a category label set of search results; A semantic vector set is generated according to the category label set of the search results; The semantic vector corresponding to the category label of each database is extracted from the semantic vector set based on the index structure; The cross-database category association between the category labels of different databases is calculated according to the semantic vectors; The field-category association strength in each database is obtained according to the conditional probability of each category label returned by each search field in each database and the cross-database category association.

3. The method of claim 2, wherein, The cross-database category association between the category labels of different databases is calculated according to the semantic vectors, which comprises: The cosine similarity of the semantic vectors corresponding to the category labels of different databases is calculated; The cosine similarity is determined as the cross-database category association between the category labels of different databases.

4. The method of claim 2, wherein, The field-category association strength in each database is obtained according to the conditional probability of each category label returned by each search field in each database and the cross-database category association, which comprises: The conditional probability of each category label returned by each search field in each database is calculated according to the following formula; ； where c l is a category label, f hi is a search field, d j is a database identifier, N(f hi ∩ c l , d j ) is the number of records in database d j containing search field f hi and returning category label c l , and N(f hi , d j ) is the number of records in database d j containing search field f hi ; The conditional probability of each category label returned by each search field in each database and the cross-database category association are substituted into the following formula to determine the field-category association strength in each database: ； where strength(f hi , c l , d j ) is the field-category association strength of field f hi with category label c l in database d j , γ is a cross-database association weight coefficient and 0 < γ < 1, represents the association of category label c j in database d l with categories in other databases.

5. The method of claim 1, wherein, The association strength of the new search field set with each search category in each database is calculated according to the field-category association strength of each search field in the new search field set in each database, which comprises: The association strength of the new search field set with each search category in each database is calculated according to the field-category association strength of each search field in the new search field set in each database, which comprises: ； Among them, F new New set of search fields, c l For category labels, d j For database identification, The search fields in the new set of search fields For database d j Search fields With category label c l The correlation strength, where n is the number of search fields in the new search field set.

6. The method of claim 1, wherein, The priority of each database is determined in the following manner: The search success rate and field coverage of different databases are calculated according to historical search information; The search success rate and field coverage are weighted and summed and normalized to obtain a database priority weight.

7. The method of claim 1, wherein, The method further comprises: All search results are aggregated and duplicate search results are removed to form a total result set; The sorting score of each search result in the total result set is calculated according to the database priority weight and the association strength of the search result; The sorting score is sorted in descending order, and a preset number of search results are output.

8. A cross-repository search apparatus, comprising: The method comprises the following steps: a parsing module, configured to parse a new search request into a new search field set when the new search request is received; a first calculating module, configured to calculate the association strength of the new search field set with each search category in each database according to the field-category association strength of each search field in the new search field set in each database; a screening module, configured to compare the association strength of the new search field set with each search category in each database with a preset association strength threshold, and screen a high-association search category set; a searching module, configured to search the new search field set according to the high-association search category set and the priority of each database to obtain a search result set.

9. A cross-repository search system, comprising: comprise: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to implement the cross-database search method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor corresponding to the cross-database search system, the cross-database search system can implement the cross-database search method according to any one of claims 1-7.

Images