Search engine data updating method, device, equipment and storage medium

Abstract

The present application provides a search engine data update method, device, equipment and storage medium, wherein the method includes: obtaining a first query statement in a first search engine and calculating a correlation value with a second search engine, selecting a third search engine from each of the second search engines, obtaining the second query statement in the data source and the first text content corresponding to each of the second query statements, and updating them to the system data source of the first search engine. The beneficial effect of the present application is that the system data source is updated by obtaining the data source of the second search engine, thereby enriching the text content in the system data source of the first search engine, and effectively reducing the possibility of deviation in the query results of users during the retrieval process.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a search engine data updating method, device, equipment and storage medium. Background Art

[0002] A search engine is a system that collects information from the Internet based on certain strategies and uses specific computer programs. After organizing and processing the information, it provides retrieval services to users and displays information related to their searches to users.

[0003] At present, with the development of the Internet, search engines are limited by the lack of data in their own system data sources, resulting in users being unable to obtain the latest results in a timely manner, which in turn leads to the problem of biased query results during the user's retrieval process. Summary of the invention

[0004] The main purpose of this application is to provide a search engine data update method, device, equipment and storage medium, aiming to solve the problem that due to the lack of rich data in the system data source, the query results of users in the retrieval process are biased.

[0005] The present application provides a search engine data updating method, comprising:

[0006] Obtaining a plurality of first query statements that do not match corresponding correct text content in a first search engine;

[0007] Calculating the correlation value between each of the first query statements and the second search engine; wherein the first search engine and the second search engine are of different categories;

[0008] Adding the correlation values ​​of the plurality of the first query statements to obtain the correlation sum of the second search engine; wherein the second search engine includes a plurality of;

[0009] selecting a third search engine from each of the second search engines according to the value of the relevant sum;

[0010] Extracting a data source of the third search engine;

[0011] Acquire second query statements in the data source and first text content corresponding to each of the second query statements;

[0012] The second query statements and the first text content corresponding to each of the second query statements are updated to the system data source of the first search engine.

[0013] Furthermore, the step of updating the second query statements and the first text content corresponding to each of the second query statements to the system data source of the first search engine includes:

[0014] Obtaining a third query statement in the first search engine and second text content corresponding to the third query statement;

[0015] Extracting identical query statements between each of the third query statements and each of the second query statements;

[0016] Identify, through a flink stream, a first matching degree between the same query statement and the first text content, and a second matching degree between the same query statement and the second text content;

[0017] Determining whether the second matching degree is greater than the first matching degree;

[0018] If so, the first text content is replaced by the second text content in the system data source.

[0019] Furthermore, after the step of obtaining the third query statement and the corresponding second text content in the first search engine, the method further includes:

[0020] Extracting different query statements from each of the second query statements, each of which has a different query content from each of the third query statements, as target query statements;

[0021] The target query statement and the first text corresponding to the target query statement are added to the system data source of the first search engine.

[0022] Furthermore, before the step of updating the second query statements and the first text content corresponding to each of the second query statements into the first search engine, the method further includes:

[0023] Obtaining a third query statement and corresponding second text content in the first search engine;

[0024] Acquire first dimensional information of the first text content, and second dimensional information corresponding to the second text content; wherein the first dimensional information and the second dimensional information respectively include multiple dimensions;

[0025] Comparing the first dimensional information with the second dimensional information to obtain a target dimension that is not present in the first text content;

[0026] Based on the target dimension, acquiring dimensional content corresponding to the target dimension from the corresponding third search engine;

[0027] The dimensional content is added to the first text content.

[0028] Furthermore, the step of calculating the correlation value between each of the first query statements and the second search engine includes:

[0029] Obtaining attribute information of each of the second search engines;

[0030] Each of the first query statements and the attribute information is vectorized respectively to obtain a statement vector X corresponding to each of the first query statements. i =(x 1i , x 2i , x 3i …x ni ), and the attribute vector Y corresponding to the attribute information j =(y 1j ,y 2j ,y 3j …y nj ), where X i represents the sentence vector corresponding to the first query sentence of the i-th query, and represents Y j represents the attribute vector corresponding to the j-th second search engine;

[0031] According to the formula:

[0032] The correlation value is calculated; wherein, sim ij Represents the correlation value between the i-th first query statement and the j-th second search engine.

[0033] Furthermore, the step of selecting a third search engine from each of the second search engines according to the value of the relevant sum includes:

[0034] Extracting a third search engine whose correlation sum is greater than a correlation threshold from the second search engine;

[0035] Acquiring attribute information of each of the third search engines;

[0036] Calculating the similarity between any two of the third search engines using a preset similarity algorithm;

[0037] Determine whether there is a similar engine combination with a similarity greater than a similarity threshold;

[0038] If yes, the search engines in the similar engine combination are recorded as similar search engines according to a preset screening method;

[0039] The search engines other than the similar search engines among the third search engines are recorded as third search engines.

[0040] Furthermore, the step of updating the second query statements and the first text content corresponding to each of the second query statements to the system data source of the first search engine includes:

[0041] Obtaining a third query statement and corresponding second text content in the first search engine;

[0042] Extracting identical query statements between each of the third query statements and each of the second query statements;

[0043] Identify the first text content and the second text content corresponding to the same query statement through the flink stream, and obtain a first timestamp of the first text content and a second timestamp of the second text content;

[0044] Determining whether the second timestamp is after the first timestamp;

[0045] If so, the first text content is replaced by the second text content in the system data source.

[0046] The present application also provides a data updating device for a search engine, comprising:

[0047] A first acquisition module is used to acquire a plurality of first query statements that do not match corresponding correct text content in a first search engine;

[0048] A first calculation module, configured to calculate a correlation value between each of the first query statements and a second search engine; wherein the first search engine and the second search engine are of different categories;

[0049] A second calculation module is used to add the correlation values ​​of the plurality of the first query statements to obtain the correlation sum of the second search engine; wherein the second search engine includes a plurality of;

[0050] A selection module, configured to select a third search engine from each of the second search engines according to the value of the relevant sum;

[0051] An extraction module, used to extract the data source of the third search engine;

[0052] A second acquisition module, used to acquire second query statements in the data source and first text content corresponding to each of the second query statements;

[0053] An updating module is used to update the second query statement and the first text content corresponding to each of the second query statements into the system data source of the first search engine.

[0054] The present application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any of the above methods when executing the computer program.

[0055] The present application also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any of the above-mentioned methods are implemented.

[0056] The beneficial effects of the present application are as follows: by acquiring the data source of the third search engine to update the system data source, the text content in the system data source of the first search engine is enriched, and the possibility of deviation in the query results during the user's retrieval process is effectively reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 It is a flowchart of a data updating method of a search engine according to an embodiment of the present application;

[0058] Figure 2 It is a schematic block diagram of the structure of a data updating device for a search engine according to an embodiment of the present application;

[0059] Figure 3 A schematic block diagram of the structure of a computer device according to an embodiment of the present application.

[0060] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0061] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0062] It should be noted that all directional indications in the embodiments of the present application (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly. The connection can be a direct connection or an indirect connection.

[0063] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and B can represent three situations: A exists alone, A and B exist at the same time, and B exists alone.

[0064] In addition, in this application, descriptions such as "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in this field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0065] Reference Figure 1 , the present application proposes a search engine data updating method, comprising:

[0066] S1: Obtaining a plurality of first query statements that do not match corresponding correct text content in a first search engine;

[0067] S2: Calculate the correlation value between each of the first query statements and the second search engine; wherein the first search engine and the second search engine are of different categories;

[0068] S3: Adding the correlation values ​​of the plurality of the first query statements to obtain the correlation sum of the second search engine; wherein the second search engine includes a plurality of search engines;

[0069] S4: selecting a third search engine from each of the second search engines according to the value of the relevant sum;

[0070] S5: extracting the data source of the third search engine;

[0071] S6: Acquire second query statements in the data source and first text content corresponding to each of the second query statements;

[0072] S7: Update the second query statements and the first text content corresponding to each of the second query statements into the system data source of the first search engine.

[0073] As described in the above step S1, a plurality of first query statements that do not match the corresponding correct text content in the first search engine are obtained. Among them, the method of obtaining can be to obtain statements in which the user has not searched for relevant text content. Specifically, it can be reflected that the user inputs a query statement, but does not click on the text content fed back next. At this time, it can be considered that the text content is incorrect, that is, it is not the text content required by the user. Therefore, the first query statement can be recorded through the background log system, and then obtained from the log system later, so as to obtain the corresponding first query statement. In addition, the first search engine can be any search engine. For example, the search engine can be an Elasticsearch engine. The Elasticsearch engine is a full-text search engine that is currently widely used. It has a wide range of applications in the field of big data search and analysis, and can provide real-time search application services. In addition, the first query statement can be a query statement directly searched by the user, or it can be a query information entered by the user, and then a query statement is obtained by extracting keyword information therein.

[0074] As described in the above step S2, the correlation value between each of the first query statements and the second search engine is calculated; wherein, the first search engine and the second search engine are of different categories; specifically, the second search engine and the first search engine are search engines with different usage scenarios, such as Baidu's search engine and JD's search engine, and can also be Weibo's search engine. This application does not limit this. It should be noted that the architecture of the second search engine and the first search engine is preferably the same architecture, for example, both are the architecture of the Elasticsearch engine, which can facilitate the subsequent data update. In addition, the calculation method of the correlation value will be described in detail later, and will not be repeated here.

[0075] As described in step S3 above, the correlation values ​​of the plurality of first query statements are added together to obtain the sum of the correlations of the second search engine; wherein the second search engine includes a plurality of second search engines. The correlation values ​​of the second search engines and the corresponding first query statements have been calculated, so the sum of the correlation values ​​of each second search engine and each first query statement can be calculated.

[0076] As described in step S4 above, a third search engine is selected from each of the second search engines based on the value of the relevant sum. Since the first query statement is a query statement that does not match the correct text content, it can be seen that the system data source of the first search engine is missing the text content corresponding to these query statements. Therefore, the larger the sum of the relevant values, the more the corresponding third search engine matches the first query statement. Therefore, based on the sum of the relevant values, the third search engine that matches each of the first query statements more can be found. In this way, the data source corresponding to the third search engine can be obtained to update the system data source of the first search engine.

[0077] As described in step S5 above, the data source of the third search engine is extracted, wherein the extraction method may be to directly obtain it from the database corresponding to the corresponding third search engine, thereby completing the acquisition of the data source of the third search engine.

[0078] As described in step S6 above, the second query statements in the data source and the first text content corresponding to each of the second query statements are obtained. The corresponding second query statements and the corresponding first text content are stored in the data source, so they can be directly obtained from the data source.

[0079] As described in the above step S7, the second query statement and the first text content corresponding to each of the second query statements are updated to the system data source of the first search engine. Among them, the updating method includes replacement, deletion and addition. The specific updating method will be described in the subsequent embodiments and will not be repeated here. This solves the problem of search engine limitations and insufficient data in the system data source itself. By obtaining the data source of the third search engine to update the system data source, the text content in the system data source of the first search engine is enriched, effectively reducing the possibility of deviation in the query results during the user's retrieval process.

[0080] In one embodiment, the step S7 of updating the second query statements and the first text content corresponding to each of the second query statements to the system data source of the first search engine includes:

[0081] S701: Acquire a third query statement and second text content corresponding to the third query statement in the first search engine;

[0082] S702: extracting identical query statements between each of the third query statements and each of the second query statements;

[0083] S703: Identify, through a flink stream, a first matching degree between the same query statement and the first text content, and a second matching degree between the same query statement and the second text content;

[0084] S704: Determine whether the second matching degree is greater than the first matching degree;

[0085] S705: If yes, replacing the first text content with the second text content in the system data source.

[0086] As described in the above steps S701-S705, the data in the system data source is updated. Specifically, the corresponding third query statement and the corresponding second text content are obtained. It should be noted that the third query statement contains the first query statement. Since the system data source may contain the same query statement as the second query statement, there are two situations at this time. The first is that the first search engine and the corresponding third search engine feedback the same text content based on the same query statement, and the second feedback is different text content. For the second situation, it is necessary to identify the first matching degree of the same query statement with the first text content and the second matching degree with the second text content through the flink stream. Flink stream is an open source stream-based stateful computing framework, which is executed in a distributed manner, has excellent performance of low latency and high throughput, and is very good at handling stateful complex computing logic scenarios. Therefore, the corresponding matching degree can be calculated through the Flik stream. The matching degree can be obtained based on the number of times the user motors the text content corresponding to the same query statement, or it can be obtained based on real-time hot spots to obtain the corresponding matching degree, thereby obtaining the corresponding first matching degree and second matching degree. Subsequently, based on the size of the second matching degree and the first matching degree, it is determined whether to replace the first text content with the second text content, thereby completing the update of the system data source.

[0087] In one embodiment, after the step S701 of obtaining the third query statement and the corresponding second text content in the first search engine, the method further includes:

[0088] S7021: extracting different query statements from each of the second query statements, each of which has a different query content from each of the third query statements, as target query statements;

[0089] S7022: Add the target query statement and the first text corresponding to the target query statement to the system data source of the first search engine.

[0090] As described in steps S7021-S7022, the first text is added. Specifically, since there is no corresponding query statement in the system data source, the corresponding different query statements can be directly added to the system data source, thereby supplementing the text content in the system data source.

[0091] In one embodiment, before the step S7 of updating the second query statements and the first text content corresponding to each of the second query statements to the first search engine, the method further includes:

[0092] S601: Obtain a third query statement and corresponding second text content in the first search engine;

[0093] S602: Acquire first dimension information of the first text content, and second dimension information corresponding to the second text content; wherein the first dimension information and the second dimension information respectively include multiple dimensions;

[0094] S603: Compare the first dimension information with the second dimension information to obtain a target dimension that is not included in the first text content;

[0095] S604: Acquire dimensional content corresponding to the target dimension from the corresponding third search engine based on the target dimension;

[0096] S605: Add the dimensional content to the first text content.

[0097] As described in the above steps S601-S605, the first text content is improved. Specifically, due to the difference in search engines, the dimensional information that the corresponding text content may contain is also different. For example, there is no corresponding text content source in the first text content, while the second text content contains the corresponding text content source. At this time, in order to make the text content in the first search engine consistent, it is necessary to obtain the missing information, specifically, first obtain the first dimensional information of the first text content, and the second dimensional information corresponding to the second text content, the dimensional information is respectively various dimensions of the text content, and then find the target dimension that is not available in the first text content, obtain the corresponding dimensional content based on the target dimension, and then complete the first text content, so that the first text content has all the dimensions in the dimensional information, thereby completing the first text content and making it consistent with the dimensional information contained in the second text content, and ensuring the consistency of the first search engine.

[0098] In one embodiment, the step S2 of calculating the correlation value between each of the first query statements and each of the preset second search engines includes:

[0099] S201: Acquire attribute information of each of the second search engines;

[0100] S202: Vectorize each of the first query statements and the attribute information respectively to obtain a statement vector X corresponding to each of the first query statements. i =(x 1i , x 2i , x 3i …x ni ), and the attribute vector Y corresponding to the attribute information j =(y 1j ,y 2j ,y 3j …y nj), where X i represents the sentence vector corresponding to the first query sentence of the i-th query, and represents Y j represents the attribute vector corresponding to the j-th second search engine;

[0101] S203: According to the formula:

[0102] The correlation value is calculated; wherein, sim ij Represents the correlation value between the i-th first query statement and the j-th second search engine.

[0103] As described in the above steps S202-S203, the calculation of the relevant value is realized. Specifically, the open source and free Python LibShortText text classifier can be used, and a custom Chinese word segmenter can be used to segment the text in each of the first query statements and each of the attribute information. The word segmentation can be calculated by any algorithm such as decision tree, multi-layer perceptron, naive Bayes (including Bernoulli Bayes, Gaussian Bayes and polynomial Bayes), random forest, AdaBoost, feedforward neural network and LSTM. Vectorize each of the first query statements and each of the attribute information after word segmentation to obtain the multi-dimensional coordinates of the target vector for subsequent calculation. The vectorization method can be vectorization through the Googleword2vec tool.

[0104] According to the formula:

[0105] The correlation value is calculated and a correlation threshold can be set. When the correlation value is greater than the correlation threshold, it can be considered that the first query statement and the attribute information are similar; when the correlation value is less than or equal to the correlation threshold, it can be considered that the first query statement and the attribute information are not similar.

[0106] In one embodiment, the step S4 of selecting one or more third search engines from each of the second search engines based on each of the relevant sums comprises:

[0107] S401: extracting a third search engine whose correlation sum is greater than a correlation threshold from the second search engine;

[0108] S402: Acquire attribute information of each of the third search engines;

[0109] S403: Calculating the similarity between any two of the third search engines using a preset similarity algorithm;

[0110] S404: Determine whether there is a similar engine combination with a similarity greater than a similarity threshold;

[0111] S405: If yes, then the search engines in the similar engine combination are recorded as similar search engines according to a preset screening method;

[0112] S406: Recording the search engines other than the similar search engines in the third search engines as third search engines.

[0113] As described in the above steps S401-S406, the third search engine is screened. Since some third search engines are similar, for example, the search engine of Baidu Library and the search engine of HowNet are similar, it is necessary to calculate the similarity between any two of the third search engines through a preset similarity algorithm. The preset similarity algorithm can be a cosine similarity algorithm, an Euclidean distance algorithm, etc., to obtain the similarity of similar search engines, and then determine whether there is a similar engine combination with a similarity greater than a similarity threshold, wherein the similar engine combination contains two third search engines with a similarity greater than the similarity threshold, and then the search engines in the similar engine combination are recorded as similar search engines according to the preset screening method. Among them, the preset screening method can be screening according to the search popularity of the two third engines, selecting one of the third engines with a lower popularity as a similar search engine and excluding it, or it can be selected according to the size of its data source. This application does not limit this, and the screening of the third search engine can be achieved to avoid repeated updates.

[0114] In one embodiment, the step S7 of updating the second query statements and the first text content corresponding to each of the second query statements to the system data source of the first search engine includes:

[0115] S711: Obtain a third query statement and corresponding second text content in the first search engine;

[0116] S712: extracting identical query statements between each of the third query statements and each of the second query statements;

[0117] S713: Identify the first text content and the second text content corresponding to the same query statement through the flink stream, and obtain a first timestamp of the first text content and a second timestamp of the second text content;

[0118] S714: Determine whether the second timestamp is after the first timestamp;

[0119] S715: If yes, replacing the first text content with the second text content in the system data source.

[0120] As described in the above steps S711-S715, the data in the system data source is updated. Specifically, the corresponding third query statement and the corresponding second text content are obtained. It should be noted that the third query statement contains the first query statement. Since the system data source may contain the same query statement as the second query statement, there are two situations at this time. The first is that the first search engine and the corresponding third search engine feedback the same text content based on the same query statement, and the second feedback is different text content. For the second situation, it is necessary to identify the first timestamp of the same query statement and the first text content, and the second timestamp of the second text content through the flink stream. Flink stream is an open source stream-based stateful computing framework, which is executed in a distributed manner, has excellent performance of low latency and high throughput, and is very good at handling stateful complex computing logic scenarios. Therefore, the corresponding timestamp can be calculated through the Flik stream. The timestamp can be obtained based on the last time the user clicked the same query statement, or it can be determined based on the hotspot time of clicking the same query statement, that is, the most recent time point when the number of clicks is greater than the set value, thereby obtaining the corresponding first timestamp and second timestamp. Subsequently, based on the size of the second timestamp and the first timestamp, it is determined whether to replace the first text content with the second text content, thereby completing the update of the system data source.

[0121] Reference Figure 2 The present application also provides a search engine data updating device, comprising:

[0122] A first acquisition module 10 is used to acquire a plurality of first query statements that do not match corresponding correct text content in a first search engine;

[0123] A first calculation module 20, configured to calculate a correlation value between each of the first query statements and a second search engine; wherein the first search engine and the second search engine are of different categories;

[0124] A second calculation module 30 is used to add the correlation values ​​of the plurality of the first query statements to obtain the correlation sum of the second search engine; wherein the second search engine includes a plurality of;

[0125] A selection module 40, configured to select a third search engine from each of the second search engines according to the value of the relevant sum;

[0126] An extraction module 50, used to extract the data source of the third search engine;

[0127] A second acquisition module 60 is used to acquire second query statements in the data source and first text content corresponding to each second query statement;

[0128] The updating module 70 is used to update the second query statement and the first text content corresponding to each of the second query statements into the system data source of the first search engine.

[0129] The beneficial effects of the present application are as follows: by acquiring the data source of the third search engine to update the system data source, the text content in the system data source of the first search engine is enriched, and the possibility of deviation in the query results during the user's retrieval process is effectively reduced.

[0130] Reference Figure 3 In an embodiment of the present application, a computer device is also provided. The computer device may be a server, and its internal structure may be as follows: Figure 3 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor designed by the computer is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store various text contents, etc. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the data update method of the search engine described in any of the above embodiments can be implemented.

[0131] Those skilled in the art will understand that Figure 3 The structure shown in is merely a block diagram of a portion of the structure related to the present application solution and does not constitute a limitation on the computer device to which the present application solution is applied.

[0132] The embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the search engine data updating method described in any of the above embodiments can be implemented.

[0133] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing related hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided in this application and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0134] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, device, article or method including the element.

[0135] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0136] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0137] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present application shall be included in the scope of the claims of the present application.

Claims

1. A method for updating data of a search engine, characterized in that: include: Obtaining a plurality of first query statements that do not match corresponding correct text content in a first search engine; Calculating the correlation value between each of the first query statements and the second search engine; wherein the first search engine and the second search engine are of different categories; Adding the correlation values ​​of the plurality of the first query statements to obtain the correlation sum of the second search engine; wherein the second search engine includes a plurality of search engines; selecting a third search engine from each of the second search engines according to the value of the relevant sum; Extracting a data source of the third search engine; Acquire second query statements in the data source and first text content corresponding to each of the second query statements; updating the second query statements and the first text content corresponding to each of the second query statements into the system data source of the first search engine; The step of calculating the correlation value between each of the first query statements and the second search engine includes: Acquiring attribute information of the second search engine; Each of the first query statements and the attribute information is vectorized respectively to obtain a statement vector X corresponding to each of the first query statements. i =(x 1i , x 2i , x 3i …x ni ), and the attribute vector Y corresponding to the attribute information j =(y 1j ,y 2j ,y 3j …y nj ), where X i represents the sentence vector corresponding to the first query sentence of the i-th query, Y j represents the attribute vector corresponding to the j-th second search engine; According to the formula: The correlation value is calculated; wherein, sim ij represents the correlation value between the i-th first query statement and the j-th second search engine; The step of selecting a third search engine from each of the second search engines according to the value of the relevant sum includes: Extracting a third search engine whose correlation sum is greater than a correlation threshold from the second search engine; Acquiring attribute information of each of the third search engines; Calculating the similarity between any two of the third search engines using a preset similarity algorithm; Determine whether there is a similar engine combination with a similarity greater than a similarity threshold; If yes, the search engines in the similar engine combination are recorded as similar search engines according to a preset screening method; The search engines other than the similar search engines in the third search engines are recorded as third search engines.

2. The search engine data updating method according to claim 1, characterized in that: The step of updating the second query statements and the first text content corresponding to each of the second query statements into the system data source of the first search engine includes: Obtaining a third query statement in the first search engine and second text content corresponding to the third query statement; Extracting identical query statements between each of the third query statements and each of the second query statements; Identify, through a flink stream, a first matching degree between the same query statement and the first text content, and a second matching degree between the same query statement and the second text content; Determining whether the second matching degree is greater than the first matching degree; If so, the first text content is replaced by the second text content in the system data source.

3. The search engine data updating method according to claim 2, characterized in that: After the step of obtaining the third query statement in the first search engine and the second text content corresponding to the third query statement, the method further includes: Extracting different query statements in the second query statements that have different query contents from each of the third query statements as target query statements; The target query statement and the first text corresponding to the target query statement are added to the system data source of the first search engine.

4. The search engine data updating method according to claim 1, characterized in that: Before the step of updating the second query statements and the first text content corresponding to each of the second query statements to the first search engine, the method further includes: Obtaining a third query statement and corresponding second text content in the first search engine; Acquire first dimensional information of the first text content, and second dimensional information corresponding to the second text content; wherein the first dimensional information and the second dimensional information respectively include multiple dimensions; Comparing the first dimensional information with the second dimensional information to obtain a target dimension that is not present in the first text content; Based on the target dimension, acquiring dimensional content corresponding to the target dimension from the corresponding third search engine; The dimensional content is added to the first text content.

5. The search engine data updating method according to claim 1, characterized in that: The step of updating the second query statements and the first text content corresponding to each of the second query statements into the system data source of the first search engine includes: Obtaining a third query statement and corresponding second text content in the first search engine; Extracting identical query statements between each of the third query statements and each of the second query statements; Identify the first text content and the second text content corresponding to the same query statement through the flink stream, and obtain a first timestamp of the first text content and a second timestamp of the second text content; Determining whether the second timestamp is after the first timestamp; If so, the first text content is replaced by the second text content in the system data source.

6. A data updating device for a search engine, used to implement the method according to any one of claims 1 to 5, characterized in that: include: A first acquisition module is used to acquire a plurality of first query statements that do not match corresponding correct text content in a first search engine; A first calculation module, configured to calculate a correlation value between each of the first query statements and a second search engine; wherein the first search engine and the second search engine are of different categories; A second calculation module is used to add the correlation values ​​of the plurality of the first query statements to obtain the correlation sum of the second search engine; wherein the second search engine includes a plurality of; A selection module, configured to select a third search engine from each of the second search engines according to the value of the relevant sum; An extraction module, used to extract the data source of the third search engine; A second acquisition module, used to acquire second query statements in the data source and first text content corresponding to each of the second query statements; An updating module is used to update the second query statement and the first text content corresponding to each of the second query statements into the system data source of the first search engine.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images