A cross-platform data integration retrieval system and its medium

Abstract

The present invention relates to a cross-platform data integration retrieval system and its medium for a data retrieval system, including a package retrieval platform server, the retrieval platform server is respectively signal-connected with an Internet module, a temporary medium, a hot word medium and a plurality of retrieval terminals. In the present application, the object data of the Internet retrieval is a multi-platform retrievalable on the Internet, and its data volume is huge. A large amount of processing resources are required during the retrieval process. Therefore, the Internet retrieval will only be enabled when the retrieval system is in idle operation and load operation. When the retrieval system is in an overloaded state, in order to reduce the pressure on the retrieval system, only the data in the temporary medium will be enabled for retrieval, so as to meet the common retrieval needs of most users, increase the high-pressure resistance of the retrieval system, and enable the retrieval platform to maintain the most basic common retrieval functions, and not easily cause the retrieval system to crash directly.

Description

Technical Field

[0001] The present invention relates to a data retrieval system and a medium thereof, and in particular to a cross-platform data integration retrieval system and a medium thereof applied in the field of data retrieval systems. Background Art

[0002] As the name implies, a cross-platform data integration and retrieval system is a comprehensive system that can cross the boundaries of different operating systems, databases, applications, and even cloud service platforms to achieve centralized integration and efficient retrieval of dispersed and heterogeneous data. The system uses advanced data integration, conversion, storage, and retrieval technologies to seamlessly connect originally isolated and difficult-to-interoperate data sources, providing users with a unified and convenient data access portal.

[0003] The specification of invention patent 202010195880.8 discloses a data retrieval method based on the Android system. The data retrieval method based on the Android system provided by the invention is modeled in Matlab, and then compiled to generate C source code, and the Android NDK cross-compilation algorithm is used to realize cross-platform code calls to complete data retrieval. This method avoids the problems of low efficiency and large errors caused by curve fitting, making the data retrieval process fast and accurate.

[0004] In the above application, a cross-compilation algorithm based on the Android system is established to realize cross-platform code calls, increase the speed and accuracy of the data retrieval process, and is suitable for low-traffic retrieval needs. With the rapid development of information technology, cross-platform data integration and retrieval have become an important part of information processing. However, when the existing system processes large-scale data, the large amount of data and complex sources will cause a sudden increase in system carrying pressure, which will lead to the overall collapse of the algorithm system and the inability to maintain normal retrieval functions. Summary of the invention

[0005] In view of the above-mentioned prior art, the technical problem to be solved by the present invention is that when the existing system processes large-scale data, the system load pressure will increase suddenly due to the large amount of data and complex sources, which will lead to the overall collapse of the algorithm system and the inability to maintain normal retrieval functions.

[0006] To solve the above problems, the present invention provides a cross-platform data integration retrieval system and its medium, including a package retrieval platform server, the retrieval platform server is respectively signal-connected with an Internet module, a temporary medium, a hot word medium and a plurality of retrieval terminals;

[0007] Its workflow includes the following steps:

[0008] S1. Search input: users input search requirements according to their own needs using the search terminal;

[0009] S2. Determination of the retrieval system status. The retrieval platform server divides its own status into three categories: idle operation, load operation and overload operation according to its own hardware working status. When the retrieval system is in the idle operation state, the retrieval platform server directly uses the Internet module to perform Internet search on multiple platforms and outputs the final search results to the retrieval end. When the retrieval system is in the load operation state, the hot word medium is first used to perform hot word search. When the search has results, the final search results are directly output to the retrieval end. When the hot word search has no results, the Internet search is performed again through the Internet module, and finally the final search results are output to the retrieval end. When the retrieval system is in the overload operation state, the hot word medium is used to perform hot word search. When there are results, the detection results are directly output to the retrieval end. When the hot word search has no results, the retrieval system enters the standby stage and re-determines the retrieval system status every 30S until the retrieval system resumes idle operation or load operation, and then performs Internet search.

[0010] In the above-mentioned cross-platform data integration retrieval system and its medium, the high pressure resistance capability of the retrieval system is increased so that the retrieval platform can maintain the most basic commonly used retrieval functions and is not prone to direct collapse of the retrieval system.

[0011] As a further improvement of the present application, the working status of the retrieval system is determined based on overload parameters, which include CPU usage, memory occupancy and network bandwidth utilization. When the overload parameters are all lower than 70%, the retrieval system is determined to be in idle operation. When an overload parameter reaches 70%-95%, the retrieval system is determined to be in load operation. When an overload parameter reaches more than 95%, the retrieval system is determined to be in overload operation.

[0012] As a further improvement of the present application, when the retrieval system is judged to be overloaded, the retrieval platform server issues a warning, and the retrieval system enters a manual intervention state. Maintenance personnel intervene in the operation of the retrieval system and generate a warning form to record the overload parameters of the warning and eliminate transportation risks in a timely manner. The warning form facilitates subsequent targeted maintenance work.

[0013] As a further improvement of the present application, the hot word search data is derived from the hot word medium, the search output results in the search platform server are stored in a temporary medium, and the temporary medium filters the high-frequency data that appears in itself and stores it in the hot word medium, forming a hot word search database for the user, which is convenient for temporary use when the data retrieval system is overloaded.

[0014] As another improvement of the present application, the data storage rule in the temporary medium is: the results of the search output in the retrieval platform server will be stored in the temporary medium in the order of the search, and there is an upper limit to the total number of data storage items in the temporary medium. When the number of data storage items reaches the upper limit, the data is updated normally, and the newly entered data will overwrite the data that has existed in the temporary medium for the longest time. When there are duplicate searches, the duplicate search data will form a new search record, and the old search record will be deleted, forming a temporary search record in the temporary medium.

[0015] As another improved supplement to the present application, the search record in the temporary medium includes search data, labels and times. The search data is the search output result, the label is the keyword extracted by the search platform server according to the user's search requirements, and the number of times is the accumulated number of searches before the search data is overwritten. When the number of times reaches the preset number, the data enters the hot word medium to form a hot word search database.

[0016] As another improved supplement to the present application, the retrieval record stored in the hot word medium includes retrieval data, labels and time, and the time is the length of time the data exists in the hot word medium. When the length of time accumulates to 1 hour, the data is deleted, and when the data enters the hot word medium from the temporary medium again, the time of the data is reset to zero, and when the data is called by the hot word retrieval, the time of the data is also reset to zero.

[0017] As another improvement of the present application, before the retrieval output results are input into the temporary medium by the retrieval platform server, data security monitoring is first performed. When there is a virus risk and the useful information rate of the retrieval output results is too low, they will not be input into the temporary medium and no retrieval record will be formed, thereby increasing the security of the temporary medium and hot word medium and reducing the pollution of the hot word retrieval data by excessive useless information.

[0018] To sum up, in this application, the object data of Internet retrieval is a multi-platform searchable on the Internet. The data volume is huge, and a large amount of processing resources are required during the retrieval process. Therefore, Internet retrieval will only be enabled when the retrieval system is in idle operation and load operation. When the retrieval system is in an overloaded state, in order to reduce the pressure on the retrieval system, only the data in the temporary medium will be enabled for retrieval, which meets the common retrieval needs of most users, increases the high-pressure resistance of the retrieval system, and enables the retrieval platform to maintain the most basic common retrieval functions, which is not easy to cause the retrieval system to crash directly.

[0019] At the same time, the data storage rules in temporary media and hot word media are disclosed, the retrieval rules of the cross-platform data integration retrieval system are confirmed, and security monitoring of data in temporary media and hot word media is carried out to increase the security of temporary media and hot word media, while reducing the pollution of hot word media and hot word retrieval data by excessive useless information. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a schematic diagram of the main structure of the cross-platform data integration and retrieval system of the first embodiment of the present application;

[0021] Figure 2 This is a workflow diagram of a cross-platform data integration and retrieval system according to a first embodiment of the present application;

[0022] Figure 3 A cross-platform data integration and retrieval system according to the first embodiment of the present application;

[0023] Figure 4 A schematic diagram of a data storage method in a temporary medium according to a first embodiment of the present application;

[0024] Figure 5 This is a schematic diagram of the structure of data storage in a temporary medium according to the first embodiment of the present application;

[0025] Figure 6 This is a structural diagram of data storage in a hot word medium according to the first embodiment of the present application.

[0026] Description of the numbers in the figure:

[0027] 1. Search platform server, 2. Internet module, 3. Search terminal, 4. Temporary medium, 5. Hot word medium. DETAILED DESCRIPTION

[0028] Two implementation modes of the present application are described in detail below with reference to the accompanying drawings.

[0029] The first implementation method:

[0030] Figure 1 A cross-platform data integration retrieval system and its medium are shown, including a retrieval platform server 1, the retrieval platform server 1 is signal-connected to an Internet module 2, a temporary medium 4, a hot word medium 5 and a plurality of retrieval terminals 3;

[0031] See also Figure 2-3 , a workflow of a cross-platform data integration retrieval system, including the following steps:

[0032] S1, search input, the user uses the search terminal 3 to input the search requirements according to his own needs;

[0033] S2. Determination of the retrieval system status. The retrieval platform server 1 divides its own status into three categories: idle operation, load operation and overload operation according to its own hardware working status. When the retrieval system is in the idle operation state, the retrieval platform server 1 directly uses the Internet module 2 to perform Internet search on multiple platforms, and outputs the final search results to the retrieval terminal 3. When the retrieval system is in the load operation, the hot word medium 5 is first used to perform hot word search. When the search has results, the final search results are directly output to the retrieval terminal 3. When the hot word search has no results, the Internet search is performed again through the Internet module 2, and finally the final search results are output to the retrieval terminal 3. When the retrieval system is in the overload operation, the hot word medium 5 is used to perform hot word search. When there are results, the detection results are directly output to the retrieval terminal 3. When the hot word search has no results, the retrieval system enters the standby stage and re-determines the retrieval system status every 30S until the retrieval system resumes idle operation or load operation, and then performs Internet search.

[0034] In the present application, the object data of Internet search is a multi-platform searchable on the Internet, and its data volume is huge. A large amount of processing resources are required during the search process. Therefore, Internet search will only be enabled when the search system is in idle operation and load operation. When the search system is in an overloaded state, in order to reduce the pressure on the search system, only the data in the temporary medium 4 will be enabled for retrieval, which meets the common search needs of most users, increases the high-pressure resistance of the search system, and enables the search platform to maintain the most basic common search functions, and is not prone to causing the search system to crash directly.

[0035] The working status of the retrieval system is determined based on overload parameters, which include CPU usage, memory occupancy and network bandwidth utilization. When the overload parameters are all lower than 70%, the retrieval system is determined to be in idle operation. When an overload parameter reaches 70%-95%, the retrieval system is determined to be in load operation. When an overload parameter reaches more than 95%, the retrieval system is determined to be in overload operation.

[0036] When the retrieval system is judged to be overloaded, the retrieval platform server 1 issues an early warning, and the retrieval system enters the manual intervention state. Maintenance personnel intervene in the operation of the retrieval system and generate an early warning form to record the overload parameters of the early warning and eliminate the transportation risks in time. The early warning form facilitates subsequent targeted maintenance work.

[0037] The hot word search data comes from the hot word medium 5. The search output results in the search platform server 1 are stored in the temporary medium 4. The temporary medium 4 filters the high-frequency data that appears in itself and stores it in the hot word medium 5 to form the user's hot word search database, which is convenient for temporary use when the data retrieval system is overloaded.

[0038] The second implementation method:

[0039] Figure 4 The data storage rules in the temporary medium 4 are shown: the results of the search output in the retrieval platform server 1 will be stored in the temporary medium 4 in the order of the search. The total number of data stored in the temporary medium 4 has an upper limit of 10,000. When there are 10,000 searches without duplication, the data is updated normally, and the newly entered data will overwrite the data that has existed in the temporary medium 4 for the longest time. When there are duplications in the search, the duplicate search data will form a new search record, and the old search record will be deleted, forming a temporary search record in the temporary medium 4.

[0040] In particular, the upper limit of data storage in the temporary medium 4 can be designed by technical personnel based on the retrieval frequency of the searching users. When there are a large number of users, the storage upper limit can be increased to avoid some high-frequency search data from being screened out due to too many search samples. Conversely, the storage upper limit can be lowered to increase the effective utilization rate of the temporary medium 4.

[0041] See also Figure 5 The search record in the temporary medium 4 includes search data, labels and times. The search data is the search output result, and the label is the keyword extracted by the search platform server 1 according to the user's search requirements. In particular, when the search output result is retrieved by multiple different users using different search terms, the search platform server 1 can expand the corresponding label for the search data, and the number of times is the number of searches accumulated before the search data is overwritten. When the number of times reaches 1000, the data enters the hot word medium 5 to form a hot word search database.

[0042] Similarly, the actual data of the number of times should also be determined by the search frequency. The search data disclosed in this embodiment is only for one usage scenario. The condition for data to enter the hot word medium 5 to form a hot word search database should not be less than 1% of the total number of searches in the current 10 minutes.

[0043] See also Figure 6 The retrieval records stored in the hot word medium 5 include retrieval data, labels and time, wherein the retrieval data and labels are the same as the data in the temporary medium 4, and the time is the length of time the data exists in the hot word medium 5. When the length of time accumulates to 1 hour, the data is deleted, and when the data enters the hot word medium 5 from the temporary medium 4 again, the time of the data is reset, and when the data is called by the hot word retrieval, the time of the data is also reset.

[0044] Before the search output results are input into the temporary medium 4 by the search platform server 1, data security monitoring is first performed. When there is a virus risk and the useful information rate of the search output results is too low, they will not be input into the temporary medium 4 and no search record will be formed, thereby increasing the security of the temporary medium 4 and the hot word medium 5 and reducing the pollution of the hot word search data of the hot word medium 5 by excessive useless information.

[0045] Compared with the first embodiment, the present embodiment discloses the data storage rules in the temporary medium 4 and the hot word medium 5, confirms the retrieval rules of the cross-platform data integration retrieval system, and performs security monitoring on the data in the temporary medium 4 and the hot word medium 5, thereby increasing the security of the temporary medium 4 and the hot word medium 5 and reducing the pollution of the hot word retrieval data of the hot word medium 5 by excessive useless information.

[0046] In view of current practical needs, the above-mentioned implementation mode adopted in this application is not limited to the scope of protection. Various changes made within the knowledge scope of technical personnel in this field without departing from the concept of this application still fall within the scope of protection of the present invention.

Claims

1. A cross-platform data integration retrieval system, characterized by: It comprises a search platform server (1), wherein the search platform server (1) is signal-connected to an Internet module (2), a temporary medium (4), a hot word medium (5) and a plurality of search terminals (3); Its workflow includes the following steps: S1, search input, the user uses the search terminal (3) to input the search requirements according to his own needs; S2. Determine the state of the retrieval system. The retrieval platform server (1) divides its own state into three categories: idle operation, load operation and overload operation according to its own hardware working state. The working state of the retrieval system is determined based on overload parameters. The overload parameters include CPU usage, memory occupancy and network bandwidth utilization. When the overload parameters are all lower than 70%, the retrieval system is determined to be in idle operation. When an overload parameter reaches 70%-95%, the retrieval system is determined to be in load operation. When an overload parameter reaches more than 95%, the retrieval system is determined to be in overload operation. When the retrieval system is in idle operation, the retrieval platform server (1) directly uses the Internet module (2) to perform Internet search on multiple platforms and outputs the final search results to the retrieval terminal (3). When the retrieval system is in load operation, it first uses The hot word medium (5) performs hot word search, and when the search has results, the final search results are directly output to the search terminal (3). When the hot word search has no results, the Internet search is performed through the Internet module (2), and finally the final search results are output to the search terminal (3); when the search system is in overload operation, the hot word medium (5) is used to perform hot word search, and when there are results, the detection results are directly output to the search terminal (3); when the hot word search has no results, the search system enters a standby stage, and re-determines the state of the search system every 30 seconds until the search system resumes idle operation or load operation, and then performs Internet search, the search output results in the search platform server (1) are stored in the temporary medium (4), and the temporary medium (4) selects the high-frequency data that appears in itself and stores them in the hot word medium (5).

2. A cross-platform data integration retrieval system according to claim 1, characterized in that: When the retrieval system is determined to be overloaded, the retrieval platform server (1) issues a warning, and the retrieval system enters a manual intervention state, with maintenance personnel intervening in the operation of the retrieval system and generating a warning table to record the overload parameters of the warning.

3. A cross-platform data integration and retrieval system according to claim 1, characterized in that: The data storage rule in the temporary medium (4) is as follows: the search results outputted in the search platform server (1) will be stored in the temporary medium (4) in the order of search. The total number of data stored in the temporary medium (4) has an upper limit. When the number of data stored reaches the upper limit, the data is updated normally, and the newly entered data will overwrite the data that has existed in the temporary medium (4) for the longest time. When there are duplicate searches, the duplicate search data will form a new search record, and the old search record will be deleted, forming a temporary search record in the temporary medium (4).

4. A cross-platform data integration retrieval system according to claim 1, characterized in that: The search record in the temporary medium (4) includes search data, labels and times, wherein the search data is the search output result, the labels are keywords extracted by the search platform server (1) according to the user's search requirements, and the times are the accumulated search times before the search data is overwritten. When the times reach a preset number, the data enters the hot word medium (5) to form a hot word search database.

5. A cross-platform data integration and retrieval system according to claim 1, characterized in that: The retrieval record stored in the hot word medium (5) includes retrieval data, a label and a time, and the time is the length of time that the data exists in the hot word medium (5). When the length of time accumulates to a preset length of time, the data is deleted, and when the data enters the hot word medium (5) from the temporary medium (4) again, the time of the data is reset to zero, and when the data is called by a hot word retrieval, the time of the data is also reset to zero.

6. A cross-platform data integration retrieval system according to claim 1, characterized in that: Before the search result output is input into the temporary medium (4) by the search platform server (1), data security monitoring is first performed. When there is a virus risk or the useful information rate of the search output result is too low, it will not be input into the temporary medium (4) and no search record will be generated.

Images