Data processing method and device based on search service

Abstract

The present application provides a data processing method and device based on a search service, the method comprising: the main search system performs data synchronization operations for each business system according to a preset data synchronization cycle; and, when receiving a search request instruction carrying a search keyword from a terminal device, obtains current status information; determines service degradation information according to the current status corresponding to each reference business; in response to the search request instruction, performs corresponding business query operations according to the search keyword and the service degradation information to obtain target search results; and sends a search response instruction carrying the target search results to the terminal device. In this way, the main search system in the server improves the stability of data storage and data processing in the server by separating and storing the business data in the business system and synchronizing them regularly; and performs corresponding service degradation according to the current status of the search service to improve search efficiency and search success rate.

Description

Technical Field

[0001] The present application belongs to the general data processing technology field of the Internet industry, and specifically relates to a data processing method and device based on search services. Background Art

[0002] At present, in order to improve the efficiency and convenience of users in searching and obtaining information, the clients or application platforms provided by the service industries with various businesses usually configure corresponding search systems for users to search.

[0003] However, since it takes a lot of resources for the client or application platform to independently develop an adaptive search system, they usually choose to configure a common search system, which results in the configured search system having high search latency, can only retrieve a single business, and search failures during the search process, reducing the efficiency and stability of the client or application platform and the corresponding server in searching for information. Summary of the invention

[0004] An embodiment of the present application provides a data processing method and device based on a search service, in which a storage engine corresponding to each business system is set up through a main search system in a server to separate and store business data in each business system on a regular and synchronous basis, and to perform corresponding service downgrades according to the current status of the search service, in order to improve the stability and efficiency of data processing for the search service of the server.

[0005] In a first aspect, an embodiment of the present application provides a data processing method based on a search service, which is applied to a main search system in a server, wherein the server further includes a business system of multiple reference businesses, and the server is communicatively connected with a terminal device; the method includes:

[0006] According to the preset data synchronization cycle, a data synchronization operation is performed for each business system respectively, and the data synchronization operation is used to synchronize and store the business data in the business database of each business system to the corresponding storage engine in the main search system. Each business system corresponds to a data synchronization cycle, and the storage engines corresponding to any two business systems are different from each other; and

[0007] When receiving a search request instruction carrying a search keyword from a terminal device, current state information is obtained, where the current state information is used to characterize the current state of the search service of the main search system for each reference business, and the current state includes a normal state and an unavailable state;

[0008] Determine service degradation information according to the current state corresponding to each reference service, where the service degradation information is used to indicate the service to be degraded involved in the search service degradation executed by the server for the current search request instruction, where the service to be degraded is a reference service whose current state is unavailable;

[0009] In response to the search request instruction, performing a corresponding service query operation according to the search keyword and the service degradation information to obtain a target search result;

[0010] Send a search response instruction carrying the target search results to the terminal device.

[0011] In a second aspect, an embodiment of the present application provides a data processing device based on a search service, which is applied to a main search system in a server, wherein the server further includes a business system of multiple reference businesses, and the server is communicatively connected with a terminal device; the device includes:

[0012] A data synchronization unit is used to perform data synchronization operations for each business system according to a preset data synchronization cycle, and the data synchronization operation is used to synchronize and store the business data in the business database of each business system to the corresponding storage engine in the main search system. Each business system corresponds to a data synchronization cycle, and the storage engines corresponding to any two business systems are different from each other; and when a search request instruction carrying a search keyword is received from a terminal device, current status information is obtained, and the current status information is used to characterize the current status of the search service of the main search system for each reference business, and the current status includes a normal state and an unusable state;

[0013] a determining unit, configured to determine service degradation information according to a current state corresponding to each reference service, wherein the service degradation information is used to indicate a service to be degraded involved in the search service degradation executed by the server for the current search request instruction, wherein the service to be degraded is a reference service whose current state is unavailable;

[0014] A response unit, configured to respond to a search request instruction and execute a corresponding service query operation according to the search keyword and the service degradation information to obtain a target search result;

[0015] The sending unit is used to send a search response instruction carrying target search results to the terminal device.

[0016] In a third aspect, an embodiment of the present application provides a server, comprising a processor, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the program includes instructions for executing the steps in the first aspect of the embodiment of the present application.

[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program / instruction stored thereon, which implements the steps in the first aspect of the embodiment of the present application when the computer program / instruction is executed by a processor.

[0018] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement some or all of the steps described in the first aspect of the embodiment of the present application.

[0019] It can be seen that in the embodiment of the present application, the main search system in the server improves the stability of data storage and data processing in the server by storing the business data in the business system separately and synchronizing it regularly; and, according to the current status of the search service, the service is downgraded accordingly to improve the search efficiency and search success rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 is a structural block diagram of an interactive search system provided in an embodiment of the present application;

[0022] Figure 2 It is a flowchart of a data processing method based on a search service provided in an embodiment of the present application;

[0023] Figure 3 It is a scenario diagram of a multi-service search data model provided in an embodiment of the present application;

[0024] Figure 4 is a scene diagram of a search framework provided in an embodiment of the present application;

[0025] Figure 5 It is a scenario diagram of a thread calling model provided in an embodiment of the present application;

[0026] Figure 6 It is a scenario diagram of an abnormal degradation processing model provided in an embodiment of the present application;

[0027] Figure 7 It is a block diagram of the functional units of a data processing device based on a search service provided in an embodiment of the present application;

[0028] Figure 8It is a block diagram of the functional units of another data processing device based on search service provided in an embodiment of the present application;

[0029] Fig. 9 It is a structural block diagram of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0031] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0032] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0033] See also Figure 1 , Figure 1 is a structural block diagram of an interactive search system provided by an embodiment of the present application. Figure 1As shown, the interactive search system 100 includes a server 110 and a terminal device 120, wherein the server 110 includes a main search system 111 and a plurality of business systems 112 for reference business, and the server 110 and the terminal device 120 are in communication connection. Among them, the main search system 111 in the server 110 is provided with a storage engine matching the plurality of business systems 112, and the business system data in the business system 112 is periodically synchronized to the corresponding storage engine. In addition, the terminal device 120 will respond to the user's operation to send a search request instruction carrying a search keyword to the server 110. The main search system 111 in the server 110 will respond to the search request instruction to perform a business query operation according to the search keyword to obtain the target search result, and finally return a search response instruction carrying the target search result to the terminal device 120. An interactive search system 100 corresponds to a server 110, and a server 110 corresponds to one or more terminal devices 120. A server 110 includes a main search system 111 and multiple business systems 112. The number of business systems 112 depends on the number of types of reference businesses.

[0034] Based on this, an embodiment of the present application provides a data processing method based on a search service, and the embodiment of the present application is described in detail below with reference to the accompanying drawings.

[0035] See also Figure 2 , Figure 2 1 is a flow chart of a data processing method based on a search service provided in an embodiment of the present application. The method is applied to a main search system 111 in a server 110 in an interactive search system 100. The server 110 also includes a business system 112 of multiple reference businesses. The server 110 is connected in communication with a terminal device 120. The method includes:

[0036] Step S201: performing data synchronization operations for each business system according to a preset data synchronization cycle.

[0037] The data synchronization operation is used to synchronize the business data in the business database of each business system to the corresponding storage engine in the main search system. Each business system corresponds to a data synchronization cycle, and the storage engines corresponding to any two business systems are different.

[0038] This step is implemented by a multi-business search data model built in the server, wherein each business system in the multi-business search data model corresponds to a business database, a storage engine corresponding to each business system is set in the main search system, and the main search system performs data synchronization operations according to a preset data synchronization cycle to store the business data in the business database into the corresponding storage engine. The types of underlying business data involved in the business system include but are not limited to ticket data, financial data, information data, video live broadcast data, and functional service data.

[0039] Furthermore, due to the large number of types of underlying business data, the principle of setting up separate storage in the multi-business search data model is that the above data are scattered in various business systems, and each business system has large differences in business attributes and system implementation, and cannot provide a unified data search interface for search. In addition, the search request volume is large. If each business system is directly connected, it is easy to cause damage to each business system and affect the transaction of the business itself. Specifically, for the storage of each business, various types of business data are stored separately to ensure that the data do not affect each other and the risks are separated. In addition, considering that the timeliness requirements for search materials are not high, the multi-business search data model will collect business data in the business system according to the preset data synchronization cycle, and decouple the search system from each business system. Among them, due to the different update frequencies of different business data, the data synchronization cycle corresponding to each business system may also be different. The faster the update frequency of business data, the shorter the data synchronization cycle can be set to ensure the timeliness of business data in the main search system.

[0040] Furthermore, the principle of setting any two storage engines in the multi-business search data model to be different from each other is that: considering the different search requirements and characteristics of each business, the data of each business is stored in different underlying storage engines, and no large database with business coupling is used. Specifically: for stock data: it is necessary to support stock code, stock code escape, stock name, full spelling, first spelling, former name, and extended abbreviation for retrieval, and it is necessary to realize personalized retrieval requirements such as uppercase and lowercase recognition, full and half-width recognition, full match, half match, and fuzzy match. After the data is recalled, it is also necessary to perform personalized sorting requirements according to stock attributes and popularity. The storage engines on the market are difficult to meet. Finally, in this example, you can choose to synchronize the stock data to the heap memory, which is more flexible and realizes the above personalized retrieval requirements; for information data: the amount of information material data is huge, and the data volume is in the tens of millions to hundreds of millions. Ordinary storage engines based on hash (an index structure implemented based on hash algorithm) or B+ tree (an index method based on B+ tree data structure) index structure are difficult to meet. As information search is sensitive to time consumption, we adopted a distributed search engine (Elasticsearch, ES) based on inverted index to address the above data characteristics and time consumption requirements. We also used the default word segmenter in the distributed search engine and the fuzzy query at the phrase level to solve the problems of time-consuming queries and unsatisfactory ranking results for information search.

[0041] For example, see Figure 3 , Figure 3 Schematic diagram of a multi-service search data model provided by an embodiment of the present application. Figure 3As shown, taking the server and terminal equipment (equivalent to "client" and "application platform") applicable to the financial securities industry as an example, the business system in the server (also called "peripheral business system") may include a stock business system, an information business system, a video / live broadcast business system, and a function / service business system, and the corresponding reference businesses are stock business, information business, video / live broadcast business, and function / service business. Each business system is provided with a business database for storing the business data of the corresponding reference business, and the business database specifically includes a stock database, an information database, a video database, and a function database. In addition, the main search system is also provided with a storage engine corresponding to each business system, which is used to store the business data synchronized to the main search system. Among them, the data synchronization cycle corresponding to each business data may be the same or different, for example, the data synchronization cycle corresponding to the information business system, the video / live broadcast business system, and the function / service business system is one minute, that is, it is synchronized once every minute, and the data synchronization cycle corresponding to the stock business system is five minutes, that is, it is synchronized once every five minutes. The specific method of data synchronization operation is JOB data synchronization, that is, the scheduled task function of the database is used to realize the synchronization of data between different databases or different parts of the database. This synchronization method is highly flexible and can be executed automatically, and is particularly suitable for scenarios where data needs to be updated regularly. According to the different search requirements and characteristics of each business, the storage engine corresponding to each business system is also different. The storage engine to which stock data is synchronized is a relational database management system (My Structured Query Language, MySQL), which then stores the data in heap memory (a memory area that allows programs to be dynamically allocated and used during operation). The storage engine to which information data and video data are synchronized is a distributed search engine (Elasticsearch, ES), and the storage engine to which function data is synchronized is a MySQL database.

[0042] Step S202: when a search request instruction carrying a search keyword is received from a terminal device, current state information is obtained.

[0043] The current status information is used to represent the current status of the search service of each reference business of the main search system, and the current status includes a normal status and an unavailable status.

[0044] Among them, it should be noted that the search keyword can be a single word, a word or a phrase (including two or more words). Therefore, the search service based on the search keyword may involve business data in a single or multiple business systems. The terminal device provides an interactive interface for the user to type or enter the search keyword by setting a search interface on the client. The terminal device obtains the search keyword of the business data of the related business that the user wants to search based on the interaction with the user. After the user completes the input, the terminal sends a search request instruction to the server to carry the search keyword to request a search response instruction with the target search result. It should be noted that compared with the common application platform or client that can only realize the search service of single business data, the design of the main search system in the server in this application can provide users with search services based on business data corresponding to multiple business systems based on the search keyword, and the search delay is low and the search result return stability is high. Among them, when there is a reference business that is temporarily unavailable, the server can temporarily shut down the search service corresponding to the reference business by modifying the configuration, that is, determining that the current state of the corresponding search service is unavailable. The reasons for the unavailability of the search service of a certain business may involve multiple levels, including technical failures, configuration errors, resource limitations, external dependency issues, security attacks, etc., which are not limited here.

[0045] Step S203: determining service degradation information according to the current state corresponding to each reference service.

[0046] The service degradation information is used to indicate the services to be degraded involved in the search service degradation executed by the server for the current search request instruction, and the services to be degraded are reference services that are currently in an unusable state.

[0047] Among them, in order to ensure that users can effectively obtain the relevant content of the business data they want, the specific business to be downgraded that needs to execute search service downgrade is determined by the reference business that is in an unusable state, so as to avoid setting too many businesses to be downgraded in order to improve search efficiency, which will result in the business data contained in the target search results being less comprehensive and affecting the user's experience of using the search service.

[0048] Step S204: in response to the search request instruction, a corresponding service query operation is performed according to the search keyword and the service degradation information to obtain a target search result.

[0049] Among them, before introducing the specific implementation method of the main search system in the server searching for keywords and service degradation information to perform corresponding business query operations to obtain target search results, it is also necessary to mention the design of the data processing method based on search service provided by this application at the code level.

[0050] Specifically, see Figure 4 , Figure 4 This is a scene diagram of a search framework provided by an embodiment of the present application. When the main search system in the present application calls the search service according to the search keyword, it needs to interact with the above-mentioned storage engines (also called "data sources"), considering that new data sources will be connected later. Figure 4 As shown, the server of this application abstracts a search base class at the code level. The search base class (code represents "Interface <t>") is considered as an abstract layer, usually an abstract class or interface (in languages ​​that support interfaces), which defines the signatures of one or more search methods but does not provide specific implementations of these methods. The purpose of this is to provide a unified interface or framework so that different implementation classes can implement search functions according to the same rules. Taking the financial securities industry as an example, the stock search class (code represented as "Class <stock>"), information search class (code represented as "Class <information>”), video search class (code represented as "Class <video>"), financial search class (code represented as "Class <financial>”), and XXX search class (code represented as "Class <xxx>” are all implementation classes of the search base class. Implementation classes are classes inherited from the search base class (or interface) and provide specific implementations of search methods. Each implementation class may have its own unique search logic, so their outputs (i.e., the return value of the search method) may be different. Considering that the outputs of each search implementation class are different, the output uses the generic type T (i.e. Figure 4 The interface T shown in <t>), the principle of using generic T is: it can improve the reusability and flexibility of the code. Through generics, a general search framework can be written without worrying about the specific return type or modifying the definition of the search base class. Implementation classes can be created for different data types as needed to reduce code duplication and improve the convenience of expanding data sources.

[0051] In one possible embodiment, the server also includes a backup search system, which is connected to the business databases of multiple business systems through multiple data search interfaces, each data search interface corresponding to a business database; in response to a search request instruction, the server performs corresponding business query operations according to the search keywords and service downgrade information to obtain target search results, including: if the downgraded business only includes part of the multiple reference businesses, determining that the current search service is downgraded to the first service downgrade for the main search system; and, performing a first query operation according to the search keywords to obtain the target search results; if the downgraded business includes all of the multiple reference businesses, determining that the current search service is downgraded to the second service downgrade for the terminal device; and, controlling the backup search system to perform a second query operation corresponding to the search keywords to obtain the target search results.

[0052] Among them, the first service downgrade is used to represent that the main search system temporarily shuts down some search services for reference businesses, and the first query operation is used to query multiple storage engines in the search system to obtain the business data indicated by the search request instruction; the second service downgrade is used to represent that the main search system temporarily shuts down all search services for reference businesses, and the second query operation is used to control the backup search system to query the business database through the data search interface to obtain the business data indicated by the search request instruction.

[0053] Among them, in order to improve the stability of the server in performing search services, in this embodiment, the server also includes a backup search system, which can be a commonly used search system on the market. The backup search system is directly connected to the business database in each business system through multiple data search interfaces. The search performance and search capabilities of the backup search system are inferior to those of the main search system, but by setting up a backup search system, the server can be provided with a way to process the remaining data for performing search services (i.e., directly connecting to the business database through a data search interface to search business data), so as to improve the flexibility of the server in performing search services and reduce the probability of server search failure. Between the main search system and the backup search system, the search service that responds to the search request instruction gives priority to using the main search system for search services. In this embodiment, only when the downgraded business includes all reference businesses (i.e., the main search system is completely unavailable), the main search system will control the backup search system to provide users with search services corresponding to the search keywords to obtain target search results.

[0054] It can be seen that in this example, when the downgraded service only includes part of the reference service, the service query operation is performed by the main search system. When the downgraded service includes all the reference services, it means that the main search system cannot perform the current service query operation, and the main search system controls the backup search system to perform service data query, thereby improving the stability of server operation and the flexibility of the server in providing search services, and reducing the probability of the server reporting search failures.

[0055] In one possible embodiment, a first query operation is performed according to a search keyword to obtain a target search result, including: when there are multiple related businesses, determining the number corresponding to the related businesses as the number of threads; calling asynchronous threads with the number of threads, and assigning the first query operation corresponding to each related business to an independent asynchronous thread; and, according to the business attribute information of each related business, performing a query optimization operation on the thread execution unit in the corresponding asynchronous thread to reduce execution time; executing each asynchronous thread according to the business to be downgraded to determine multiple reference search results associated with the search keyword; and integrating the reference search results into the target search results.

[0056] The associated business is a reference business associated with the search keyword, each asynchronous thread corresponds to a thread execution unit, and each reference search result corresponds to an asynchronous thread.

[0057] Specifically, in order to provide search services for multiple related businesses, the main search system can execute the first query operation through a preset thread call model. Figure 5 , Figure 5 This is a scenario diagram of a thread calling model provided by an embodiment of the present application. After the terminal device sends a search request instruction to the server to input a search keyword to the server, the main search system in the server determines multiple related businesses based on the search keyword, and determines the number of threads based on the number of related businesses. Then, the threads corresponding to the number of threads are called to respectively execute the first query operation corresponding to each related business to retrieve the business data in the corresponding storage engine. It should be noted that the threads in the thread calling model are asynchronous threads. Asynchronous threads mean that threads in the program can execute multiple tasks at the same time without waiting for the previous thread to complete execution. The execution process of asynchronous threads is parallel, and each thread can execute different tasks at the same time. In addition, the thread calling model supports horizontal expansion, that is, asynchronous thread n can be expanded to execute the first query operation corresponding to n related businesses, so that the time consumption can be minimized when calling business data in multiple storage engines, thereby improving search efficiency. Combined with Figure 3 In the multi-business search data model shown, it is assumed that the businesses associated with the search-related words include stock business, information business, video / live broadcast business, function / service business, and the expansion business that the server may add later. Figure 5 The first asynchronous thread, the second asynchronous thread, the third asynchronous thread, the fourth asynchronous thread and the nth asynchronous thread will be called to query the storage engine (heap memory) corresponding to the stock business, the storage engine (ES) corresponding to the information business, the storage engine (ES) corresponding to the video / live broadcast business, the storage engine (MySQL database) corresponding to the function / service business and the storage engine corresponding to the expansion business according to the search keywords respectively, and then integrate and return the target search results, that is, by integrating the reference search results returned by each asynchronous thread into the target search results, and then returning the target search results to the terminal device.

[0058] Furthermore, compared with the solution of synchronously calling threads, the corresponding time consumption of the solution of synchronously calling threads is t1+t2+t3+t4+t5. The time consumption of the thread calling model to perform the search operation is: min(500, max(t1, t2, t3, t4, t5)). In the case of business expansion, the calling time consumption can be greatly reduced and the user search experience can be improved. In most cases, when calling multiple threads to perform the first query operation, the execution time of each asynchronous thread will not reach the 500ms downgrade limit. When it is detected that the execution time t of the asynchronous thread is greater than 500ms, the 500ms downgrade setting is executed to terminate the execution of the corresponding asynchronous thread. The purpose of improving performance and reducing the time consumption of a single business is achieved by reducing the time consumption of the highest delay part among t1, t2, t3, t4, and t5. The thread execution unit in each thread in the thread calling model provided in this embodiment has been correspondingly optimized.

[0059] It can be seen that in this example, the main search system executes the first query operation through a preset thread calling model to support asynchronous multi-threading to execute multi-business queries, and supports horizontal expansion, and ensures that the time consumption can be minimized when calling multiple data sources, thereby reducing search latency. Compared with common search systems, it can support multi-business queries and improve the practicality of the server in providing search services.

[0060] In one possible embodiment, based on the business attribute information of each associated business, query optimization operations are performed on the thread execution unit in the corresponding asynchronous thread to reduce execution time, including: obtaining a preset optimization operation set; determining the target query requirements and target data characteristics of the corresponding associated business based on the business attribute information corresponding to the asynchronous thread being processed; determining the reference optimization operation in the optimization operation set that corresponds to the target query requirements and target data characteristics as the query optimization operation; and performing the query optimization operation on the thread execution unit in the asynchronous thread being processed.

[0061] Among them, the optimization operation set includes the correspondence between multiple reference optimization operations and multiple reference query requirements and multiple reference data characteristics. Each reference optimization operation corresponds to a reference query requirement and a reference data characteristic. The reference data characteristic is used to characterize the structural complexity, data magnitude, and number of types of the corresponding business data. The reference optimization operation includes resource configuration operations (that is, according to the memory resources of the server, the memory parameters of the storage engine are reasonably configured, such as buffer size, connection pool size, etc., to improve the performance of storage engine operations) and query optimization operations. The query optimization operation includes at least one of the following operation types: search engine optimization, word segmenter optimization, and query statement optimization.

[0062] Among them, the characteristics of reference data include, but are not limited to, complex data structure, simple data structure, diverse data types, and large data volume. Reference query requirements include, but are not limited to, fast response (i.e., high real-time requirements), accurate query, and full-text retrieval. Generally speaking, according to the corresponding reference query requirements and reference data characteristics, when it is determined that the data processing process needs to process a large amount of data, a simple data structure, and high real-time requirements, the main search system will choose to perform resource allocation operations on the thread execution unit in the asynchronous thread being processed. If it is determined that the data processing process needs to process complex data structures, a large number of data types, and requires accurate queries, the search system will choose to perform query optimization operations on the thread execution unit in the asynchronous thread being processed.

[0063] Exemplarily, the query optimization operations corresponding to the above-mentioned information business may include search engine optimization, word segmenter optimization and query statement optimization, specifically adopting a reasonable search engine, selecting a reasonable word segmenter and a suitable ES query type (writing efficient query statements); and the query optimization operations corresponding to the above-mentioned information business are resource allocation operations, specifically the heap memory database itself is designed to run in memory, so that it can directly search for keywords in the corresponding heap memory, thereby efficiently and accurately completing the search of business data.

[0064] It can be seen that in this example, the main search system performs corresponding reference optimization operations on the thread execution unit in each asynchronous thread according to the query requirements and data characteristics of the specific related business, and combines the synchronous execution of multiple asynchronous threads. The main search system can significantly improve the efficiency and performance of various business query operations, as well as improve the content integrity and comprehensiveness of the target search results returned to the terminal device.

[0065] In one possible embodiment, each asynchronous thread is executed according to the business to be downgraded to determine multiple reference search results associated with the search keyword, including: starting the execution of multiple asynchronous threads; and when an asynchronous thread that triggers a preset abnormal downgrade condition is detected, terminating the execution of the corresponding asynchronous thread and performing an abnormal downgrade operation; after the execution of the multiple asynchronous threads is completed, obtaining the reference search results returned by each asynchronous thread.

[0066] Among them, the abnormal downgrade conditions include a first preset condition, a second preset condition and a third preset condition, and the trigger timing of the first preset condition takes precedence over the trigger timing of the second preset condition, and the trigger timing of the second preset condition takes precedence over the trigger timing of the third preset condition. The first preset condition is that the associated business corresponding to the asynchronous thread belongs to the business to be downgraded, the second preset condition is that a query abnormality event occurs when executing the asynchronous thread, and the third preset condition is that the query duration corresponding to the first query operation in the asynchronous thread exceeds the preset duration. The abnormal downgrade operation is to control the backup search system to execute the second query operation corresponding to the associated business corresponding to the asynchronous thread, or to determine that the corresponding returned reference search result is empty data.

[0067] Specifically, please refer to Figure 6 , Figure 6 Schematic diagram of a scenario of an abnormal degradation processing model provided in an embodiment of the present application. Figure 6 As shown, the main search system in this embodiment is designed with an abnormal degradation processing model to perform multi-level degradation processing. First, the first level is the division of the above-mentioned improved first service degradation and second service degradation. The second service degradation is that the main search system temporarily shuts down all search services for reference services. The second service degradation belongs to the service degradation with a full range of impact, also known as terminal device degradation. When it is detected that the second service degradation is turned on, the main search system controls the backup search system to provide search services for the terminal device. When it is detected that the second service degradation is turned off (that is, the first service degradation is turned on), taking the specific execution steps of the first asynchronous thread and the second asynchronous thread as an example, first, the main search system will provide search services for the terminal device, and this search service is executed by multiple asynchronous threads at the same time for the storage engine of multiple related businesses. Next, the main search system detects whether there is an asynchronous thread that executes the first query operation abnormally by setting the first preset condition, the second preset condition, and the third preset condition in the abnormal degradation condition, and terminates the asynchronous thread that triggers the abnormal degradation condition, and performs an abnormal degradation operation to prevent the asynchronous thread that executes the abnormality from affecting the overall search delay of returning the reference search results, and the stability of the main search system returning the target search results. It can be seen that there are two types of abnormal degradation operations corresponding to the first asynchronous thread and the second asynchronous thread. The first is to directly return empty data (that is, to support the rapid failure of the search service), and the second is to control the backup search system to search. Specifically, the backup search system can support multi-business search services at the same time. If the backup search system cannot support multi-business services, and the number of asynchronous threads that execute the abnormality is greater than or equal to 2, then the subsequent detection of the abnormal degradation operation corresponding to the asynchronous thread that executes the abnormality is to return empty data.

[0068] It can be seen that in this example, the main search system improves the efficiency of the main search system when executing business query operations by setting up multi-level degradation during the execution of asynchronous threads. When there is a query anomaly, it supports fast failure without affecting the overall search data return, reducing the impact of query anomalies on users and improving system stability. In addition, the main search system also supports service degradation of terminal device degradation, switching the main search system to the backup search system can minimize the impact on users and improve the user experience.

[0069] In one possible embodiment, after executing each asynchronous thread according to the service to be downgraded to determine multiple reference search results associated with the search keyword, the method also includes: if the number of results of the reference search results is greater than a preset number, determining the reference data volume of the associated data in each reference search result; sorting the multiple reference search results from high to low according to the real-time levels corresponding to the preset multiple reference services to determine a target sequence; according to the target sequence, performing superposition operations on the reference data volumes of the reference search results in turn to determine the target data volume; and, if it is detected that the superposition operation meets the result sending condition, integrating the reference search results corresponding to the reference data volume included in the target data volume as sub-search results, and resetting the target data volume; sending a search response instruction carrying the sub-search results to the terminal device.

[0070] Among them, the target sequence is used to indicate the arrangement order of multiple reference search results, the real-time level is used to characterize the demand for data timeliness of the reference business for the corresponding business data, and the result sending condition is that the target data volume is greater than or equal to the preset data volume, or the overlay operation is completed.

[0071] Among them, the real-time level can be determined by relevant staff based on the business characteristics of multiple reference businesses, or it can be determined by the main search system directly according to the duration of the preset data synchronization cycle. Specifically, the longer the data synchronization cycle corresponding to the reference business, the lower the real-time level is determined; the shorter the data synchronization cycle corresponding to the reference business, the higher the real-time level is determined. The principle of determining the target sequence according to the real-time level and determining the target data volume according to the target sequence is that when the main search system returns the search results in batches according to the data volume, in order to meet the user's high real-time requirements for special businesses, the reference search results corresponding to the associated business with a high real-time level are prioritized to achieve the division of them into priority batches of sub-search results for return. When the terminal device receives the first batch of search response instructions sent by the main search system, it will display the business information contained in the sub-search results of the first batch according to the sub-search results of the first batch. When the terminal device receives the search response instructions of the subsequent batches, the business data in the sub-search results of the subsequent batches and the business data in the sub-search results of the previous batches will be combined and displayed to the user for viewing.

[0072] Furthermore, the data processing process of this embodiment can also be applied to the scenario where the main search system detects the existence of multiple simultaneous user search request instructions. The reference services can be re-sorted according to their real-time levels based on the associated services involved in a large number of search request instructions, so that the main search system can subsequently give priority to returning reference search results for core associated services to meet user search needs for core associated services, and achieve dynamic resource allocation through the return of graded interleaved results, thereby slowing down resource occupancy of non-core services during peak usage periods and freeing up resources for core services to meet the search services of multiple search request instructions for core associated services.

[0073] It can be seen that in this example, when the main search system returns search results to the terminal device, it can choose to return sub-search results in batches according to the amount of business data in multiple reference search results to improve user experience, avoid the poor user experience caused by long search waiting time, and reduce the amount of data that needs to be processed by the main search system to integrate the search results of multiple related businesses, thereby improving the efficiency of returning search results.

[0074] In one possible embodiment, the server also includes a temporary database; after responding to a search request instruction, performing a corresponding business query operation according to the search keywords and service degradation information to obtain target search results, before sending a search response instruction carrying the target search results to the terminal device, the method also includes: obtaining a keyword set; determining the frequency of occurrence of the search keyword in the keyword set; when the frequency of occurrence is greater than a preset frequency, determining that the search keyword belongs to a hot keyword; and storing the search keyword, the current target search result, and the correspondence between the search keyword and the current target search result in the temporary database; if a search request instruction carrying a hot keyword is received from the terminal device, the temporary database is queried according to the hot keyword to determine the corresponding target search result.

[0075] The keyword set is a set of historical keywords carried by the search request instructions received by the server within a preset time period. Whether the corresponding search keyword belongs to a hot keyword is determined by the frequency of occurrence of the corresponding search keyword in the historical keywords within the preset time period. If the search keyword is determined to be a hot keyword, the corresponding target search result is stored in a temporary database to improve the search response time when the main search system responds to the same keyword.

[0076] Furthermore, in order to prevent the business data included in the target search results from being updated (affecting the real-time requirements of information), or the amount of data stored in the temporary database is large and reduces the search efficiency of the main search system. The main search system can set a preset keyword detection cycle to regularly detect whether the frequency of occurrence of each hot keyword received by the main search system decreases, so as to update the temporary database in time to delete the target search results corresponding to the hot keywords.

[0077] Furthermore, in order to cope with the update of business data with high real-time requirements contained in the target search results, the main search system can synchronously update the content of the target search results in the temporary database according to the data synchronization cycle corresponding to the associated business corresponding to the hot keywords and the hot keywords. The specific process can be: after the main search system performs the data synchronization operation according to the data synchronization cycle, the main search system determines the newly added business data or updated business data corresponding to each business system; according to the hot keywords in the temporary database, and the newly added business data or updated business data, the target search results corresponding to each hot keyword are updated.

[0078] It can be seen that in this example, the main search system caches the query results corresponding to high-frequency search keywords by setting up a temporary database and obtaining search requests within a preset time period, so as to reduce the access pressure of the server for multiple users searching for popular keywords, improve the data processing flexibility of the server in executing search services, and reduce the search delay of multiple users performing repeated searches for popular keywords.

[0079] Step S205: Send a search response instruction carrying the target search results to the terminal device.

[0080] Among them, after receiving the target search results, the terminal device will respond to the search response instruction to display the content in the target search results on the search interface for the user to read the corresponding business information.

[0081] visible, Figure 2 This is a flow chart of a data processing method based on a search service provided in an embodiment of the present application. The main search system in the server sets a storage engine corresponding to each business system to periodically and synchronously separate and store the business data in each business system, and performs corresponding service downgrade according to the current status of the search service, in order to improve the stability and efficiency of the data processing of the server for the search service.

[0082] The following is an embodiment of the device of the present application. The embodiment of the device of the present application and the embodiment of the method of the present application are of the same concept and are used to execute the method described in the embodiment of the present application. For the convenience of explanation, the embodiment of the device of the present application only shows the part related to the embodiment of the device of the present application. For the specific technical details not disclosed, please refer to the description of the embodiment of the method of the present application, which will not be repeated here one by one.

[0083] The present invention provides a data processing device based on a search service. The data processing device based on a search service is applied to Figure 1 The interactive search system 100 shown in the figure includes a main search system 111 in the server 110, the interactive search system 100 also includes a terminal device 120, the server 110 also includes a business system 112 of multiple reference services, and the server 110 is in communication connection with the terminal device 120. Specifically, the data processing device based on the search service is used to execute the steps executed by the server in the above data processing method based on the search service. The data processing device based on the search service provided in the embodiment of the present application may include modules corresponding to the corresponding steps.

[0084] The embodiment of the present application can divide the functional modules of the data processing device based on the search service according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules. The division of modules in the embodiment of the present application is schematic and is only a logical functional division. There may be other division methods in actual implementation.

[0085] In the case of dividing each functional module into corresponding functional modules, Figure 7 1 is a functional unit composition block diagram of a data processing device based on a search service provided in an embodiment of the present application; a data processing device 70 based on a search service is applied to a main search system 111 in a server 110, the server 110 also includes a business system 112 of multiple reference businesses, and the server 110 is in communication with a terminal device 120; the data processing device 70 based on a search service includes: a data synchronization unit 701, which is used to perform data synchronization operations for each business system according to a preset data synchronization cycle, and the data synchronization operation is used to synchronize and store the business data in the business database of each business system to the corresponding storage engine in the main search system, each business system corresponds to a data synchronization cycle, and the storage engines corresponding to any two business systems are different; and, when receiving a data synchronization signal from a terminal device When a search request instruction carrying a search keyword is received, current status information is obtained, and the current status information is used to characterize the current status of the search service of the main search system for each reference business, and the current status includes a normal status and an unusable status; a determination unit 702 is used to determine the service degradation information according to the current status corresponding to each reference business, and the service degradation information is used to indicate the business to be downgraded involved in the search service downgrade executed by the server for the current search request instruction, and the business to be downgraded is a reference business whose current status is an unusable status; a response unit 703 is used to respond to the search request instruction, perform a corresponding business query operation according to the search keyword and the service degradation information, so as to obtain a target search result; a sending unit 704 is used to send a search response instruction carrying a target search result to the terminal device.

[0086] In a possible embodiment, the server also includes a backup search system, which is connected to the business databases of multiple business systems through multiple data search interfaces, and each data search interface corresponds to a business database; in response to the search request instruction, the corresponding business query operation is performed according to the search keywords and the service degradation information to obtain the target search results, and the response unit 703 is specifically used to: if the degraded business only includes part of the multiple reference businesses, determine that the current search service is downgraded to the first service downgrade for the main search system, and the first service downgrade is used to represent that the main search system temporarily shuts down part of the search services for the reference business; and, perform a first query operation according to the search keyword to obtain the target search results, and the first query operation is used to query multiple storage engines in the search system to obtain the business data indicated by the search request instruction; if the degraded business includes all of the multiple reference businesses, determine that the current search service is downgraded to the second service downgrade for the terminal device, and the second service downgrade is used to represent that the main search system temporarily shuts down all search services for the reference business; and, control the backup search system to perform a second query operation corresponding to the search keyword to obtain the target search results, and the second query operation is used to control the backup search system to query the business database through the data search interface to obtain the business data indicated by the search request instruction.

[0087] In one possible embodiment, in terms of executing a first query operation according to a search keyword to obtain a target search result, the response unit 703 is specifically used to: when there are multiple associated businesses, determine the number corresponding to the associated businesses as the number of threads, and the associated businesses are reference businesses associated with the search keywords; call asynchronous threads with the number of threads, and assign the first query operation corresponding to each associated business to an independent asynchronous thread, and each asynchronous thread corresponds to a thread execution unit; and, according to the business attribute information of each associated business, perform a query optimization operation on the thread execution unit in the corresponding asynchronous thread to reduce execution time; execute each asynchronous thread according to the business to be downgraded to determine multiple reference search results associated with the search keyword, and each reference search result corresponds to an asynchronous thread; and integrate the reference search results into the target search results.

[0088] In a possible embodiment, in terms of executing each asynchronous thread according to the business to be downgraded to determine multiple reference search results associated with the search keyword, the response unit 703 is specifically used to: start executing multiple asynchronous threads; and when an asynchronous thread that triggers a preset abnormal downgrade condition is detected, terminate the execution of the corresponding asynchronous thread and perform an abnormal downgrade operation, wherein the abnormal downgrade condition includes a first preset condition, a second preset condition and a third preset condition, and the trigger timing of the first preset condition takes precedence over the trigger timing of the second preset condition, and the trigger timing of the second preset condition takes precedence over the trigger timing of the third preset condition, the first preset condition is that the associated business corresponding to the asynchronous thread belongs to the business to be downgraded, the second preset condition is that a query abnormal event occurs when executing the asynchronous thread, the third preset condition is that the query duration corresponding to the first query operation in the asynchronous thread exceeds the preset duration, and the abnormal downgrade operation is to control the standby search system to execute a second query operation corresponding to the associated business corresponding to the asynchronous thread, or determine that the corresponding returned reference search result is empty data; after the execution of the multiple asynchronous threads is completed, obtain the reference search result returned by each asynchronous thread.

[0089] In one possible embodiment, in terms of executing query optimization operations for thread execution units in corresponding asynchronous threads according to business attribute information of each associated business to reduce execution time, the response unit 703 is specifically used to: obtain a preset optimization operation set, the optimization operation set including a correspondence between multiple reference optimization operations and multiple reference query requirements and multiple reference data characteristics, each reference optimization operation corresponds to a reference query requirement and a reference data characteristic, the reference data characteristic is used to characterize the structural complexity, data level, and number of types of the corresponding business data, the reference optimization operation includes a resource allocation operation and a query optimization operation, the query optimization operation includes at least one of the following operation types: search engine optimization, word segmenter optimization, and query statement optimization; determine the target query requirement and target data characteristic of the corresponding associated business according to the business attribute information corresponding to the asynchronous thread being processed; determine the reference optimization operation corresponding to the target query requirement and target data characteristic in the optimization operation set as the query optimization operation; and execute the query optimization operation for the thread execution unit in the asynchronous thread being processed.

[0090] In a possible embodiment, after executing each asynchronous thread according to the service to be downgraded to determine multiple reference search results associated with the search keyword, the response unit 703 is specifically used to: if the number of results of the reference search results is greater than a preset number, determine the reference data volume of the associated data in each reference search result; sort the multiple reference search results from high to low according to the real-time levels corresponding to the preset multiple reference services to determine a target sequence, the target sequence is used to indicate the arrangement order of the multiple reference search results, and the real-time level is used to characterize the high or low demand of the reference service for the data timeliness of the corresponding service data; according to the target sequence, perform superposition operations on the reference data volumes of the reference search results in turn to determine the target data volume; and, if it is detected that the superposition operation meets the result sending condition, integrate the reference search results corresponding to the reference data volume contained in the target data volume as sub-search results, and reset the target data volume, the result sending condition is that the target data volume is greater than or equal to the preset data volume, or the superposition operation is completed; send a search response instruction carrying the sub-search results to the terminal device.

[0091] In a possible embodiment, the server also includes a temporary database; after responding to a search request instruction, performing a corresponding business query operation according to the search keywords and service degradation information to obtain target search results, and before sending a search response instruction carrying the target search results to the terminal device, the response unit 703 is specifically used to: obtain a keyword set, the keyword set being a set of historical keywords carried by the search request instructions received by the server within a preset time period; determining the frequency of occurrence of the search keyword in the keyword set; when the frequency of occurrence is greater than the preset frequency, determining that the search keyword belongs to a hot keyword; and storing the search keyword, the current target search result, and the correspondence between the search keyword and the current target search result in the temporary database; if a search request instruction carrying a hot keyword is received from the terminal device, querying the temporary database according to the hot keyword to determine the corresponding target search result.

[0092] In the case of integrated units, such as Figure 8 As shown, Figure 8 This is a block diagram of the functional units of another data processing device based on search services provided by an embodiment of the present application. Figure 8 In the embodiment, the data processing device 70 based on the search service includes: a processing module 802 and a communication module 801. The processing module 802 is used to control and manage the actions of the data processing device based on the search service, for example, the steps of the data synchronization unit 701, the determination unit 702, the response unit 703 and the sending unit 704, and / or other processes for executing the technology described herein. The communication module 801 is used to support the interaction between the data processing device based on the search service and other devices. Figure 8 As shown, the data processing device based on the search service may include a storage module 803, and the storage module 803 is used to store program codes and data of the data processing device based on the search service.

[0093] Among them, the processing module 802 can be a processor or a controller, for example, a central processing unit (CPU), a general processor, a digital signal processor (DSP), an ASIC, an FPGA or other programmable logic device, a transistor logic device, a hardware component or any combination thereof. It can implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements a computing function, for example, a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like. The communication module 801 can be a transceiver, an RF circuit or a communication interface, and the like. The storage module 803 can be a memory.

[0094] All relevant contents of each scenario involved in the above method embodiment can be referred to the functional description of the corresponding functional module, and will not be repeated here. Figure 2 The data processing method based on the search service is shown.

[0095] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented using software, the above embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired or wireless means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium may be a solid-state hard disk.

[0096] Fig. 9 is a structural block diagram of a server provided in an embodiment of the present application. Fig. 9 As shown, the server 110 may include one or more of the following components: a processor 910, a memory 920 coupled to the processor 910, wherein the memory 920 may store one or more programs 921, and the one or more programs 921 may be configured to implement the methods described in the above embodiments when executed by one or more processors 910. The server described here is the server 110 in the above embodiments, and the server may implement the main search system 111 and the business system 112 described in the above embodiments, as well as the related methods of the backup search system.

[0097] The processor 910 may include one or more processing cores. The processor 910 uses various interfaces and lines to connect the various parts of the entire server 110, and executes various functions and processes data of the server 110 by running or executing instructions, programs, code sets or instruction sets stored in the memory 920, and calling data stored in the memory 920. Optionally, the processor 910 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 910 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 910, but may be implemented separately through a communication chip.

[0098] The memory 920 may include a random access memory (RAM) or a read-only memory (ROM). The memory 920 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 920 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc. The data storage area may also store data created by the server 110 during use, etc.

[0099] It is understandable that the server 110 may include more or fewer structural elements than those in the above structural block diagram, which is not limited here.

[0100] An embodiment of the present application also provides a computer storage medium, on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, part or all of the steps of any method recorded in the above method embodiments are implemented.

[0101] An embodiment of the present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute part or all of the steps of any method recorded in the above method embodiments.

[0102] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0103] In the several embodiments provided in the present application, it should be understood that the disclosed methods, devices and systems can be implemented in other ways. For example, the device embodiments described above are merely schematic; for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation; for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0104] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0105] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may be physically included separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0106] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform some steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a volatile memory or a non-volatile memory. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM) or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM) and direct RAM bus random access memory (DR RAM), among other media that can store program code.

[0107] Although the present invention is disclosed as above, the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions without departing from the spirit and scope of the present invention, and can make various changes and modifications, including the combination of the above-mentioned different functions and implementation steps, including software and hardware implementation methods, all of which are within the scope of protection of the present invention.< / t> < / xxx> < / financial> < / video> < / information> < / stock> < / t>

Claims

1. A data processing method based on search service, characterized in that: A main search system applied to a server, the server further comprising a plurality of reference business systems and a backup search system, the server being communicatively connected to a terminal device, the backup search system being connected to a plurality of business databases of the business systems via a plurality of data search interfaces, each of the data search interfaces corresponding to a business database; the method comprising: According to a preset data synchronization cycle, a data synchronization operation is performed for each of the business systems respectively, wherein the data synchronization operation is used to synchronously store the business data in the business database of each of the business systems to the corresponding storage engine in the main search system, each of the business systems corresponds to one data synchronization cycle, and the storage engines corresponding to any two of the business systems are different from each other; and When receiving a search request instruction carrying a search keyword from the terminal device, current state information is obtained, where the current state information is used to characterize the current state of the search service of the main search system for each reference service, where the current state includes a normal state and an unavailable state; Determine service degradation information according to the current state corresponding to each of the reference services, wherein the service degradation information is used to indicate the service to be degraded involved in the search service degradation executed by the server for the current search request instruction, and the service to be degraded is the reference service whose current state is unavailable; If the downgraded service only includes part of the multiple reference services, determining that the current search service downgrade is a first service downgrade for the main search system, the first service downgrade is used to represent that the main search system temporarily shuts down the search service for part of the reference services; and, When there are multiple businesses of the associated business, determine the number corresponding to the associated business as the number of threads, and the associated business is the reference business associated with the search keyword; and call the asynchronous threads of the number of threads, and assign the first query operation corresponding to each of the associated businesses to an independent asynchronous thread, and each of the asynchronous threads corresponds to a thread execution unit; and obtain a preset optimization operation set, the optimization operation set includes a correspondence between multiple reference optimization operations and multiple reference query requirements and multiple reference data characteristics, each of the reference optimization operations corresponds to a reference query requirement and a reference data characteristic, and the reference data characteristic is used to characterize the structural complexity, data level, and number of types of the corresponding business data, the reference optimization operation includes a resource configuration operation and a query optimization operation, and the query optimization operation includes at least one of the following operation types: search engine optimization, word segmenter optimization, and query statement optimization; and, according to the business attribute information corresponding to the asynchronous thread being processed, determine the target query requirement and target data characteristic of the corresponding associated business; and determine the reference optimization operation corresponding to the target query requirement and the target data characteristic in the optimization operation set as the query optimization operation; Execute the query optimization operation on the thread execution unit in the asynchronous thread being processed; start executing multiple asynchronous threads; and, when detecting the existence of an asynchronous thread that triggers a preset abnormal downgrade condition, terminate the execution of the corresponding asynchronous thread and perform an abnormal downgrade operation, wherein the abnormal downgrade condition includes a first preset condition, a second preset condition and a third preset condition, and the triggering timing of the first preset condition takes precedence over the triggering timing of the second preset condition, and the triggering timing of the second preset condition takes precedence over the triggering timing of the third preset condition, the first preset condition is that the associated business corresponding to the asynchronous thread belongs to the business to be downgraded, and the second preset condition is that when executing the When an abnormal query event occurs in an asynchronous thread, the third preset condition is that the query duration corresponding to the first query operation in the asynchronous thread exceeds the preset duration, and the abnormal downgrade operation is to control the backup search system to execute a second query operation corresponding to the associated business corresponding to the asynchronous thread, or to determine that the corresponding returned reference search result is empty data, and the second query operation is used to control the backup search system to query the business database through the data search interface to obtain the business data indicated by the search request instruction; after multiple asynchronous threads are executed, obtain the reference search result returned by each asynchronous thread; integrate the reference search results into target search results; If the downgraded service includes all of the multiple reference services, determining that the current search service is downgraded to a second service downgrade for the terminal device, the second service downgrade is used to indicate that the primary search system temporarily shuts down all search services for the reference services; and controlling the backup search system to execute the second query operation corresponding to the search keyword to obtain the target search result; A search response instruction carrying the target search result is sent to the terminal device.

2. The method according to claim 1, characterized in that After determining that the reference optimization operation corresponding to the target query requirement and the target data characteristic in the optimization operation set is a query optimization operation, the method further includes: If the number of results of the reference search results is greater than a preset number, determining the reference data amount of the associated data in each of the reference search results; According to the preset real-time levels corresponding to the multiple reference services, the multiple reference search results are sorted from high to low to determine a target sequence, wherein the target sequence is used to indicate the arrangement order of the multiple reference search results, and the real-time level is used to characterize the level of the demand of the reference service for the data timeliness of the corresponding service data; According to the target sequence, a superposition operation is sequentially performed on the reference data amounts of the reference search results to determine a target data amount; and If it is detected that the superposition operation satisfies the result sending condition, the reference search results corresponding to the reference data amount contained in the target data amount are integrated as sub-search results, and the target data amount is reset. The result sending condition is that the target data amount is greater than or equal to the preset data amount, or the superposition operation is completed. A search response instruction carrying the sub-search result is sent to the terminal device.

3. The method according to claim 1, characterized in that The server also includes a temporary database; after controlling the backup search system to perform the second query operation corresponding to the search keyword to obtain the target search result, before sending the search response instruction carrying the target search result to the terminal device, the method also includes: Acquire a keyword set, where the keyword set is a set of historical keywords carried by the search request instruction received by the server within a preset time period; Determining the frequency of occurrence of the search keyword in the keyword set; When the occurrence frequency is greater than a preset frequency, determining that the search keyword is a hot keyword; and, storing the search keyword, the current target search result, and the corresponding relationship between the search keyword and the current target search result in the temporary database; If a search request instruction carrying the hot keyword is received from the terminal device, the temporary database is queried according to the hot keyword to determine the corresponding target search result.

4. A data processing device based on search service, characterized in that: A main search system applied to a server, the server further comprising a plurality of reference business systems and a backup search system, the server being communicatively connected to a terminal device, the backup search system being connected to a plurality of business databases of the business systems via a plurality of data search interfaces, each of the data search interfaces corresponding to a business database; the device comprising: A data synchronization unit, configured to perform data synchronization operations for each of the business systems according to a preset data synchronization cycle, wherein the data synchronization operation is used to synchronize and store the business data in the business database of each of the business systems to a corresponding storage engine in the main search system, wherein each of the business systems corresponds to one data synchronization cycle, and the storage engines corresponding to any two of the business systems are different from each other; and, when a search request instruction carrying a search keyword is received from the terminal device, current status information is obtained, wherein the current status information is used to characterize the current status of the search service of the main search system for each of the reference businesses, wherein the current status includes a normal status and an unusable status; a determining unit, configured to determine service degradation information according to a current state corresponding to each of the reference services, wherein the service degradation information is used to indicate a service to be degraded involved in the search service degradation executed by the server for the current search request instruction, wherein the service to be degraded is a reference service whose current state is an unusable state; A response unit is used to determine that the current search service is downgraded to a first service downgrade for the main search system if the downgraded business only includes part of the multiple reference businesses, the first service downgrade is used to characterize that the main search system temporarily shuts down part of the search services for the reference businesses; and, when there are multiple associated businesses, determine that the number corresponding to the associated businesses is the number of threads, the associated businesses are reference businesses associated with the search keywords; and, call asynchronous threads with the number of threads, and assign the first query operation corresponding to each of the associated businesses to an independent asynchronous thread, each of the asynchronous threads corresponds to a thread execution unit; and, obtain a preset set of optimization operations, the optimization The optimization operation set includes a correspondence between multiple reference optimization operations and multiple reference query requirements and multiple reference data characteristics. Each of the reference optimization operations corresponds to a reference query requirement and a reference data characteristic. The reference data characteristic is used to characterize the structural complexity, data magnitude, and number of types of the corresponding business data. The reference optimization operation includes a resource allocation operation and a query optimization operation. The query optimization operation includes at least one of the following operation types: search engine optimization, word segmenter optimization, and query statement optimization; and, according to the business attribute information corresponding to the asynchronous thread being processed, determining the target query requirement and target data characteristic of the corresponding associated business; and determining the optimization operation set corresponding to the target query requirement, The reference optimization operation corresponding to the target data characteristic is a query optimization operation; the query optimization operation is executed for the thread execution unit in the asynchronous thread being processed; a plurality of the asynchronous threads are started for execution; and when an asynchronous thread that triggers a preset abnormal downgrade condition is detected, the execution of the corresponding asynchronous thread is terminated, and an abnormal downgrade operation is performed, wherein the abnormal downgrade condition includes a first preset condition, a second preset condition, and a third preset condition, and the triggering timing of the first preset condition takes precedence over the triggering timing of the second preset condition, and the triggering timing of the second preset condition takes precedence over the triggering timing of the third preset condition, and the first preset condition is that the associated business corresponding to the asynchronous thread belongs to the business to be downgraded, The second preset condition is that a query exception event occurs when executing the asynchronous thread, the third preset condition is that the query duration corresponding to the first query operation in the asynchronous thread exceeds the preset duration, the abnormal downgrade operation is to control the backup search system to execute a second query operation corresponding to the associated business corresponding to the asynchronous thread, or to determine that the corresponding returned reference search result is empty data, the second query operation is used to control the backup search system to query the business database through the data search interface to obtain the business data indicated by the search request instruction; after multiple asynchronous threads are executed, obtain the reference search result returned by each asynchronous thread; integrate the reference search results into target search results;If the downgraded service includes all of the multiple reference services, determining that the current search service is downgraded to a second service downgrade for the terminal device, the second service downgrade is used to indicate that the primary search system temporarily shuts down all search services for the reference services; and controlling the backup search system to execute the second query operation corresponding to the search keyword to obtain the target search result; A sending unit is used to send a search response instruction carrying the target search result to the terminal device.

5. A server, characterized in that: The method comprises a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the programs include instructions for executing the steps in the method according to any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that: A computer program for electronic data exchange is stored, wherein the computer program enables a computer to execute the method according to any one of claims 1 to 3.

Images