Information retrieval method and terminal equipment

Abstract

The embodiment of the invention provides an information retrieval method and terminal equipment, and the method comprises the steps: receiving a query statement input by a user, the query statement being used for indicating to query first personal information; performing semantic understanding on the query statement to obtain a first understanding result; the first understanding result comprises a target type to which the first personal information belongs, a target data source corresponding to the target type, entity information contained in the query statement, character information contained in the query statement and a keyword in the query statement, and the target data source is an application in the terminal equipment; and according to the first understanding result, performing retrieval in the target data source to obtain a retrieval result. The method is convenient to retrieve, the memory burden of the user can be relieved, and the user experience is improved.

Description

Technical Field

[0001] This application relates to the field of terminal technology, specifically to an information retrieval method and a terminal device. Background Technology

[0002] With the development of technology, terminal devices can provide users with various intelligent services, such as information retrieval. Currently, most applications have retrieval functions (or search functions), allowing users to enter keywords into the application, which then retrieves information matching those keywords and displays it to the user.

[0003] Information retrieval has an important application scenario: the retrieval of information strongly related to a user's individual identity (referred to as personal information), such as searching for a user's ID number, flight tickets, traffic violation information, and medical examination reports. However, different types of personal information are often stored in different formats in different applications on terminal devices, increasing the user's memory burden and causing inconvenience in information retrieval. For example, an ID number might be stored as an image in an image application, flight ticket information and traffic violation information might be stored as SMS messages in an SMS application, and a medical examination report might be stored as text in a file application; when users need to find specific information, they often don't know which application to search in, leading to inconvenience. Summary of the Invention

[0004] This application provides an information retrieval method and terminal device that facilitates retrieval, reduces the user's memory burden, and improves the user experience.

[0005] In a first aspect, this application provides an information retrieval method executed by a terminal device. The method includes: receiving a query statement input by a user, the query statement indicating a query for first personal information; performing semantic understanding on the query statement to obtain a first understanding result; the first understanding result includes a target type to which the first personal information belongs, a target data source corresponding to the target type, entity information contained in the query statement, person information contained in the query statement, and keywords in the query statement, wherein the target data source is an application in the terminal device; and performing a search in the target data source based on the first understanding result to obtain search results.

[0006] The information retrieval method provided in the first aspect obtains a first understanding result, including target type, target data source, entity information, person information, and keywords, by performing semantic understanding on the query statement. The retrieval is then performed based on this first understanding result. Firstly, this method can determine the target data source for retrieval based on different personal information, eliminating the need for the user to remember data storage paths, thus reducing the user's memory burden, facilitating retrieval, and improving user experience. Secondly, this method can perform semantic understanding on statements, thus supporting retrieval through various query statements, including but not limited to question-and-answer, command-based, and constraint query statements and their combinations. It is easy to use and can achieve accurate retrieval without requiring secondary filtering by the user, further enhancing the user experience.

[0007] In one possible implementation, based on the first understanding result, a search is performed in the target data source to obtain search results, including: if the first understanding result includes referential information, determining the actual information corresponding to the referential information, and performing a search in the target data source based on the first understanding result and the actual information to obtain search results; if the first understanding result does not include referential information, performing a search in the target data source based on the first understanding result to obtain search results.

[0008] In this implementation, by identifying referential information and determining the actual information corresponding to the referential information, the retrieval of referential query statements is realized, which further facilitates user queries, improves retrieval accuracy, and enhances user experience.

[0009] In one possible implementation, the first understanding result also includes a preprocessed statement obtained by preprocessing the query statement. Based on the first understanding result and actual information, a search is performed in the target data source to obtain search results, including: supplementing the preprocessed statement based on the actual information to obtain supplementary statements; searching in the target data source based on the supplementary statements and the first understanding result to obtain candidate results (also called candidate search results); performing relevance filtering and relevance ranking on the candidate results to obtain ranking results; the ranking results represent the order of relevance between the candidate results and the query statement and the first understanding result; and taking the N candidate results with the highest relevance in the ranking results as search results, where N is a positive integer.

[0010] This implementation preprocesses the query statement to facilitate more accurate and comprehensive subsequent retrieval, improving the accuracy of search results and thus enhancing the user experience. Furthermore, by filtering candidate results based on relevance, weakly relevant results are removed, and the candidate results are ranked by relevance, making it easier to select search results based on relevance and improving the accuracy of the search results.

[0011] In one possible implementation, the candidate results are filtered and ranked based on relevance to obtain a ranking result, including: sending a first request (also called a filtering request) to the server; the first request carries supplementary statements, keywords and candidate results, and the first request is used to request the candidate results to be filtered and ranked based on relevance; and receiving the ranking result sent by the server.

[0012] In this implementation, relevance filtering and sorting are performed by the server, which can obtain the sorting results more quickly and reduce the burden on the terminal devices, saving power consumption.

[0013] In one possible implementation, preprocessing includes one or more of the following: simplified / traditional Chinese conversion, phonetic character conversion, initial letter to character conversion, and case conversion.

[0014] In one possible implementation, the method further includes: generating answer text corresponding to the query statement based on the first understanding result and the retrieval result; and outputting the retrieval result and the answer text.

[0015] Optionally, the initial understanding results and search results can be processed using natural language generation (NLG) to generate the answer text corresponding to the query. This allows users to obtain the answer directly without entering the application, improving the convenience and intelligence of the search and further enhancing the user experience.

[0016] In one possible implementation, the terminal device includes a voice assistant application; receiving a query statement input by the user, including: the voice assistant application receiving the query statement input by the user; and outputting search results and answer text, including: the voice assistant application outputting search results and answer text.

[0017] In this implementation, the voice assistant application receives the user's query and displays the search results and answer text to the user. The user can input the query by voice, thus improving intelligence and human-computer interaction, and enhancing the user experience.

[0018] One possible implementation involves outputting search results and answer text, including: integrating the search results and answer text to obtain query results; and outputting the query results.

[0019] This implementation integrates the search results and answer text, making the generated query results easier for users to view and improving the user experience.

[0020] In one possible implementation, the first understanding result also includes a preprocessed statement obtained by preprocessing the query statement. Based on the first understanding result and the retrieval result, an answer text corresponding to the query statement is generated, including: sending a second request (also called an answer generation request) to the server; the second request carries the preprocessed statement and the retrieval result, and the second request is used to request the generation of an answer text corresponding to the preprocessed statement and matching the retrieval result; and receiving the answer text sent by the service.

[0021] In this implementation, the answer text is generated by the server, which can obtain the answer text more quickly and reduce the burden on the terminal device, saving the power consumption of the terminal device.

[0022] In one possible implementation, semantic understanding of the query statement to obtain a first understanding result includes: sending a third request (also known as a semantic understanding request) to the server; the third request carrying the query statement, the third request being used to request semantic understanding of the query statement; and receiving the first understanding result sent by the service.

[0023] In this implementation, semantic understanding is achieved through a server, which can obtain the first understanding result more quickly, reduce the burden on terminal devices, and save power consumption of terminal devices.

[0024] In one possible implementation, the first understanding result is structured data.

[0025] In this implementation, the query statement is understood as structured data, which facilitates retrieval and analysis, simplifies the retrieval process, and further improves retrieval accuracy.

[0026] In one possible implementation, semantic understanding of the query statement is performed to obtain a first understanding result, including: classifying the query statement by intent to obtain an intent classification result; if the intent classification result is a first type (also known as personal information type), then semantic understanding of the query statement is performed to obtain a first understanding result; the first type represents that the intent of the statement is to query personal information.

[0027] In this implementation, the query statement is classified by intent. Only when the intent classification result is personal information will the relevant semantic understanding and personal information retrieval process be executed to obtain results that match the user's query intent, thereby improving the user experience.

[0028] In one possible implementation, the target type is one of the following: ID card, appointment, SMS, calendar, or other types. The target data source corresponding to the ID card type includes photo library application, note application, and file application. The target data source corresponding to the appointment type includes calendar application, SMS application, and photo library application. The target data source corresponding to the SMS type includes SMS application. The target data source corresponding to the calendar type includes calendar application. The target data source corresponding to other types includes photo library application, SMS application, note application, calendar application, and file application.

[0029] It is understandable that different types of personal information exist in different forms and are stored in different paths on terminal devices. Therefore, in this implementation, the target data source corresponding to the type of the first piece of personal information is selected, and subsequent retrieval is performed in the target data source to narrow the search scope and save power consumption of the terminal device.

[0030] Secondly, this application provides an apparatus included in a terminal device, which has the function of implementing the terminal device behaviors described in the first aspect and possible implementations of the first aspect. The function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above functions. For example, a receiving module or unit, a processing module or unit, etc.

[0031] Thirdly, this application provides a terminal device, which includes a processor, a memory, and an interface; the processor, memory, and interface cooperate with each other to enable the terminal device to execute any one of the methods in the technical solution of the first aspect.

[0032] Fourthly, this application provides a chip system including a processor. The processor is configured to read and execute a computer program stored in a memory to perform the methods in the first aspect and any possible implementation thereof.

[0033] Optionally, the chip system may also include memory, which is connected to the processor via circuitry or wires.

[0034] Alternatively, the chip system may also include a communication interface.

[0035] Fifthly, this application provides a server for performing the server-related methods described in the first aspect above.

[0036] Sixthly, this application provides an information retrieval system, including a terminal device and a server, wherein the terminal device is used to execute any one of the methods in the technical solution of the first aspect.

[0037] In a seventh aspect, this application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform any one of the methods in the first aspect of the technical solution.

[0038] Eighthly, this application provides a computer program product, which includes computer program code, which, when run on a terminal device, causes the terminal device to execute any one of the methods in the technical solution of the first aspect. Attached Figure Description

[0039] Figure 1 This is an application scenario diagram of an information retrieval method provided in an embodiment of this application;

[0040] Figure 2 This is another application scenario diagram of the information retrieval method provided in the embodiments of this application;

[0041] Figure 3 This is a schematic diagram of the structure of an information retrieval system provided in an embodiment of this application;

[0042] Figure 4 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application;

[0043] Figure 5 This is an example of a software structure block diagram for the terminal side and the cloud side provided in an embodiment of this application;

[0044] Figure 6 This is a schematic diagram of an example of interface changes provided in an embodiment of this application;

[0045] Figure 7 This is another example of interface change diagram provided in the embodiments of this application;

[0046] Figure 8 This is a schematic diagram of an interface provided in an embodiment of this application;

[0047] Figure 9 This is another example of an interface diagram provided in the embodiments of this application;

[0048] Figure 10 This is a flowchart illustrating an example of an information retrieval method provided in an embodiment of this application;

[0049] Figure 11 This is another example of interface change diagram provided in the embodiments of this application;

[0050] Figure 12 This is another example of interface change diagram provided in the embodiments of this application;

[0051] Figure 13 This is yet another example of an interface diagram provided in the embodiments of this application;

[0052] Figure 14 This is yet another example of an interface diagram provided in the embodiments of this application;

[0053] Figure 15 This is yet another example of an interface diagram provided in the embodiments of this application;

[0054] Figure 16 This is yet another example of an interface diagram provided in the embodiments of this application;

[0055] Figure 17 This is yet another example of an interface diagram provided in the embodiments of this application;

[0056] Figure 18 This is another example of an interface diagram provided in the embodiments of this application. Detailed Implementation

[0057] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in this text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments of this application, "multiple" refers to two or more than two.

[0058] Hereinafter, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first," "second," or "third" may explicitly or implicitly include one or more of that feature.

[0059] References to "one embodiment" or "some embodiments" as described in this application specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this application specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0060] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0061] To better understand the embodiments of this application, the terms or concepts that may be involved in the embodiments are explained below.

[0062] 1. Data source

[0063] A data source refers to the origin of data, the place where data is provided. Data sources can be databases, files, sensors, etc. In this embodiment, the data source can be various applications. For example, the data source can be a gallery application, a file application, a text messaging application, etc.

[0064] 2. Entity

[0065] An entity is a thing or concept that has a clear meaning and is distinct from one another. Entities can include time, personal names, place names, organization names, etc.

[0066] 3. Structured data

[0067] Structured data (also known as structured information, structured data, etc.) refers to data information that has been organized and has a clear structure and format.

[0068] Structured data has the following characteristics:

[0069] 1) Clear format: Structured data is usually stored in relational databases or in tables or documents with specific formats. For example, an employee information table contains fields such as employee number, name, age, and department, each with a clear data type and specific meaning.

[0070] 2) Easy to query and analyze: Due to its clear structure, structured data can be quickly queried, statistically analyzed, and processed using database query languages ​​or data analysis tools. For example, it is easy to find information on all employees in a department or to calculate the number of employees in a specific age group.

[0071] 3) Facilitates data exchange and sharing: Due to the unified structure, different systems and applications can more easily exchange and share structured data. For example, two companies can transmit business data through a standard structured data format.

[0072] 4. Referencing people and places

[0073] Person reference refers to using specific words to represent a person or groups of people. Person reference can be personal pronouns, such as "you," "I," "she," "they," etc., or it can be noun phrases with referential meaning, such as "my son," "this child," etc. It should be noted that, for terminal devices, "I" in person reference refers to the device owner.

[0074] Location reference uses specific words to represent a particular place or places. For example, "here," "home," "company," etc.

[0075] Personal information retrieval is an important usage scenario when users use terminal devices. Generally, when retrieving personal information, users first need to determine from memory which application stores the information they want to retrieve. Then, they open that application and perform the search. For example, Figure 1 This is a schematic diagram illustrating an application scenario of an information retrieval method provided in this application. Taking vehicle traffic violation information as an example, if a user needs to view vehicle traffic violation information, they need to determine from memory that the vehicle traffic violation information is stored in the SMS application in the form of a text message. Then, the user opens the SMS application. Specifically, as... Figure 1 As shown in Figure (a), when the user clicks the SMS application icon 101, the terminal device displays the SMS application interface 102, as follows. Figure 1 As shown in Figure (b), interface 102 includes a search box 103. When the user clicks the search box 103, the terminal device displays the following... Figure 1 The interface 104 is shown in Figure (c). Interface 104 includes a soft keyboard 105. The user then enters the keyword "violation" into the search box using the soft keyboard 105. The terminal device searches based on the user-provided keyword and displays the search results, such as... Figure 1 As shown in Figure (d) in the diagram.

[0076] This retrieval method requires users to remember the storage path of each piece of personal information, which is a heavy memory burden and causes inconvenience to users.

[0077] In one implementation, the terminal device provides a global search function. Users can use a global search application to search for various information across different applications on the terminal device. For example, Figure 2 This is a schematic diagram illustrating another application scenario of the information retrieval method provided in this application embodiment. Taking ID card information as an example, the user can click the search box 202 (the search entry point of the global search application) in the negative one screen 201, and the terminal device displays as follows: Figure 2 Interface 203 is shown in Figure (b). When the user enters the keyword "ID card" in the search box 203, the global search application responds to the user's action, retrieving information related to the ID card from various applications and displaying it as shown in Figure (b). Figure 2 The search results are shown in Figure (c).

[0078] However, the inventors discovered that this retrieval method has the following problems:

[0079] 1) Query limitations. Specifically, global search uses the user-input query as keywords for retrieval. Results are only retrieved when the data source index fully or partially matches the complete keyword. Therefore, if the input query contains content outside the data source index, it cannot be matched, and retrieval cannot be achieved. Thus, this retrieval method cannot be used with question-and-answer, command-type, constraint-type, or referential queries and their combinations. For example, a user inputting "ID card" will get results; however, inputting "What is your ID card number?" (question-and-answer query), "Find my ID card" (command-type query), or "My wife's ID card number" (referential query) will not yield results, making it inconvenient to use.

[0080] 2) Limited search accuracy. Specifically, because this search method cannot be achieved through restrictive or referential queries, relying solely on keyword searches may result in a large amount of data with limited accuracy, requiring users to perform secondary filtering. For example, users cannot find the desired results using the query "my wife's medical examination report in Beijing last year," but can only find multiple report documents using the keyword "medical examination report," and then filter from them for reports with the time "last year," location "Beijing," and person "my wife," making it inconvenient to use.

[0081] 3) The app cannot directly provide the user with the required content; the user needs to enter the application to view the details. For example... Figure 2 As shown in Figure (c), although the search results include photos or text messages related to ID cards, users need to click on the search results to enter the corresponding application to obtain details such as ID card numbers, which is not convenient to use.

[0082] 4) The interaction method is too simplistic and lacks intelligence. This search method primarily relies on users entering keywords via a soft keyboard, resulting in insufficient human-computer interaction and a need for improved user experience.

[0083] In view of this, embodiments of this application provide an information retrieval method. The method classifies the user-input query statement (query) by intent. If the intent classification result is personal information, the method performs semantic understanding on the query statement, extracting the personal information type, constraint entities (time, location, etc.), person information, keywords, and the target data source corresponding to the personal information type, thus obtaining a query understanding result (also called a first understanding result). Furthermore, if the person information or constraint entity in the query statement contains referential information, the method determines the actual information corresponding to the referential information. Then, based on the query understanding result and the actual information corresponding to the referential information, the method retrieves the target data source in the terminal device. This method can determine the target data source required for retrieval, thus eliminating the need for the user to remember the data storage path, reducing the user's memory burden, facilitating retrieval, and improving the user experience. Moreover, this method can perform semantic understanding on the statement, thus supporting retrieval through various query statements, including but not limited to question-and-answer, instruction, constraint, and referential query statements and their combinations. It is easy to use and can achieve accurate retrieval without requiring secondary filtering by the user, further improving the user experience. Optionally, in the method provided in this application embodiment, users can input query statements through a voice assistant, thereby improving intelligence and human-computer interaction, and enhancing user experience. Optionally, the method can also perform natural language generation (NLG) processing on the query statement, query understanding results, and search results to generate answer text corresponding to the query statement, and then integrate the answer text with the search results to display to the user. In this way, users can directly obtain the answer to their question without entering the application, improving the convenience and intelligence of the search, and further enhancing user experience.

[0084] The structure of the device to which the information retrieval method provided in the embodiments of this application is applicable will be described below.

[0085] The information retrieval method provided in this application embodiment can... Figure 3 The information retrieval system shown is implemented. For example... Figure 3 As shown, the system includes a terminal device 100 and a server 200. The server 200 can be a physical server or a cloud server. The structure of the server 200 will not be described here.

[0086] Terminal device 100 can be a mobile phone, tablet computer, wearable device, in-vehicle device, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), or other device that can install applications (APP). This application embodiment does not limit the specific type of terminal device 100.

[0087] For example, Figure 4 This is a schematic diagram of the structure of a terminal device 100 provided in an embodiment of this application. The terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0088] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the terminal device 100. In other embodiments of this application, the terminal device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0089] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0090] The controller can serve as the central nervous system and command center of the terminal device 100. The controller can generate operation control signals based on the instruction opcode and timing signals to control the fetching and execution of instructions.

[0091] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0092] Speaker 170A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. Terminal device 100 can listen to music or hands-free calls through speaker 170A. In this embodiment, the user can listen to answers to questions played by a voice assistant application through speaker 170A.

[0093] Microphone 170C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 170C, inputting sound signals into microphone 170C. In this embodiment, the user can also use receiver 170B to input query statements into a voice assistant.

[0094] Terminal device 100 may be equipped with at least one microphone 170C. In some embodiments, terminal device 100 may be equipped with two microphones 170C, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, terminal device 100 may also be equipped with three, four or more microphones 170C, which can collect sound signals, reduce noise, identify the sound source, and perform directional recording, etc.

[0095] The software architecture of terminal device 100 and server 200 is described below. For ease of description, terminal device 100 will be referred to as the terminal side, and server 200 will be taken as a cloud server, which will be referred to as the cloud side.

[0096] Figure 5 This is a block diagram of the software architecture on the terminal side and the cloud side according to an embodiment of this application. The software system on the terminal side can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This embodiment of the application uses the layered architecture Android system as an example to illustrate the software architecture on the terminal side.

[0097] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer. The application layer can include a series of application packages.

[0098] like Figure 5 As shown, the application package can include applications such as gallery, SMS, notes, files, calendar, voice assistant, intent control, learning center, and local search service. Notes applications refer to applications that enable note-taking functionality; their names are not limited, such as memo, notes, or notepad. File applications refer to applications that enable file management; their names are not limited, such as file manager, document manager, or files. Calendar applications refer to applications that enable schedule recording; their names are not limited, such as schedule or calendar.

[0099] A voice assistant is used to interact with users via voice, receiving and responding to users' voice commands, and engaging in intelligent dialogue and instant question-and-answer sessions. In this embodiment, the voice assistant can receive query statements input by the user via voice or text. Simultaneously, the voice assistant is also used to receive query results transmitted by a local retrieval service and to display and / or output the query results via voice. Optionally, the voice assistant application can be... This application does not limit this to such matters.

[0100] Intent control is used to categorize query intents. Intent categorization results can include personal information queries, general question-and-answer queries (also known as online searches), and command queries. Personal information queries are those with the intent to retrieve personal information, such as "What is my ID number?". General question-and-answer queries are those with the intent to ask questions about non-personal information, such as "What's the weather like today?". Command queries are those with the intent to instruct the terminal to complete a corresponding task, such as "Open the video application." In essence, the intent of a query can represent the user's intent.

[0101] The learning center is used to learn users' personal information based on data from the mobile phone on the terminal side, such as the owner's residence, names, and to build relationships between people. In addition, the learning center can also determine the actual information corresponding to the personal information it learns, for example, determining that "my son" corresponds to the name "Xiaoming", and that "company" corresponds to the name "Company A".

[0102] The local retrieval service, also known as the local data retrieval service, is used to index and build an index of data stored on the terminal side (i.e., local data). In this embodiment, the local retrieval service can also interact with the cloud side, requesting semantic understanding of the query statement from the cloud side, and retrieving the local data based on the results of the cloud side's semantic understanding to obtain candidate retrieval results (also known as candidate results). The local retrieval service is also used to request the cloud side to perform relevance filtering and ranking of the candidate retrieval results. The local retrieval service sends the query statement, query understanding results, and retrieval results to the cloud side, requesting the cloud side to generate the answer text corresponding to the query statement. The local retrieval service is also used to integrate the answer text returned by the cloud side and the retrieval results to generate query results.

[0103] It is understandable that each application in the application layer can implement its functions by calling the underlying modules.

[0104] Of course, in other embodiments, some or all of the modules of the application layer described above may also be located in other layers. For example, the native retrieval service may also be located in the application framework layer.

[0105] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0106] like Figure 5 As shown, the application framework layer may include a window manager, content provider, view system, phone manager, resource manager, notification manager, etc.

[0107] The Android runtime consists of core libraries and a virtual machine. The Android runtime is responsible for scheduling and managing the Android system.

[0108] The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library.

[0109] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0110] System libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0111] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver, camera driver, audio driver, and sensor driver.

[0112] like Figure 5 As shown, the cloud-side implementation may include a semantic understanding module, a relevance filtering module, and an answer generation module. The semantic understanding module performs semantic understanding on the query statement to obtain the query understanding result. The relevance filtering module filters and sorts the candidate search results based on their relevance. The answer generation module generates the answer text corresponding to the query statement based on the input content.

[0113] For ease of understanding, the following embodiments of this application will be described using the following methods: Figure 4 and Figure 5 Taking the terminal device with the structure shown as an example, and in conjunction with the accompanying drawings and application scenarios, the information retrieval method provided in this application embodiment will be specifically described.

[0114] First, the corresponding interface diagram for this method will be explained.

[0115] For example, Figure 6 This is a schematic diagram illustrating an example of interface changes provided in an embodiment of this application. The terminal device is a mobile phone, and the voice assistant is... The following explanation uses a passport number lookup as an example. Figure 6 As shown in Figure (a), the phone displays desktop 601. The user can wake the phone by inputting the wake word "Hello YOYO" into the phone's voice input. After being woken up, the phone displays the following: Figure 6Interface 602 is shown in Figure (b) of the diagram. Meanwhile, It can output voice responses such as "I'm here" or "How can I help you?". Then, the user inputs the query "What is Linda's passport number?" into the phone's voice input system. The phone performs a search based on this query and displays the results on interface 603, such as... Figure 6 As shown in Figure (c). The query results include answer text 6031, information card 6032, and data source 6033. Answer text 6031 can contain the information "Linda's passport number is 17490714". Information card 6032 displays information such as the retrieved passport image. Data source 6033 can contain the information "Content sourced from an image library". Additionally, The answer text 6031 can be output via voice.

[0116] Depend on Figure 6 As can be seen, after a user asks a question, the phone can directly generate an answer corresponding to the user's question. The user doesn't need to filter the information further or navigate to the gallery app to find the answer. Of course, users can also view data details through information card 6032 for further confirmation. Specifically, users can click... Figure 6 Information card 6032 shown in Figure (c) redirects the phone to the gallery app and displays a passport image 604, as shown. Figure 6 As shown in Figure (d) in the diagram.

[0117] For example, Figure 7 This is another example of interface change diagram provided in the embodiments of this application. Continuing with the terminal device as a mobile phone and the voice assistant as... For example, let's take querying flight information as an example to illustrate. (And...) Figure 6 Similarly, such as Figure 7 As shown in Figures (a) and (b), users can wake up using a wake word. Next, the user inputs the query "What day is the flight I booked for my dad departing?" into the phone's voice input system. The phone then performs a search based on this query and displays the results on screen 701, such as... Figure 7 As shown in Figure (c). The query results include answer text 7011, information card 7012, and data source 7013, etc. Answer text 7011 could contain the message "The flight booked for Dad will depart from Beijing Capital Airport T3 to Shanghai Hongqiao Airport T2 on October 1st, 2024 at 15:00." Information card 7012 displays the retrieved booking SMS messages, etc. Data source 7013 could contain the message "Content sourced from SMS." Additionally, It can output the answer text 7011 via voice.

[0118] Optionally, the information card 7012 can display only a portion of the ticket booking SMS message, while the rest can be hidden. Users can click the "Expand" control 7014 to display the full content of the ticket booking SMS message on their phone, such as... Figure 7 As shown in Figure (d) in the diagram.

[0119] Depend on Figure 7 As can be seen, after a user asks a question, the phone can directly generate an answer corresponding to the question. The user doesn't need to filter the information further or switch to the messaging app to get the answer. Of course, users can also switch to the messaging app via message card 7012 to view more data details for further confirmation. Specifically, users can click... Figure 7 As shown in Figure (c) 7012, the mobile phone can redirect to the SMS application and display the ticket booking SMS message, such as... Figure 8 As shown.

[0120] It should be noted that, Figure 6 and Figure 7 This explanation uses voice input for querying on the desktop 601 as an example. In practical applications, query statements can also be input via voice on other interfaces, for example... Figure 9 shown Enter your query statement in the dialog interface 901; there are no restrictions on this.

[0121] in addition, Figure 6 and Figure 7 This example uses user voice input for query statements. In other embodiments, users can also input query statements via text, such as through a soft keyboard. This is not a limitation.

[0122] The method will be explained below with reference to the flowchart.

[0123] Figure 10 This is a flowchart illustrating an example of an information retrieval method provided in an embodiment of this application. The method includes:

[0124] S101, The voice assistant on the terminal side receives the query statement entered by the user.

[0125] In this embodiment, we will use the user's query "Find my wife's medical examination report from Beijing last year" as an example. Here, "physical examination" is the traditional Chinese character for "physical checkup".

[0126] S102, The voice assistant sends the query statement to the intent control center.

[0127] S103. The intent control classifies the query statement into intent categories and obtains the intent classification results.

[0128] Optionally, the intent control center may include an intent classification model. The intent control center inputs a query into the intent classification model, and the intent classification model can output intent classification results. As mentioned above, intent classification results may include personal information categories, general question-and-answer categories, and command categories. In this embodiment, inputting the query "Find my wife's medical examination report from Beijing last year" into the intent classification model results in a classification of "personal information."

[0129] Optionally, the intent classification model can be obtained by training an initial intent classification model. Specifically, query sample statements can be collected and labeled. These query sample statements contain multiple personal information retrieval statements. The query sample statements and their corresponding result labels are then input into the initial intent classification model for training, resulting in the intent classification model.

[0130] S104. If the intent classification result is personal information, the intent control sends a personal information retrieval request to the local retrieval service.

[0131] Optionally, the personal information retrieval request may include a query statement. The personal information retrieval request is used to request the retrieval of personal information based on the query statement.

[0132] It's understandable that after the intent classification model outputs intent classification results, different intent classification results can call different modules to implement them. For example, if the intent classification result is determined to be personal information, a personal information retrieval request is sent to the local retrieval service to perform the personal information retrieval; if the intent classification result is determined to be an instruction, a request is sent to the module in the terminal device used to process instruction statements to execute the task corresponding to the instruction; if the intent classification result is determined to be a general question-and-answer type, a request is sent to the module in the terminal device used to process general question-and-answer statements to answer non-personal information questions.

[0133] S105. In response to a personal information retrieval request, the local retrieval service sends a semantic understanding request to the semantic understanding module on the cloud side.

[0134] Optionally, the semantic understanding request may include a query statement. A semantic understanding request is used to request semantic understanding of the query statement.

[0135] S106. The semantic understanding module on the cloud side responds to the semantic understanding request (also known as the third request), performs semantic understanding on the query statement, and obtains the query understanding result (also known as the first understanding result).

[0136] Optionally, the semantic understanding module can understand the query statement as preset structured data. That is, the semantic understanding module performs semantic understanding on the query statement and outputs the understanding result in the form of preset structured data. In one embodiment, the data members of the preset structured data may include: the preprocessed statement, personal information type, constrained entity (also known as entity information), person information, keywords, the target data source corresponding to the personal information type, word segmentation, synonyms of the keywords, etc., one or more of them. In other words, the query understanding result includes: the preprocessed statement, personal information type, constrained entity, person information, keywords, the target data source corresponding to the personal information type, word segmentation, synonyms of the keywords, etc., one or more of them.

[0137] Among them, the preprocessed statement refers to the statement obtained after preprocessing the query statement. Optionally, the preprocessing may include, but is not limited to, traditional and simplified Chinese conversion, pinyin-to-character conversion, initials-to-words conversion, case conversion, etc. The traditional and simplified Chinese conversion mainly refers to converting traditional Chinese characters to simplified Chinese characters. For example, converting "還原" to "还原". The pinyin-to-character conversion mainly refers to converting pinyin to characters. For example, converting "Zhaopian" to "照片". The initials-to-words conversion means converting common initials into corresponding words. For example, converting to The case conversion refers to converting English capital letters to lowercase letters. For example, converting "TIME" to "time".

[0138] Optionally, the semantic understanding module may include a pre-trained preprocessing model. The query statement is input into the preprocessing model, and the preprocessing model preprocesses the query statement and outputs the corresponding preprocessed statement.

[0139] Specifically, in the example of this embodiment, after preprocessing the query statement "Find my wife's physical examination report in Beijing last year", the obtained preprocessed statement is "Find my wife's physical examination report in Beijing last year".

[0140] By preprocessing the query statement, it is convenient for subsequent more accurate and comprehensive retrieval, improving the accuracy of the retrieval result and thus enhancing the user experience.

[0141] The personal information type refers to the category to which the personal information to be queried understood from the query statement belongs. Optionally, the personal information type may include document type, reservation type, SMS type, schedule type, other type, etc. For the information included in each personal information type, please refer to Table 1. It should be noted that Table 1 is only an example and is not used for limitation. In actual applications, the information included in various information types can be limited according to requirements.

[0142] Table 1

[0143]

[0144] Optionally, the semantic understanding module may include a pre-trained information classification model. Inputting the pre-processed statement into the information classification model will output the corresponding personal information type.

[0145] In the specific example of this embodiment, the preprocessed statement "Find my wife's medical examination report from Beijing last year" is input into the information classification model, and the output personal information type is "document type".

[0146] Constraint entities refer to entity information contained in the query statement, such as time and location. Constraint entities can be used to constrain the scope of subsequent retrieval. Optionally, the semantic understanding module may include a pre-trained entity recognition model. Inputting the preprocessed statement into the entity recognition model will output the constraint entities within the preprocessed statement.

[0147] Specifically, in the example of this embodiment, the preprocessed statement "Find my wife's medical examination report from Beijing last year" is input into the entity recognition model, and the output constraint entities include: time: 2023, location: Beijing.

[0148] Person information refers to information indicating persons contained in the query statement. Optionally, person information can be actual names or person designations, such as "my son" or "my wife." Person information can be used by the user to subsequently constrain the search scope. Optionally, the semantic understanding module can include a pre-trained person information extraction model. Inputting the preprocessed statement into the person information extraction model can output the person information within the preprocessed statement.

[0149] In this embodiment, the preprocessed statement "Find my wife's medical examination report from Beijing last year" is input into the person information extraction model, and the obtained person information is "my wife".

[0150] Keywords refer to key words or phrases contained in a query statement. In this embodiment, for queries involving personal information, keywords are generally words or phrases in the query statement that represent personal information. Optionally, the semantic understanding module may include a pre-trained keyword extraction model. Inputting the pre-processed statement into the keyword extraction model can output the keywords in the pre-processed statement.

[0151] In this embodiment, the preprocessed statement "Find my wife's medical examination report from Beijing last year" is input into the keyword extraction model, and the keyword obtained is "medical examination report".

[0152] It is understandable that different types of personal information exist in different forms and are stored in different paths on terminal devices. Therefore, the semantic understanding module can determine the corresponding target data source based on the type of personal information, and then perform a search within the target data source to narrow the search scope and save power consumption of the terminal device.

[0153] Optionally, the semantic understanding module may also include a data source mapping module. The data source mapping module is used to determine the target data source corresponding to the personal information type based on a preset mapping relationship.

[0154] In one embodiment, the mapping relationship between personal information types and target data sources is shown in Table 2 below. It should be understood that Table 2 is merely an example and not intended to limit the scope. In practical applications, depending on the form and location of the data stored on the terminal side, other correspondences may exist between personal information types and target data sources.

[0155] Table 2

[0156] Serial Number Personal Information Types Target data source 1 Certificates Gallery app, note-taking app, file app 2 Reservation Calendar app, SMS app, photo gallery app 3 SMS SMS application 4 Schedule Calendar application 5 Other categories Gallery app, SMS app, Notes app, Calendar app, Files app

[0157] In this embodiment, the personal information type obtained by classifying the preprocessed statement "Find my wife's medical examination report in Beijing last year" is "ID document type". Therefore, according to Table 2, the corresponding target data source is determined to be "photo library application, note application and file application".

[0158] As one possible implementation, the amount of data retrieved can be limited to reduce the computational load of subsequent processing. Based on this, the mapping relationship between personal information types and target data sources can further include a retrieval quantity threshold corresponding one-to-one with the personal information type and the target data source. For example, the retrieval quantity threshold for ID card-related and photo gallery applications could be 10, while the threshold for ID card-related and note-taking applications could be 5. During subsequent searches, for ID card-related personal information, a maximum of 10 sets of data could be retrieved from the photo gallery application, and a maximum of 5 sets of data from the note-taking application.

[0159] Word segmentation refers to the result of splitting a preprocessed sentence into words. Optionally, the semantic understanding module may include a pre-defined word segmentation model. The preprocessed sentence is input into the word segmentation model, which then performs word segmentation to obtain the segmented words.

[0160] Specifically, in the example of this embodiment, the word segmentation of the preprocessed sentence "Find my wife's medical examination report from last year in Beijing" is: find, my wife, last year, in Beijing, of, medical examination report.

[0161] The semantic understanding module may include a synonym processing unit. This unit is used to determine synonyms for a given word. In this embodiment, the synonym processing unit expands the meaning of keywords to obtain their synonyms. During subsequent searches, combining information such as word segmentation and synonyms improves the comprehensiveness and accuracy of the search results, thereby enhancing the user experience.

[0162] Specifically, in the example of this embodiment, synonyms for the keyword "physical examination report" may include: physical examination results, health check report, physical examination report, physical examination report, etc.

[0163] In summary, the semantic understanding module performs semantic understanding on the query "Find my wife's medical examination report from Beijing last year," and the resulting structured query understanding results can include:

[0164] Preprocessed statement: Find my wife's medical examination report from Beijing last year;

[0165] Personal information type: Card / certificate information;

[0166] Constraint Entity: Time: 2023, Location: Beijing;

[0167] Person information: My wife;

[0168] Keywords: medical examination report;

[0169] Target data sources: gallery apps, note-taking apps, file apps;

[0170] Word segmentation: find, my wife, last year, in Beijing, of, medical examination report;

[0171] Synonyms for keywords: physical examination results, health check report, physical examination report, physical examination report.

[0172] S107. The semantic understanding module on the cloud side returns the query understanding results to the local retrieval service on the terminal side.

[0173] S108. The local retrieval service on the terminal side determines whether there is referential information in the query understanding results; if yes, proceed to step S109; if no, proceed to step S113.

[0174] Optionally, the local retrieval service can determine whether there are person references and location references (collectively referred to as reference information) in the query understanding results. If person references or location references exist, steps S109 to S112 are executed to obtain the actual information corresponding to the reference information from the learning center, and then a retrieval is performed. If there are no person references or location references, step S113 is executed to perform a direct retrieval.

[0175] In the specific example of this embodiment, the query understanding result contains a person referring to "my wife", so steps S109 to S112 are executed.

[0176] S109. The local search service will send the referencing information to the learning center.

[0177] S110. The learning center determines the actual information corresponding to the referential information.

[0178] Specifically, the learning center can determine the actual information corresponding to the referent information based on the personal information such as relationships, names, and places of residence that it has learned in advance.

[0179] Here is a brief explanation of the learning center's process for learning about character relationships, names, and resident figures:

[0180] 1) Learn about relationships between people and names

[0181] In one embodiment, the learning center can learn the name of the device owner through the account information logged in by the relevant application on the terminal device. For example, through... account, Account or Real-name authentication in accounts and other documents confirms the owner's name.

[0182] Additionally, gallery apps can perform face clustering on images containing people (also known as portraits), grouping each person into a separate category. Users can then label people with relationships or names within the gallery app. For example, Figure 11 This is another example of an interface diagram provided for an embodiment of this application. For example, a user can... Figure 11 In the interface 1101 shown, the people in the cluster are marked, that is, the relationship between the person and the owner is added, or the person's name, etc.

[0183] The learning center can build relationships between characters based on user tags. For example, based on... Figure 11 The learning center can determine the markings. Figure 11 Character 1102 in the story is the owner's wife. Figure 11 The marker indicates that the name of person 1103 is Xiaoming.

[0184] Furthermore, the images in the people cluster are identified. If the image contains documents such as ID cards, driver's licenses, or passes, the name on the document is extracted and a correspondence is established between the name and the person. For example, if the name "Wang Hua" is identified from the ID card image in the "wife" cluster, then the name of the "wife" is determined to be "Wang Hua".

[0185] 2) Study location

[0186] It is understandable that if a terminal device stays at the same location for a set period of time, that location can be considered its permanent address. In one embodiment, the learning center can learn the permanent address by acquiring the signaling data of the terminal device. It is understood that when a terminal device powers on / off, makes a call, sends a text message, moves its location, or switches base stations, it records corresponding signaling data. Therefore, by analyzing the signaling data over a period of time, the terminal device's locations at various times within a unit of time (e.g., each day) can be determined. Analyzing these locations yields the user's permanent address, such as the location of their home or workplace. For example, a learning center can obtain signaling data from terminal devices every day from 00:00 to 06:00 within the past month. Based on this signaling data, it can determine the dwell points of the terminal devices during this period. For each dwell point, nearest neighbor clustering is used to merge them, resulting in the corresponding target dwell point for each day. Based on nearest neighbor clustering, identical target dwell points within the past month are clustered into a single cluster. For each cluster, if the number of target dwell points in the cluster meets a set requirement (e.g., greater than 20), that target dwell point is designated as a preset permanent location (e.g., home). Similar methods can be used to obtain the company's location, etc., which will not be elaborated upon here.

[0187] In addition, based on the company's location, combined with text information extracted from images in a learning image library, or attendance records from terminal devices, the company's name and other information can be determined.

[0188] In this embodiment, it is assumed that the person identified by the learning center as "my wife" is actually named "Wang Hua".

[0189] S111, The learning center sends the actual information to the local retrieval service.

[0190] S112. The local retrieval service supplements the preprocessed statements based on the actual information to generate supplementary statements.

[0191] Optionally, the actual information corresponding to the referential information can be added to the preprocessed statement. For example, if the name corresponding to "I" is "Wang Xiaoxiao", then "My name is Wang Xiaoxiao" can be added; if the company name is A, then "The company name is A" can be added.

[0192] Specifically, in the example of this embodiment, the preprocessed statement "Find my wife's medical examination report in Beijing last year" is supplemented to generate the supplemented statement: "Find my wife's medical examination report in Beijing last year, my wife is Wang Hua".

[0193] S113. The local retrieval service retrieves within the target data source included in the query understanding result according to the supplementary statement and the query understanding result, and obtains candidate retrieval results.

[0194] Specifically, the local retrieval service can use the keywords in the query understanding result and the corresponding limited content as the main conditions for retrieval, and use the constraint entities, people, etc. in the query understanding result as the constraint conditions for retrieval, and perform retrieval in the target data source. Specifically, the time information in the constraint entity is used as the time constraint, the location information in the constraint entity is used as the location constraint, and the person information is used as the person constraint. Among them, the limited content corresponding to the keyword refers to the content used to modify the keyword except for time and location limitations, generally words or phrases that limit the ownership relationship of the keyword. For example, person information used to limit the ownership relationship of the keyword. When the limited content corresponding to the keyword is person information limitation, if the person information is a reference information, the reference information is replaced with the actual information. For example, in the example of this embodiment, the limited content corresponding to the keyword "physical examination report" is "my wife", and "my wife" is reference information, and the corresponding actual information is "Wang Hua". Therefore, the main condition for retrieval can be "Wang Hua + physical examination report".

[0195] It should be noted that the time constraint, location constraint, and person constraint are optional conditions, not necessary conditions, and can be specifically determined according to the target data source and the storage form of the data. For example, for the location constraint, it can be used to constrain the generation location of the picture, rather than the generation location of the file.

[0196] Optionally, when the target data source is a gallery application, the optical character recognition (OCR) text of the pictures stored in the gallery can be retrieved. The OCR text of the picture can be OCRed and stored by the gallery application when the terminal device is idle.

[0197] Specifically, the retrieval conditions in the example of this embodiment can be listed as follows:

[0198] Data source: Gallery application, file application;

[0199] Main condition: The OCR text of the picture, the content of the file, or the note content contains "Wang Hua" and contains "physical examination report";

[0200] Time constraint: The generation time of the picture is 2023, the modification time of the file is 2023, and the modification time of the note is 2023;

[0201] Location constraint: The shooting location of the picture is "Beijing";

[0202] Person Restriction: The person in the picture is labeled with the name "Wang Hua" or "Wife".

[0203] Based on the above search criteria, the candidate search results are assumed to include one image and one PDF file named "Beijing A Medical Examination Center - Wang Hua Medical Examination Report.pdf".

[0204] S114. The local search service sends a filtering request (also known as the first request) to the relevance filtering module on the cloud side. The filtering request carries supplementary statements, keywords and person information in the query understanding results, and candidate search results.

[0205] Person information can be optional. In some other embodiments, the filtering request may not include person information. Including person information in the filtering request can improve the accuracy of filtering and sorting results.

[0206] It should be noted that if the person information in the query result is a person reference, then the filtering request should carry the actual name (not the person reference) corresponding to the person information.

[0207] The filtering request is used to request relevance filtering and ranking of candidate search results. Optionally, the local search service can encrypt the candidate search results before sending them to the cloud along with the filtering request to protect user privacy. It's understandable that sending other user information to the cloud would follow the same principle, so we won't elaborate further.

[0208] Specifically, in the example of this embodiment, the filtering request carries the preprocessed statement "Find my wife's physical examination report from Beijing last year", as well as the candidate search results obtained in step S113, namely, one image and one PDF file named "Beijing A Physical Examination Center - Wang Hua Physical Examination Report.pdf".

[0209] S115. The cloud-side relevance filtering module responds to the filtering request, performs relevance filtering and sorting on the candidate search results, and obtains the sorted results.

[0210] S116. The cloud-side relevance filtering module returns the sorting results to the local retrieval service on the terminal side.

[0211] Optionally, the relevance filtering module includes a filtering model. The filtering model can determine the relevance (or relevance) of each candidate search result to the keywords, person information, and supplementary statements, and filter out candidate search results that do not meet the relevance requirements. Then, the candidate search results that meet the relevance requirements are sorted from strong to weak (and from largest to smallest) according to relevance to obtain the sorting results.

[0212] In this specific example, the candidate search results—namely, one image and one PDF file named "Beijing A Medical Examination Center - Wang Hua's Medical Examination Report.pdf," the supplementary statement "Find my wife's medical examination report from Beijing last year; my wife is Wang Hua," the keyword "medical examination report," and the person information "Wang Hua"—are input into the filtering model. The filtering model removes results that do not meet the relevance requirements and returns the results that do meet the relevance requirements to the local search service in descending order of relevance. Specifically, both the image and the PDF file meet the relevance requirements; the PDF file has a stronger relevance than the image. Therefore, the results are output to the local search service in the order of PDF file first, then image.

[0213] By filtering based on relevance, candidate search results with weak relevance are filtered out, and the candidate search results are sorted by relevance. This makes it easier to select search results from the candidate results based on relevance in the future, thereby improving the accuracy of the search results.

[0214] S117. The local retrieval service on the terminal side sends an answer generation request (also known as the second request) to the answer generation module on the cloud side. The answer generation request carries the query understanding result (including the preprocessed statement) and the retrieval result, wherein the retrieval result is the TOP N candidate retrieval result in the ranking result.

[0215] The answer generation request is used to request the generation of answer text that corresponds to the preprocessed statement (i.e., the query statement) and matches the search results. In other words, the answer text is the text used to respond to the query statement, and the response content matches the search results.

[0216] N is a positive integer, and N can be set according to actual needs. In a specific embodiment, N can be 1. That is, the candidate search result with the strongest relevance in the ranking results is taken as the final search result. Specifically, in the example of this embodiment, the PDF file named "Beijing A Medical Examination Center - Wang Hua Medical Examination Report.pdf" is determined as the final search result.

[0217] It should be noted that the query understanding results, excluding the preprocessed statements, can be optional content carried in the answer generation request. That is, in some embodiments, the answer generation request may only include the preprocessed statements and the search results. Including all query understanding results in the answer generation request results in a more accurate answer text generated by the cloud side based on those results.

[0218] S118. The cloud-based answer generation module generates answer text corresponding to the query statement and search results based on the search results and query understanding results.

[0219] Optionally, the answer generation module can be a pre-trained NLG model, which takes the retrieval results and query understanding results as inputs to the NLG model and outputs the answer text.

[0220] Specifically, in the example of this embodiment, the query understanding result (which includes preprocessed statements) and the PDF file are input into the NLG model, and the generated answer text can be: The following is your wife's physical examination report from last year in Beijing.

[0221] S119. The cloud-side answer generation module returns the answer text to the local retrieval service on the terminal side.

[0222] S120. The local retrieval service on the terminal side integrates the answer text and retrieval results to obtain the query results.

[0223] Optionally, the integration of answer text and search results may include, but is not limited to: 1) combining answer text and search results; 2) fine-tuning the answer text; for example, adding the name of the voice assistant (such as "YOYO") to the answer text; 3) adding data source information; for example, adding the text "Data source: SMS"; 4) determining the display format of the answer text, search results, and data sources; for example, when the search results include multiple items, each search result and its corresponding data source can be displayed in a list format, or key information can be extracted from each search result to form an answer list.

[0224] Specifically, in the example of this embodiment, the answer text can be fine-tuned. The fine-tuned answer text can be: The following is your wife's physical examination report in Beijing last year, which YOYO found. Combine the fine-tuned answer text with a PDF file named "Beijing A Physical Examination Center - Wang Hua Physical Examination Report.pdf" and the data source "Content comes from file" to obtain the final query result.

[0225] S121. The local search service returns the query results to the voice assistant.

[0226] S122, The voice assistant displays the query results.

[0227] Optionally, when the voice assistant displays the query results, it can display the search results in the form of cards, which are called information cards.

[0228] Specifically, in the example of this embodiment, the query results can be displayed as follows: Figure 12Interface 1201 in Figure (a) shows the query results, which include answer text 1211, information card 1212, and data source 1213. Answer text 1211 reads, "The following is your wife's medical examination report from last year in Beijing, found by YOYO." Information card 1212 displays a PDF file named "Beijing A Medical Examination Center - Wang Hua Medical Examination Report.pdf". Data source 1213 states, "Content originates from a file."

[0229] Optionally, in response to a user clicking information card 1212, the terminal device can jump to the file application and display the content interface 1202 of the PDF file, such as... Figure 12 As shown in Figure (b) of the document.

[0230] The information retrieval method provided in this application can intelligently retrieve personal information without requiring users to remember data storage paths, reducing the user's memory burden. It also eliminates the need for secondary filtering, making it convenient for users and offering high human-computer interaction, thus improving the user experience. Furthermore, this method can perform semantic understanding of statements, supporting retrieval through various query statements, including but not limited to question-and-answer, instruction, constraint, and referential queries and their combinations. It is easy to use and can achieve accurate retrieval. In addition, by interacting with the cloud to achieve semantic understanding, relevance filtering and sorting, and the generation of answer text, the required data can be obtained more quickly, reducing the burden on the terminal side and saving power consumption of terminal devices. Moreover, in this method, the cloud side interprets the query statement as specified structured data, facilitating retrieval and analysis, simplifying the retrieval process, and further improving retrieval accuracy.

[0231] The above embodiments illustrate the method using a medical examination report query scenario as an example. Below, we list the query statements, data sources, retrieval results, and query outcomes for several other application scenarios, as shown in Table 3.

[0232] Table 3

[0233]

[0234]

[0235] The following is a brief explanation of the process of building an index and performing a search on the local search service.

[0236] During use, terminal devices can generate a large amount of searchable content through various applications, and generate corresponding indexes from this large amount of searchable content according to index templates, recording the indexes in index information. Therefore, a large number of indexes can be stored in the index information. To facilitate index management and searching, in some cases, the terminal device can divide the index information according to vertical domains, obtaining K index information fragments. One index information fragment corresponds to one vertical domain. Alternatively, one searchable content can correspond to one vertical domain, and when recording the index corresponding to the searchable content, the index corresponding to the searchable content can be recorded in the index information fragment corresponding to that vertical domain (corresponding to the searchable content). The vertical domain can be used to describe the type of searchable content. It can also be understood that the vertical domain corresponding to a searchable content can be used to indicate the application from which the searchable content comes, or which index information fragment to record it in. The number of vertical domains and the method of dividing vertical domains can be determined according to actual usage and should not constitute a limitation on the embodiments of this application.

[0237] In one embodiment, the local search service can perform a search according to the following process:

[0238] 1) The local retrieval service determines the N1 searchable vertical domains corresponding to the target data source.

[0239] The N1 searchable verticals can be represented as the names of N1 index information fragments. When index information is recorded in the index information file, an index information fragment can be regarded as an index information sub-file, and the N1 searchable verticals can be represented as the names of N1 index information sub-files.

[0240] 2) Determine the M1 searchable fields and their corresponding matching methods for the preprocessed statement based on the N1 searchable verticals.

[0241] The M1 searchable fields are the searchable fields in the index information shards corresponding to the N1 searchable vertical domains. Specifically, the searchable fields in an index information shard corresponding to a searchable vertical domain include fields whose corresponding index is not null. If the index corresponding to a field is not null, then the field value and its extended value are recorded in the index information.

[0242] It should be understood that, generally speaking, fields and matching methods correspond to each other. Optionally, the matching method can be one or more of the following: token matching (including token AND matching, token OR matching), synonym matching, prefix matching, and exact matching.

[0243] 3) The local retrieval service searches within N1 index information shards based on the preprocessed statement, query understanding results, and matching methods to determine the documents in each index information shard that successfully match the query content. Different index information shards correspond to different searchable verticals in N1 searchable verticals.

[0244] 4) The local retrieval service takes the successfully matched documents in each index information shard as candidate retrieval results, and aggregates each candidate retrieval result according to the vertical domain to obtain N2 candidate retrieval results corresponding to the searchable vertical domain.

[0245] The foregoing has detailed examples of the information retrieval methods provided in the embodiments of this application. It is understood that, in order to implement the above functions, the terminal device includes hardware and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in conjunction with the embodiments, but such implementation should not be considered beyond the scope of this application.

[0246] This application embodiment can divide the terminal device into functional modules according to the above method example. For example, each function can be divided into a separate functional module, such as a detection unit, a processing unit, a display unit, etc., or two or more functions can be integrated into one module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0247] It should be noted that all relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and will not be repeated here.

[0248] The terminal device provided in this embodiment is used to execute the above-described information retrieval method, and therefore can achieve the same effect as the above-described implementation method.

[0249] When using integrated units, the terminal device may further include a processing module, a storage module, and a communication module. The processing module is used to control and manage the actions of the terminal device. The storage module supports the execution of stored program code and data by the terminal device. The communication module supports communication between the terminal device and other devices.

[0250] The processing module can be a processor or a controller. It can implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a digital signal processor (DSP), and a microprocessor, etc. The storage module can be a memory. The communication module can specifically be a radio frequency circuit, a Bluetooth chip, a Wi-Fi chip, or other devices that interact with other terminal devices.

[0251] In one embodiment, when the processing module is a processor and the storage module is a memory, the terminal device involved in this embodiment can be a device having... Figure 4 The device with the structure shown.

[0252] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform the information retrieval method of any of the above embodiments.

[0253] This application also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned steps to implement the information retrieval method described in the above embodiments.

[0254] In addition, embodiments of this application also provide an apparatus, which may specifically be a chip, component, or module. The apparatus may include a connected processor and a memory; wherein the memory is used to store computer execution instructions, and when the apparatus is running, the processor may execute the computer execution instructions stored in the memory to cause the chip to execute the information retrieval methods in the above-described method embodiments.

[0255] In this embodiment, the terminal device, computer-readable storage medium, computer program product or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0256] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0257] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0258] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0259] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0260] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0261] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An information retrieval method, wherein the method is executed by a terminal device, characterized in that, The method includes: Receive a query statement input by the user, the query statement being used to instruct the query of first personal information; The query statement is semantically understood to obtain a first understanding result; the first understanding result includes the target type to which the first personal information belongs, the target data source corresponding to the target type, the entity information contained in the query statement, the person information contained in the query statement, and the keywords in the query statement, wherein the target data source is an application in the terminal device; Based on the first understanding result, a search is performed in the target data source to obtain the search results.

2. The method according to claim 1, characterized in that, The step of retrieving data from the target data source based on the first understanding result to obtain retrieval results includes: If the first understanding result includes referential information, determine the actual information corresponding to the referential information, and perform a search in the target data source based on the first understanding result and the actual information to obtain the search result; If the referential information is not included in the first understanding result, the search is performed in the target data source based on the first understanding result to obtain the search result.

3. The method according to claim 2, characterized in that, The first understanding result also includes a preprocessed statement obtained by preprocessing the query statement. The step of retrieving data from the target data source based on the first understanding result and the actual information to obtain the retrieval result includes: Based on the actual information, the preprocessed statement is supplemented to obtain the supplemented statement; Based on the supplementary statement and the first understanding result, a search is performed in the target data source to obtain candidate results; The candidate results are subjected to relevance filtering and relevance sorting to obtain a sorting result; the sorting result represents the order of relevance between the candidate results and the query statement and the first understanding result. The N candidate results with the highest relevance in the sorting results are taken as the search results, where N is a positive integer.

4. The method according to claim 3, characterized in that, The candidate results are subjected to relevance filtering and relevance ranking to obtain the ranking results, including: Send a first request to the server; the first request carries the supplementary statement, the keyword and the candidate results, and the first request is used to request relevance filtering and relevance ranking of the candidate results; Receive the sorting result sent by the server.

5. The method according to claim 3 or 4, characterized in that, The preprocessing includes one or more of the following: Traditional / Simplified Chinese conversion, Pinyin character conversion, initial letter to character conversion, and capitalization conversion.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: Based on the first understanding result and the search result, generate the answer text corresponding to the query statement; Output the search results and the answer text.

7. The method according to claim 6, characterized in that, The terminal device includes a voice assistant application; The process of receiving user-inputted query statements includes: The voice assistant application receives query statements input by the user; The output of the search results and the answer text includes: The voice assistant application outputs the search results and the answer text.

8. The method according to claim 6 or 7, characterized in that, The output of the search results and the answer text includes: The search results and the answer text are integrated to obtain the query results; Output the query results.

9. The method according to any one of claims 6 to 8, characterized in that, The first understanding result also includes a preprocessed statement obtained by preprocessing the query statement. The step of generating the answer text corresponding to the query statement based on the first understanding result and the search result includes: Send a second request to the server; the second request carries the preprocessed statement and the search result, and the second request is used to request the generation of answer text that corresponds to the preprocessed statement and matches the search result; Receive the answer text sent by the service.

10. The method according to any one of claims 1 to 9, characterized in that, The semantic understanding of the query statement to obtain a first understanding result includes: A third request is sent to the server; the third request carries the query statement and is used to request semantic understanding of the query statement. Receive the first understanding result sent by the service.

11. The method according to any one of claims 1 to 10, characterized in that, The first understanding result is structured data.

12. The method according to any one of claims 1 to 11, characterized in that, The semantic understanding of the query statement to obtain a first understanding result includes: The query statement is classified by intent to obtain the intent classification result; If the intent classification result is the first type, then the query statement is semantically understood to obtain the first understanding result; the first type represents that the intent of the statement is to query personal information.

13. The method according to any one of claims 1 to 11, characterized in that, The target type is one of the following: ID card, appointment, SMS, calendar, or other types. The target data source corresponding to the ID card type includes photo library application, note application, and file application. The target data source corresponding to the appointment type includes calendar application, SMS application, and photo library application. The target data source corresponding to the SMS type includes SMS application. The target data source corresponding to the calendar type includes calendar application. The target data source corresponding to the other types includes photo library application, SMS application, note application, calendar application, and file application.

14. A terminal device, characterized in that, The terminal device includes: one or more processors, and a memory; The memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the terminal device to perform the method as described in any one of claims 1 to 13.

15. A chip system, characterized in that, The chip system is applied to a terminal device, and the chip system includes one or more processors, the one or more processors being used to invoke computer instructions to cause the terminal device to perform the method as described in any one of claims 1 to 13.

16. An information retrieval system, characterized in that, It includes a terminal device and a server, wherein the terminal device is used to perform the method as described in any one of claims 1 to 13.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes instructions that, when executed on a terminal device, cause the terminal device to perform the method as described in any one of claims 1 to 13.

Images