Intention searching method and device, equipment, medium and product

Abstract

The invention provides an intention search method and device, equipment, a medium and a product, and relates to the field of intelligent search. Comprising the steps of obtaining search request information input by a user; performing input type judgment processing on the search request information to determine the input type of the search request information; according to the input type, analyzing the search request information to obtain an input vector corresponding to the input type; performing feature fusion processing on a plurality of input vectors corresponding to the search request information to obtain a unified intention vector; acquiring scene information of a user; generating a scene vector according to the scene information; and executing search processing according to the unified intention vector and the scene vector to obtain a search result corresponding to the search request information. The technical problem that in the prior art, intention understanding is single, accurate search cannot be achieved, and consequently the intention search efficiency is low is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent search, and in particular to an intent search method and device, equipment, medium and product. BACKGROUND

[0002] The intelligent search engine system and method mainly faces the dynamic information retrieval demand in the multi-modal interaction scene, and is widely applicable to the fields such as telecommunications, e-commerce, industrial services, etc. which need high-precision semantic understanding and real-time situation adaptation.

[0003] The current mainstream search engine mainly relies on text keyword matching, PageRank, static index and user behavior data optimization, etc. to sort and present the results.

[0004] However, the existing technology has single intent understanding, which cannot realize accurate search, thereby resulting in low efficiency of intent search. SUMMARY

[0005] The present application provides an intent search method, device, equipment, medium and product to solve the problem of low efficiency of intent search caused by single intent understanding in the prior art.

[0006] In a first aspect, the present application provides an intent search method, comprising:

[0007] obtaining search request information input by a user;

[0008] performing input type judgment processing on the search request information to determine the input type of the search request information; wherein the input type includes multiple types of text, image and voice;

[0009] According to the input type, the search request information is analyzed and processed to obtain an input vector corresponding to the input type;

[0010] performing feature fusion processing on the multiple input vectors corresponding to the search request information to obtain a unified intent vector;

[0011] obtaining scenario information of the user;

[0012] According to the scenario information, a scenario vector is generated;

[0013] According to the unified intent vector and the scenario vector, search processing is performed to obtain a search result corresponding to the search request information.

[0014] In a possible design, the feature fusion processing on the multiple input vectors corresponding to the search request information to obtain a unified intent vector comprises:

[0015] The plurality of input vectors are linearly combined according to a preset weight matrix and a preset bias term to obtain a combination result.

[0016] The combination result is nonlinearly transformed according to a preset activation function to obtain a unified intention vector.

[0017] In a possible design, the preset weight matrix is trained by historical input vectors corresponding to a user and unified intention vectors corresponding to the historical input vectors.

[0018] In a possible design, the search request information is parsed according to an input type to obtain an input vector corresponding to the input type, including:

[0019] If the input type includes a text type, a natural language model is used to perform word segmentation processing and semantic encoding processing on the search request information of the text type to obtain an input vector corresponding to the text type.

[0020] If the input type includes an image type, a visual model is used to perform visual feature extraction processing and normalization processing on the search request information of the image type to obtain an input vector corresponding to the image type.

[0021] If the input type includes a speech type, a speech recognition model is used to perform speech text conversion processing, encoding processing and background noise filtering processing on the search request information of the speech type to obtain an input vector corresponding to the speech type.

[0022] In a possible design, the context information includes multiple of a geographical location, a time, a device, a network state and a user identity.

[0023] In a possible design, search processing is performed according to the unified intention vector and the context vector to obtain a search result corresponding to the search request information, including:

[0024] The unified intention vector and the context vector are fused to obtain an intention-context fusion vector.

[0025] Retrieval is performed according to the intention-context fusion vector to obtain a retrieval result.

[0026] The search result corresponding to the search request information is determined according to the retrieval result.

[0027] In a possible design, retrieval is performed according to the intention-context fusion vector to obtain a retrieval result, including:

[0028] Retrieval is performed according to the intention-context fusion vector in at least one of a knowledge graph, a database and real-time network resources to obtain a retrieval result.

[0029] In a possible design, according to the search result, the search result corresponding to the search request information is determined, including:

[0030] The preconfigured reinforcement learning model is acquired;

[0031] According to the preconfigured reinforcement learning model, the search result is sorted to obtain a sorted result;

[0032] According to the sorted result, the search result corresponding to the search request information is determined.

[0033] In a possible design, after the search result corresponding to the search request information is determined, the method further includes:

[0034] The preference data and / or the device capability data of the user are acquired;

[0035] According to the preference data and / or the device capability data, the display mode is determined, where the display mode includes at least one of a list, a graph, and a three-dimensional model.

[0036] In a second aspect, the present application provides an intent search device, including:

[0037] The first acquisition module is configured to acquire the search request information input by the user;

[0038] The classification module is configured to perform input type judgment processing on the search request information to determine the input type of the search request information, where the input type includes multiple types of text, image, and voice.

[0039] The analysis module is configured to perform analysis processing on the search request information according to the input type to obtain an input vector corresponding to the input type.

[0040] The fusion module is configured to perform feature fusion processing on multiple input vectors corresponding to the search request information to obtain a unified intent vector.

[0041] The second acquisition module is configured to acquire the scenario information of the user.

[0042] The generation module is configured to generate a scenario vector according to the scenario information.

[0043] The search module is configured to perform search processing according to the unified intent vector and the scenario vector to obtain a search result corresponding to the search request information.

[0044] In a third aspect, the present application provides an intent search device, including a memory and a processor.

[0045] The memory stores computer execution instructions.

[0046] The processor executes computer-executed instructions stored in the memory, so that the processor executes the intent search method as the summary of the first aspect.

[0047] In a fourth aspect, the present application provides a computer-readable storage medium, which stores computer-executed instructions. When the computer-executed instructions are executed by a processor, the computer-executed instructions are used to implement the intent search method as the summary of the first aspect.

[0048] In a fifth aspect, the present application provides a computer program product, which comprises a computer program. When the computer program is executed by a processor, the computer program implements the intent search method as the summary of the first aspect.

[0049] The intent search method, device, equipment, medium and product provided by the present application comprise the following steps: obtaining search request information input by a user; performing input type judgment processing on the search request information to determine the input type of the search request information; performing analysis processing on the search request information according to the input type to obtain an input vector corresponding to the input type; performing feature fusion processing on multiple input vectors corresponding to the search request information to obtain a unified intent vector; obtaining scenario information of the user; generating a scenario vector according to the scenario information; and performing search processing according to the unified intent vector and the scenario vector to obtain a search result corresponding to the search request information. Compared with the prior art, the intent understanding is single, and accurate search cannot be realized, thereby resulting in low intent search efficiency. The present application unifies text, image, voice and other multi-modal inputs into a semantic consistent intent vector, and then generates a context vector by combining real-time contexts such as geographical position, device type and time. Finally, the result sorting strategy is dynamically adjusted through reinforcement learning to improve search efficiency and user experience. The present application breaks through the limitation of traditional search engines relying on single mode and static index, realizes the unification of multi-modal fusion and dynamic adaptation, and improves the intent search efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0051] Figure 1 A system architecture schematic diagram of an intent search method provided by the embodiments of the present application;

[0052] Figure 2 A flowchart of an intent search method provided by the embodiments of the present application Figure 1 ;

[0053] Figure 3 An intent search method flowchart provided for an embodiment of the present application Figure 2

[0054] Figure 4 An intent search method flowchart provided for an embodiment of the present application Figure 3

[0055] Figure 5 An intent search system architecture diagram provided for an embodiment of the present application

[0056] Figure 6 An intent search system operation flowchart provided for an embodiment of the present application

[0057] Figure 7 A structure diagram of an intent search device provided for an embodiment of the present application

[0058] Figure 8 A structure diagram of an intent search device provided for an embodiment of the present application DETAILED DESCRIPTION

[0059] The exemplary embodiments will be described in detail herein below with reference to the drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The following exemplary embodiments described in the exemplary embodiments do not represent all of the implementations in accordance with this application. Rather, they are merely some examples consistent with some aspects of this application as detailed in the appended claims.

[0060] In the embodiments of the present application, the terms "first", "second", and the like are used to distinguish between similar or identical items or elements having substantially the same function and action. Those skilled in the art can understand that the terms "first", "second", and the like do not limit the quantity and execution order, and the terms "first", "second", and the like do not necessarily mean different. It should be noted that in the embodiments of the present application, the words "exemplary" or "for example" are used to mean serving as an example, instance, or illustration. Any embodiment or design described herein as "exemplary" or "for example" should not be construed as being preferred or superior over other embodiments or designs. Rather, the use of the words "exemplary" or "for example" is intended to present concepts in a particular manner. In the embodiments of the present application, "at least one" means one or more, and "multiple" means two or more.

[0061] ​​It should be noted that the "at" in the embodiments of the present application can be the moment when a certain condition occurs, or a period of time after a certain condition occurs, and the embodiments of the present application do not make specific limitations. In addition, the intent search method provided by the embodiments of the present application is only as an example, and the intent search method can also include more or less content.

[0062] The intelligent search engine system and method mainly faces the dynamic information retrieval demand in the multi-modal interaction scene, and is widely applicable to the fields such as telecommunications, e-commerce, AR / VR, industrial services and the like which need high-precision semantic understanding and real-time scene adaptation.

[0063] The existing search engine mainly relies on text keyword matching, static indexing and user behavior data to optimize result sorting, and the technical solution has the following limitations:

[0064] Optionally, single-modal input processing: traditional search engines are text-centered, and keyword matching is performed through algorithms, which is difficult to analyze the deep semantics of non-text inputs such as images and voice, resulting in deviation of intent understanding in multi-modal scenes.

[0065] Optionally, static result generation: search results are sorted based on predefined indexes or historical data, lacking dynamic adaptation to real-time scenarios of users (such as geographic location, device type, time, user identity, etc.), and unable to meet personalized needs.

[0066] Optionally, inefficient interaction mode: users need to click, filter or input multiple times to accurately locate target information, and the system lacks active optimization mechanism, resulting in cumbersome operation and low information acquisition efficiency.

[0067] Optionally, fixed sorting strategy: existing sorting algorithms rely on static feature extraction and do not introduce dynamic optimization mechanisms such as reinforcement learning, making it difficult to adjust result priorities according to real-time user feedback.

[0068] In view of the above problems, the inventors found during the research on the low efficiency of intent search that the existing technology has single intent understanding, which cannot achieve accurate search, thereby leading to low efficiency of intent search. Accordingly, the inventors consider that the multi-modal inputs such as text, image and voice are uniformly represented as semantic consistent intent vectors, and then the real-time context vectors such as geographic location, device type and time are generated, and finally the result sorting strategy is dynamically adjusted through reinforcement learning to improve the search efficiency and user experience, breaking through the limitations of traditional search engines relying on single mode and static index, and realizing the unification of multi-modal fusion and dynamic adaptation. Based on this, the embodiments of the present application provide an intent search method, device, equipment, medium and product, which can be used in the field of intelligent search, and aims to solve the problem of low efficiency of intent search in the prior art.

[0069] The technical solutions of the present application and how the technical solutions solve the above technical problems will be described in detail below with specific examples. The following specific examples can be combined with each other, and the same or similar concepts or processes can not be described again in some examples. The embodiments of the present application will be described below with reference to the drawings.

[0070] Figure 1 A system architecture diagram of an intention search method provided by an embodiment of the present application is shown, and the intention search system is a computer device. Figure 1 In the above architecture, the above architecture includes at least one of the data acquisition device 101, the processing device 102 and the display device 103.

[0071] It can be understood that the structure shown in the embodiment of the present application does not constitute a specific limitation on the architecture of the processing system of the intention search method. In other feasible embodiments of the present application, the above architecture can include more or fewer components than the diagram, or combine some components, or split some components, or different component arrangement, which can be determined according to the actual application scene, and is not limited here. Figure 1 The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0072] In the specific implementation process, the data acquisition device 101 can include an input / output interface and can also include a communication interface. The data acquisition device 101 can be connected with the processing device through the input / output interface or the communication interface to obtain relevant data.

[0073] The processing device 102 can obtain the search result corresponding to the search request information according to the relevant data.

[0074] The display device 103 can also be a touch display screen or a screen of a terminal device, which is used to receive user instructions while displaying the above content to realize interaction with the user.

[0075] It should be understood that the above processing device can be implemented by a processor reading and executing instructions in a memory, or by a chip circuit.

[0076] In addition, the network architecture and business scenarios described in the embodiments of the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. It can be known by those skilled in the art that, with the evolution of network architecture and the appearance of new business scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0077] The technical solutions of the present application will be described in detail below with specific examples:

[0078] Figure 2An intent search method flowchart provided for an embodiment of the present application Figure 1 As shown in Figure 2 The method comprises the following steps:

[0079] S201, obtaining search request information input by a user.

[0080] S202, performing input type judgment processing on the search request information to determine the input type of the search request information.

[0081] The input type includes multiple types selected from the group consisting of a text type, an image type, and a voice type.

[0082] S203, performing analysis processing on the search request information according to the input type to obtain an input vector corresponding to the input type.

[0083] In one possible embodiment, the relevance of cross-modal features is modeled by introducing a graph neural network on the basis of existing multi-modal intent fusion. By constructing a graph structure (such as a text-image-voice triadic relationship graph) among multi-modal features, the neighbor aggregation mechanism of the graph neural network is used to dynamically adjust the weights of the features of each modality. For example, for “image+voice” input, the system takes the text features and image features as graph nodes and the voice features as edge attributes, and through GNN iterative calculation of information transmission between nodes, a more accurate intent vector is finally generated.

[0084] The semantic relevance between multi-modal data is explicitly modeled through the graph structure, solving the problem of insufficient modeling of long-distance dependency relationships by traditional attention mechanisms. For example, in the “picture+voice” mixed input scenario, the system can more accurately identify the complementarity between the image and the voice (such as a picture showing a “coffee cup” and a voice describing “recommend the same one”), and the system can strengthen the association weight between the image and the text to avoid the deviation of the intent caused by the misjudgment of the voice.

[0085] S204, performing feature fusion processing on the multiple input vectors corresponding to the search request information to obtain a unified intent vector.

[0086] Specifically, the multiple input vectors are linearly combined according to a preset weight matrix and a preset bias term to obtain a combination result.

[0087] The preset weight matrix is trained by a historical input vector corresponding to the user and a unified intent vector corresponding to the historical input vector.

[0088] Further, the combination result is nonlinearly transformed according to a preset activation function to obtain the unified intent vector.

[0089] For example, a unified user intent vector is generated by combining natural language processing, computer vision, and speech recognition technology.

[0090] Specifically, the unified intention vector formula is:

[0091]

[0092] wherein I is the fused intention vector; Ft, Fi, and Fa represent the feature vectors of the text, image, and voice respectively; Wt, Wi, and Wa represent the weight matrices corresponding to the text, image, and voice respectively; b is a bias term; is an activation function.

[0093] More specifically, the weight matrices Wt, Wi, and Wa are randomly initialized using Xavier, and are trained and optimized through user historical interaction data (such as 'image + voice' search intention annotation data).

[0094] More specifically, the activation function ReLU is selected to avoid the problem of gradient disappearance.

[0095] More specifically, the dimension of the fused intention vector I is unified to 1024.

[0096] S205, obtain the scenario information of the user.

[0097] The scenario information includes multiple of the geographical location, time, device, network state, and user identity.

[0098] S206, generate a scenario vector according to the scenario information.

[0099] S207, perform search processing according to the unified intention vector and the scenario vector, to obtain the search result corresponding to the search request information.

[0100] In this embodiment, the context vector is fused with the intention vector to enhance the contextual understanding of the query.

[0101] In one possible embodiment, a time series model is introduced to model the time series of the user context data. For example, the system analyzes the historical location trajectory, device usage period, and other time series features of the user through the time series model, and dynamically adjusts the current context weight (such as increasing the geographical location weight during the commuting period, and reducing the device type influence in the night search scenario).

[0102] The dynamic evolution relationship between the user behavior pattern and the context is captured through time series modeling, and the real-time adaptability of the search result is improved. For example, when a user searches for 'nearby coffee shops' during the rush hour, the system can combine the historical trajectory to predict the user's walking path, and preferentially recommend the high-rated stores along the path; while searching for the same keyword at night, the system may focus on the '24-hour operation' attribute to avoid returning closed stores.

[0103] The intent search method provided by the embodiment comprises: obtaining search request information input by a user; performing input type judgment processing on the search request information to determine the input type of the search request information; performing analysis processing on the search request information according to the input type to obtain an input vector corresponding to the input type; performing feature fusion processing on multiple input vectors corresponding to the search request information to obtain a unified intent vector; obtaining scenario information of the user; generating a scenario vector according to the scenario information; and performing search processing according to the unified intent vector and the scenario vector to obtain a search result corresponding to the search request information. Compared with the prior art, the intent understanding is single, and accurate search cannot be realized, thereby resulting in low intent search efficiency. The application unifies text, image, voice and other multi-modal inputs into a semantic consistent intent vector, and then generates a context vector by combining a geographic location, a device type, a time and other real-time contexts, and finally adjusts a result sorting strategy dynamically through reinforcement learning to improve search efficiency and user experience. The application breaks through the limitation of traditional search engines relying on a single mode and static index, realizes the unification of multi-modal fusion and dynamic adaptation, and improves the intent search efficiency.

[0104] Figure 3 The intent search method provided by the embodiment of the application Figure 2 As shown in the figure, the above step S203 specifically comprises:

[0105] S301, if the input type comprises a text type, a natural language model is used to perform word segmentation processing and semantic encoding processing on the search request information of the text type, so as to obtain an input vector corresponding to the text type.

[0106] For example, a BERT-base model is used to perform word segmentation and semantic encoding on the input text.

[0107] Specifically, the text feature vector Ft with an output dimension of 768 is obtained, wherein the model parameters are fine-tuned through Chinese search corpus (such as dialogue data), and the semantic matching accuracy is greater than or equal to 92%.

[0108] S302, if the input type comprises an image type, a visual model is used to perform visual feature extraction processing and normalization processing on the search request information of the image type, so as to obtain an input vector corresponding to the image type.

[0109] For example, based on a CLIP-ViT-B / 32 model, image global features are extracted, and a 512-dimensional image feature vector Fi is output, and normalization processing is performed to ensure that the text feature dimension is adapted.

[0110] S303, if the input type includes a speech type, a speech recognition model is used to perform speech text conversion processing, encoding processing, and background noise filtering processing on the search request information of the input type of the speech type to obtain an input vector corresponding to the speech type.

[0111] For example, using the Whisper-small model, the speech signal is converted into text, and then a 768-dimensional speech feature vector Fa is output through BERT encoding, while filtering background noise (when the signal-to-noise ratio is ≥20 dB, the recognition accuracy is ≥95%).

[0112] It should be noted that by using multi-modal feature extraction and attention weight fusion, the intention understanding deviation problem caused by the dependence of traditional search engines on a single mode is solved, and unified representation of cross-modal semantics is realized.

[0113] In this embodiment, by using a natural language model to perform word segmentation and semantic encoding on text types, a visual model to perform visual feature extraction and normalization on image types, and a speech recognition model to perform differential processing strategies such as text conversion and background noise filtering on speech types, unified vectorization representation of multi-modal input information is realized, which not only improves the feature extraction accuracy of different modal data, but also enhances the compatibility and retrieval efficiency of cross-modal search through standardized vector output, and finally builds a full-modal intelligent search processing framework covering text, image, and speech. Thus, the intention search efficiency is improved.

[0114] Figure 4 An intention search method flow provided by the embodiment of the present application Figure 3 As shown in the figure, the specific implementation steps of the above S207 include:

[0115] S401, the unified intention vector and the scene vector are fused to obtain an intention scene fusion vector.

[0116] Specifically, the fusion formula is:

[0117]

[0118] Where C is the fused context vector; I is the intention vector; G, T, D, and U represent the geographical location, time, device, and user identity, respectively, as context vectors. : is the attention weight.

[0119] More specifically, the attention weight Through multi-layer perception calculation, the input is the importance feature of the current context of the user (such as the time weight β increasing by 30% in the 'commuting time' scenario), and finally through Softmax normalization, the weight sum is ensured to be 1.

[0120] S402、According to the intention-scenario fusion vector, a retrieval operation is performed to obtain a retrieval result.

[0121] Specifically, according to the intention-scenario fusion vector, a retrieval process is performed in at least one of a knowledge graph, a database, and real-time network resources to obtain a retrieval result.

[0122] For example, based on the context vector, a matching result is retrieved in real time from at least one of a knowledge graph, a database, and real-time network resources (such as retrieving a nearby service hall according to a geographic location).

[0123] S403、According to the retrieval result, a search result corresponding to the search request information is determined.

[0124] Specifically, a pre-configured reinforcement learning model is obtained.

[0125] Further, according to the pre-configured reinforcement learning model, the retrieval result is sorted to obtain a sorted result.

[0126] For example, the sorting objective function in the reinforcement learning model is:

[0127]

[0128] where f(d) is a score function of a document d; is a positive sample and a negative sample document.

[0129] Further, according to the sorted result, a search result corresponding to the search request information is determined.

[0130] Among them, the search result most suitable for the current situation is returned preferentially.

[0131] S404、Obtain user preference data and / or device capability data.

[0132] S405、According to the preference data and / or device capability data, determine the display mode.

[0133] Among them, the display mode includes at least one of a list, a graph, and a three-dimensional model.

[0134] For example, according to user preferences and device capabilities (such as mobile terminals, AR glasses), search results (such as lists, graphs, 3D models, etc.) are adaptively presented.

[0135] Specifically, a mathematical expression formula for device adaptation is added:

[0136]

[0137] where P is a probability distribution of a presentation mode (such as a list, a graph, and a 3D model); C is a context vector; and Wp and bp are learnable parameters.

[0138] In a possible embodiment, a meta-learning framework is introduced to enable the model to quickly adapt to different user groups or scene requirements. For example, the system pre-trains a general ranking strategy through a meta-learning (Model-Agnostic Meta-Learning, MAML) algorithm, and when a new user or a new scene appears, only a small amount of samples are needed to fine-tune the model parameters to generate an adaptive ranking strategy.

[0139] In this way, the dependence of the model on large-scale user data is reduced through meta-learning, and the generalization ability of personalized ranking is improved. For example, in an e-commerce scene, for a user who uses for the first time, the system can quickly generate a personalized ranking strategy (for example, a user who prefers image search is preferentially returned with an image list result) based on a small amount of interaction data of the user, without waiting for long-term behavior data to accumulate.

[0140] In this embodiment, through the fusion processing of the unified intention vector and the scene vector, the deep coupling of user demand and scene information is realized, and an intention-scene fusion vector with high semantic density is generated; based on the cross-domain retrieval of the vector in the knowledge graph, the database and the real-time network resource, the accurate matching and dynamic integration of multi-source heterogeneous information are realized; through the ranking optimization of the retrieval result by the pre-configured reinforcement learning model, the intelligent result optimization based on user feedback is realized, and the relevance and credibility of the search result are improved; finally, the dynamic adaptive display mode (list / map / 3D model) is realized by combining the user preference data and the device capability data, realizing the whole-link personalized service from information retrieval to result presentation, and significantly enhancing the scene adaptability, result accuracy and user experience continuity of the search system. Thus, the intention search efficiency is improved.

[0141] The application also provides a possible embodiment, Figure 5 The intention search system architecture provided in the embodiment of the application is shown in FIG. 1. Figure 5 As shown in the figure, the system architecture starts from a dynamic result generation and ranking engine, integrates the data flow of a knowledge graph, real-time network resources, a structured database and a reinforcement learning ranking model, generates adaptive results (such as a list / map / 3D model, etc.) through a personalized presentation module, and outputs through a Web / mobile terminal / voice assistant and the like. After multi-modal input (text / image / voice) of a user is processed by a corresponding analysis unit, it is fused into a unified intention vector through NLP / CV / ASR technology, and a dynamic context perception module generates a context vector based on location / time / device / identity. Finally, bidirectional optimization is completed in an intention-context fusion device, and a closed-loop data flow is formed.

[0142] In this embodiment, through the synergy of the three-source fusion of knowledge graph, real-time resources and structured data and the reinforcement learning sorting model, efficient and accurate generation of dynamic results is realized; the introduction of multi-modal input analysis and context perception module enables the system to deeply understand user intent and scene information, improving response adaptability; the intent and context dual-fusion mechanism further guarantees the accuracy and relevance of the output results, and the overall architecture significantly enhances the flexibility of cross-scene applications and the coherence of user experience while improving the level of intelligence.

[0143] The present application also provides a possible embodiment, Figure 6 The intent search system running process schematic diagram provided by the embodiment of the present application is shown in Figure 6 As shown, the complete processing link of multi-modal search request: after the user initiates a request, the system first performs input type judgment (text / image / voice), then realizes multi-modal understanding through NLP semantic analysis, CV visual feature extraction, and semantic analysis after ASR speech recognition to text, generates a unified intent vector; at the same time, the dynamic context perception module generates a context vector based on location, time, device, identity, etc., and the two complete bidirectional optimization in the intent-context fusioner; the system then dynamically retrieves the knowledge graph, structured database and real-time network resources, generates the final result through the reinforcement learning sorting model, adopts adaptive presentation strategy (list / graph / 3D model, etc.) to output through Web, mobile terminal, voice assistant, etc., and finally completes the sorting model iterative optimization through user feedback behavior record.

[0144] In this embodiment, through the deep fusion of multi-modal input analysis and context perception, the leap from "single-dimensional understanding" to "full-scene adaptation" is realized—integrating text, image, and voice three-source information significantly improves the relevance and personalization level of search results, combining the dynamic retrieval mechanism of real-time context information enhances the adaptability of the results to the scene where the user is, and the introduction of the reinforcement learning sorting model and the user feedback closed loop builds a continuously optimized intelligent ecosystem, ultimately improving user experience while realizing the dual breakthroughs of system intelligence level and application scene flexibility.

[0145] Figure 7 The structure schematic diagram of the intent search device provided by the embodiment of the present application is shown in Figure 7 As shown, the device comprises a first acquisition module 71, a classification module 72, an analysis module 73, a fusion module 74, a second acquisition module 75, a generation module 76 and a search module 77.

[0146] The first acquisition module 71 is configured to acquire the search request information input by the user.

[0147] The classification module 72 is configured to perform input type judgment processing on the search request information to determine an input type of the search request information, wherein the input type includes multiple types of text, image, and voice.

[0148] The analysis module 73 is configured to perform analysis processing on the search request information according to the input type to obtain an input vector corresponding to the input type.

[0149] The fusion module 74 is configured to perform feature fusion processing on multiple input vectors corresponding to the search request information to obtain a unified intention vector.

[0150] The second acquisition module 75 is configured to acquire scene information of the user.

[0151] The generation module 76 is configured to generate a scene vector according to the scene information.

[0152] The search module 77 is configured to perform search processing according to the unified intention vector and the scene vector to obtain a search result corresponding to the search request information.

[0153] In a possible design, the feature fusion processing on the multiple input vectors corresponding to the search request information to obtain the unified intention vector includes:

[0154] The fusion module 74 is further configured to perform linear combination processing on the multiple input vectors according to a preset weight matrix and a preset bias term to obtain a combination result.

[0155] The combination result is subjected to nonlinear transformation processing according to a preset activation function to obtain the unified intention vector.

[0156] In a possible design, the preset weight matrix is obtained by training historical input vectors corresponding to the user and unified intention vectors corresponding to the historical input vectors.

[0157] In a possible design, the analysis processing on the search request information according to the input type to obtain the input vector corresponding to the input type includes:

[0158] If the input type includes the text type, the analysis module 73 is further configured to perform word segmentation processing and semantic encoding processing on the search request information of the text type by using a natural language model to obtain the input vector corresponding to the text type.

[0159] If the input type includes the image type, the analysis module 73 is further configured to perform visual feature extraction processing and normalization processing on the search request information of the image type by using a visual model to obtain the input vector corresponding to the image type.

[0160] If the input type includes a speech type, a speech recognition model is adopted to perform speech text conversion processing, encoding processing and background noise filtering processing on the search request information of the speech type, to obtain an input vector corresponding to the speech type.

[0161] In a possible design, the context information includes multiple of the following: a geographical location, a time, a device, a network status and a user identity.

[0162] In a possible design, the search processing is performed according to the unified intent vector and the context vector, to obtain a search result corresponding to the search request information, including:

[0163] The search module 77 is further configured to perform fusion processing on the unified intent vector and the context vector, to obtain an intent-context fusion vector.

[0164] The search processing is performed according to the intent-context fusion vector, to obtain a search result.

[0165] The search result corresponding to the search request information is determined according to the search result.

[0166] In a possible design, the search processing is performed according to the intent-context fusion vector, to obtain a search result, including:

[0167] The search module 77 is further configured to perform search processing according to the intent-context fusion vector, in at least one of the following: a knowledge graph, a database and real-time network resources, to obtain a search result.

[0168] In a possible design, the search result corresponding to the search request information is determined according to the search result, including:

[0169] The search module 77 is further configured to obtain a preconfigured reinforcement learning model.

[0170] The search result is sorted according to the preconfigured reinforcement learning model, to obtain a sorted result.

[0171] The search result corresponding to the search request information is determined according to the sorted result.

[0172] In a possible design, after the search result corresponding to the search request information is determined, the method further includes:

[0173] The search module 77 is further configured to obtain preference data and / or device capability data of the user.

[0174] The display mode is determined according to the preference data and / or the device capability data, where the display mode includes at least one of the following: a list, a graph and a three-dimensional model.

[0175] The embodiment provides an intention search device, which can execute the intention search method in the above embodiment, and has similar implementation principles and technical effects, which will not be described here again.

[0176] In the specific implementation of the intention search method, each module can be implemented as a processor, and the processor can execute computer execution instructions stored in the memory, so that the processor executes the intention search method.

[0177] Figure 8 A structural schematic diagram of an intention search device provided by the embodiment is shown in the figure. Figure 8 As shown in the figure, the intention search device 80 includes at least one processor 81 and a memory 82. The intention search device 80 also includes a communication component 83. The processor 81, the memory 82 and the communication component 83 are connected through a bus 84.

[0178] In the specific implementation process, the at least one processor 81 executes computer execution instructions stored in the memory 82, so that the at least one processor 81 executes the method of searching in the intelligent field as executed by the intention search device.

[0179] The specific implementation process of the processor 81 can refer to the above method embodiment, which has similar implementation principles and technical effects, and will not be described here again.

[0180] In the above embodiment, it should be understood that the processor can be a central processing unit (English: Central Processing Unit, CPU for short), and can also be other general-purpose processors, digital signal processors (English: Digital Signal Processor, DSP for short), application specific integrated circuits (English: Application Specific Integrated Circuit, ASIC for short) and the like. The general-purpose processor can be a microprocessor, or the processor can also be any conventional processor, etc. The steps of the method disclosed in the application can be directly embodied as hardware processor execution, or executed by hardware and software modules in the processor.

[0181] The memory can contain a high-speed RAM memory, and can also include a non-volatile storage NVM, for example, at least one disk memory.

[0182] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, the bus in the drawings of the present application does not limit to only one bus or one type of bus.

[0183] The functions implemented by the intent search device and the master device described above are introduced for the scheme provided by the embodiments of the present application. It can be understood that the intent search device or the master device includes the hardware structure and / or software module for executing the respective functions in order to implement the above functions. The units and algorithm steps of each example described in combination with the embodiments disclosed in the embodiments of the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is implemented in hardware or in the form of computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the technical solutions of the embodiments of the present application.

[0184] The present application also provides a computer readable storage medium, the computer readable storage medium stores computer execution instructions, when the processor executes the computer execution instructions, the computer readable storage medium is used to implement the method in the intelligent search field as above.

[0185] The above readable storage medium can be implemented by any type of volatile or non-volatile storage device or their combination, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0186] An exemplary readable storage medium is coupled to the processor, so that the processor can read information from the readable storage medium, and can write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist as discrete components in the intent search device or the master device.

[0187] The application further provides a computer program product, which comprises a computer program stored in a readable storage medium, and at least one processor of an intent search device can read the computer program from the readable storage medium, and the at least one processor executes the computer program so that the intent search device executes the scheme provided in any one of the above embodiments.

[0188] Those skilled in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by relevant hardware instructed by programs. The foregoing programs can be stored in a computer readable storage medium. When the programs are executed, the steps of the above-mentioned method embodiments are executed; and the foregoing storage medium includes various media capable of storing program codes, such as ROM, RAM, magnetic disks, or optical disks.

[0189] So far, the technical scheme of the application has been described in combination with the preferred embodiments shown in the drawings, but those skilled in the art can easily understand that the protection scope of the application is obviously not limited to these specific embodiments, and the above embodiments are only used to illustrate the technical scheme of the application, rather than limit it; although the application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical scheme recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical scheme deviate from the scope of the technical scheme of the embodiments of the application.

Claims

1. An intent search method, characterized in that, include: Obtain the search request information entered by the user; The search request information is subjected to input type determination processing to determine the input type of the search request information; wherein, the input type includes multiple types such as text type, image type and voice type; Based on the input type, the search request information is parsed to obtain the input vector corresponding to the input type; The multiple input vectors corresponding to the search request information are subjected to feature fusion processing to obtain a unified intent vector; Obtain the user's context information; Based on the scenario information, a scenario vector is generated; Based on the unified intent vector and the context vector, a search process is performed to obtain the search results corresponding to the search request information.

2. The method according to claim 1, characterized in that, The step of performing feature fusion processing on multiple input vectors corresponding to the search request information to obtain a unified intent vector includes: Based on a preset weight matrix and a preset bias term, the multiple input vectors are linearly combined to obtain a combined result. The combined result is subjected to a nonlinear transformation based on a preset activation function to obtain a unified intent vector.

3. The method according to claim 2, characterized in that, The preset weight matrix is ​​obtained by training the user's historical input vector and the unified intent vector corresponding to the historical input vector.

4. The method according to claim 1, characterized in that, The step of parsing the search request information according to the input type to obtain the input vector corresponding to the input type includes: If the input type includes a text type, then a natural language model is used to perform word segmentation and semantic encoding on the search request information with the input type being the text type, so as to obtain the input vector corresponding to the text type; If the input type includes an image type, then a visual model is used to perform visual feature extraction and normalization on the search request information with the input type being the image type, so as to obtain the input vector corresponding to the image type. If the input type includes a speech type, a speech recognition model is used to perform speech-to-text conversion, encoding, and background noise filtering on the search request information with the input type being the speech type, in order to obtain the input vector corresponding to the speech type.

5. The method according to any one of claims 1 to 4, characterized in that, The contextual information includes multiple factors such as geographical location, time, device, network status, and user identity.

6. The method according to claim 5, characterized in that, The step of performing search processing based on the unified intent vector and the context vector to obtain search results corresponding to the search request information includes: The unified intent vector and the context vector are fused to obtain an intent-context fusion vector; Based on the intent context fusion vector, a retrieval operation is performed to obtain the retrieval results; Based on the search results, determine the search results corresponding to the search request information.

7. The method according to claim 6, characterized in that, The step of performing a retrieval operation based on the intent context fusion vector to obtain retrieval results includes: Based on the intent context fusion vector, retrieval processing is performed in at least one of the knowledge graph, database, and real-time network resources to obtain retrieval results.

8. The method according to claim 7, characterized in that, The step of determining the search results corresponding to the search request information based on the search results includes: Obtain a pre-configured reinforcement learning model; The retrieval results are sorted according to the pre-configured reinforcement learning model to obtain the sorted results; Based on the sorting results, the search results corresponding to the search request information are determined.

9. The method according to claim 8, characterized in that, After determining the search results corresponding to the search request information based on the sorting results, the method further includes: Obtain the user's preference data and / or device capability data; The display method is determined based on the preference data and / or the device capability data; wherein the display method includes at least one of list, graph and 3D model.

10. An intent-searching device, characterized in that, include: The first acquisition module is used to acquire the search request information input by the user; The classification module is used to perform input type judgment processing on the search request information to determine the input type of the search request information; wherein, the input type includes multiple types such as text type, image type and voice type; The parsing module is used to parse the search request information according to the input type to obtain the input vector corresponding to the input type; The fusion module is used to perform feature fusion processing on multiple input vectors corresponding to the search request information to obtain a unified intent vector; The second acquisition module is used to acquire the user's context information; The generation module is used to generate a scenario vector based on the scenario information; The search module is used to perform search processing based on the unified intent vector and the context vector to obtain the search results corresponding to the search request information.

11. An intent-searching device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-9.

13. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method described in any one of claims 1-9.

Images