Information search method and system, electronic equipment and storage medium

Abstract

The invention discloses an information search method and system, electronic equipment and a storage medium. The method comprises the steps of obtaining current context information, a current search text, a current figure image and a historical search result sequence of a user; analyzing the current context information based on a context sensing mechanism to obtain a current analysis search intention of the user; fusing features of the current search text and features of the current figure image into multi-modal data in a pre-trained convolutional neural network of the current search text and the current figure image, and analyzing current emotion information of the user by using the multi-modal data; inputting the historical search result sequence into a pre-trained prediction model, and predicting to obtain a current predicted search intention of the user; according to the current analysis search intention, the current emotion information and the current prediction search intention of the user, current search results searched based on the current search text are adjusted and sorted, and a final search result is obtained and output.

Description

Technical Field

[0001] The present application relates to the field of information search technology, and in particular to an information search method and system, an electronic device, and a storage medium. Background Art

[0002] Currently, a search function is provided in all applications so that users can search for required data or web pages, etc., by combining with a corresponding search engine, which performs a search and provides feedback based on the query text input by the user.

[0003] Traditional search engines mainly extract keywords from the query text entered by the user, and then rely on keyword matching and preset static ranking algorithms, such as PageRank, to extract and sort related web pages or information from the index database, and feed back the top results to the user. In order to make the search results suitable for different users, some new search engines have also introduced personalized search functions, which will analyze the recent search history of users and the records of selected search results to determine the type of the user's recent search results, so that they can feed back results similar to the type of the user's recent search results.

[0004] However, for the same search text, different users may have different actual search intentions, so the accuracy of search results from traditional search engines is low. And users' search needs are constantly changing. For example, if it is a weekday recently, their search intention is usually to search for work-related information, but if it is a weekend, their search intention is to search for information unrelated to work, and the search results will also be affected by the environment they are in, so feedback of results similar to the recent search results cannot effectively guarantee that the search results meet the user's actual search intentions. Therefore, the accuracy of search results in existing methods is limited, and users usually need to search multiple times. Summary of the invention

[0005] Based on the above-mentioned deficiencies of the prior art, the present application provides an information search method and system, an electronic device, and a storage medium to solve the problem that the prior art cannot guarantee the accuracy of search results.

[0006] In order to achieve the above objectives, this application provides the following technical solutions:

[0007] The first aspect of the present application provides an information search method, comprising:

[0008] Acquire the user's current context information, the current search text input by the user, the user's current character image and a sequence of historical search results; wherein the current context information includes current environment information and historical behavior information within a preset time period; the current character image includes a facial image and a posture image of the current character; the sequence of historical search results includes the results searched and selected by the user at multiple consecutive time points before the current moment;

[0009] Analyze the current context information based on a context awareness mechanism to obtain the current analyzed search intention of the user;

[0010] In a convolutional neural network pre-trained for the current search text and the current character image, the features of the current search text and the features of the current character image are fused into multimodal data through the convolutional neural network, and the current emotion information of the user is analyzed using the multimodal data;

[0011] Inputting the historical search result sequence into a pre-trained prediction model, and using the prediction model to predict the user's search intention based on the historical search result sequence to obtain the user's current predicted search intention;

[0012] Using the current analyzed search intent, the current sentiment information, and the current predicted search intent of the user, multiple current search results searched based on the current search text are adjusted and sorted to obtain a final search result;

[0013] The final search result is output based on the current sentiment information.

[0014] Optionally, in the above-mentioned information search method, before the current search text and the current person image are put into a pre-trained convolutional neural network, the method further includes:

[0015] Extracting text features of the current search text using natural language processing technology; wherein the text features include keyword features and semantic features;

[0016] The current character image is processed according to the input format of the convolutional neural network.

[0017] Optionally, in the above-mentioned information search method, in the convolutional neural network pre-trained with the current search text and the current character image, the features of the current search text and the features of the current character image are fused into multimodal data through the convolutional neural network, and the current emotion information of the user is analyzed using the multimodal data, including:

[0018] Inputting the text features of the current search text and the processed current character image into the convolutional neural network;

[0019] The current character image is processed in sequence by multiple layers of feature processing layers in the convolutional neural network to obtain image features of the current character image;

[0020] By means of a feature fusion layer in the convolutional neural network, the image features of the current person image are fused with the text features of the current search text to obtain the multimodal data;

[0021] Through the fully connected layer in the convolutional neural network, the corresponding emotion category is mapped based on the multimodal data and output as the current emotion information of the user.

[0022] Optionally, in the above-mentioned information search method, inputting the historical search result sequence into a pre-trained prediction model, and predicting the user's search intention based on the historical search result sequence by the prediction model to obtain the user's current predicted search intention includes:

[0023] Using natural language processing technology, converting the historical search result sequence into a historical search result sequence vector;

[0024] The historical search result sequence vector is input into the prediction model, and the historical search result sequence vector is analyzed by the prediction model to predict the current predicted search intention of the user.

[0025] Optionally, in the above-mentioned information search method, the use of the current analyzed search intent, the current emotional information and the current predicted search intent of the user to adjust and sort multiple current search results searched based on the current search text to obtain the final search results includes:

[0026] Calculate the matching degree of each of the current search results selected and the current analyzed search intent, the current sentiment information, the current predicted search intent and the search text of the user;

[0027] Filter out the current search results that match any one of the current analyzed search intent, the current sentiment information, and the current predicted search intent of the user from among the current search results obtained based on the current search text;

[0028] Weighting the matching degree of each current search result with the current analyzed search intent of the user, the current sentiment information, the current predicted search intent and the search text to obtain a comprehensive matching degree of each current search result;

[0029] Sorting the current search results in descending order of comprehensive matching degree;

[0030] The current search results ranked in the top N positions are used as the final search results according to the sorting result.

[0031] Optionally, in the above-mentioned information search method, outputting the final search result based on the current emotion information includes:

[0032] The result feedback interface is rendered using the interface rendering element corresponding to the current emotion information, and the final search result is displayed through the result feedback interface.

[0033] The second aspect of the present application provides an information search system, comprising:

[0034] An information acquisition unit, used to acquire the user's current context information, the current search text input by the user, the user's current character image and a historical search result sequence; wherein the current context information includes current environment information and historical behavior information within a preset time period; the current character image includes a facial image and a posture image of the current character; the historical search result sequence includes the results searched and selected by the user at multiple consecutive time points before the current moment;

[0035] A context analysis unit, configured to analyze the current context information based on a context awareness mechanism to obtain the current analysis search intention of the user;

[0036] A sentiment analysis unit, configured to integrate the features of the current search text and the current character image into a pre-trained convolutional neural network, fuse the features of the current search text and the features of the current character image into multimodal data through the convolutional neural network, and analyze the current sentiment information of the user using the multimodal data;

[0037] A prediction unit, configured to input the historical search result sequence into a pre-trained prediction model, and predict the user's search intention based on the historical search result sequence through the prediction model to obtain the user's current predicted search intention;

[0038] A result optimization unit, configured to adjust and sort a plurality of current search results obtained based on the current search text by using the current analyzed search intent, the current emotional information and the current predicted search intent of the user, to obtain a final search result;

[0039] An output unit is used to output the final search result based on the current emotion information.

[0040] Optionally, the above information search system further includes:

[0041] A text feature extraction unit, used to extract text features of the current search text using natural language processing technology; wherein the text features include keyword features and semantic features;

[0042] A format processing unit is used to process the current character image according to the input format of the convolutional neural network.

[0043] Optionally, in the above-mentioned information search system, the sentiment analysis unit includes:

[0044] An input unit, used for inputting the text features of the current search text and the processed current character image into the convolutional neural network;

[0045] A feature processing unit, used to process the current person image in sequence through multiple layers of feature processing layers in the convolutional neural network to obtain image features of the current person image;

[0046] A fusion unit, configured to fuse the image features of the current person image with the text features of the current search text through a feature fusion layer in the convolutional neural network to obtain the multimodal data;

[0047] The emotion determination unit is used to map the corresponding emotion category based on the multimodal data through the fully connected layer in the convolutional neural network, and output it as the current emotion information of the user.

[0048] Optionally, in the above-mentioned information search system, the prediction unit includes:

[0049] A sequence processing unit, used to convert the historical search result sequence into a historical search result sequence vector by using natural language processing technology;

[0050] The intention prediction unit is used to input the historical search result sequence vector into the prediction model, analyze the historical search result sequence vector through the prediction model, and predict the user's current predicted search intention.

[0051] Optionally, in the above-mentioned information search system, the result optimization unit includes:

[0052] A matching degree calculation unit, used to calculate the matching degree of each of the screened current search results with the user's current analyzed search intent, the current emotional information, the current predicted search intent and the search text;

[0053] A primary screening unit, configured to screen out the current search results that match any one of the current analyzed search intent, the current emotional information, and the current predicted search intent of the user from among the current search results searched based on the current search text;

[0054] A comprehensive calculation unit, used to weight the matching degree of each current search result with the current analyzed search intent of the user, the current emotional information, the current predicted search intent and the search text, to obtain a comprehensive matching degree of each current search result;

[0055] A sorting unit, used to sort the current search results in descending order of comprehensive matching degree;

[0056] The result selection unit is used to select the current search results ranked in the top N positions as the final search results according to the sorting results.

[0057] Optionally, in the above-mentioned information search system, the output unit includes:

[0058] The output subunit is used to render the result feedback interface using the interface rendering element corresponding to the current emotion information, and to display the final search result through the result feedback interface.

[0059] A third aspect of the present application provides an electronic device, including:

[0060] Memory and processor;

[0061] Wherein, the memory is used to store programs;

[0062] The processor is used to execute the program, and when the program is executed, it is specifically used to implement any one of the information search methods described above.

[0063] A fourth aspect of the present application provides a computer storage medium for storing a computer program, wherein when the computer program is executed by a processor, it is used to implement the information search method as described in any one of the above.

[0064] The present application provides an information search method, which obtains the current context information of the user, the current search text input by the user, the current character image of the user, and the historical search result sequence. Among them, the current context information includes the current environment information and the historical behavior information within the current preset time period. The current character image includes the facial image and the posture image of the current character. The historical search result sequence includes the results searched and selected by the user at multiple consecutive time points before the current moment. Then, the current context information is analyzed based on the context perception mechanism to obtain the current analysis search intention of the user, so that the current search intention of the user can be analyzed according to the relationship between the current context information and the search intention. In addition, the features of the current search text and the features of the current character image are fused into multimodal data through the convolutional neural network in the pre-trained convolutional neural network, and the current emotional information of the user is analyzed by the multimodal data, so as to analyze the search content that meets the normal intention of the user from the emotional aspect. The historical search result sequence is also input into a pre-trained prediction model, and the search intention of the user is predicted based on the historical search result sequence by the prediction model to obtain the current predicted search intention of the user, so as to predict the intention of the user through the search rules of the user, so as to take into account the changes in the search needs of the user. Finally, the user's current analyzed search intent, current emotional information and current predicted search intent are used to adjust and sort the multiple current search results based on the current search text to obtain the final search results, and the final search results are output based on the current emotional information, thereby fully considering the current context information, user emotions and predicted user search intent, so as to effectively ensure that the search results are in line with the user's actual search intent, that is, to ensure the accuracy of the search results. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0066] Figure 1 A flowchart of an information search method provided in an embodiment of the present application;

[0067] Figure 2 A flowchart of a method for analyzing a user's current emotional information provided by an embodiment of the present application;

[0068] Figure 3 A flowchart of a method for predicting a user's current predicted search intention provided by an embodiment of the present application;

[0069] Figure 4 A flowchart of a method for adjusting and sorting current search results provided in an embodiment of the present application;

[0070] Figure 5 A schematic diagram of the architecture of an information search system provided in an embodiment of the present application;

[0071] Figure 6 A schematic diagram of the architecture of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0072] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0073] In this application, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0074] The present application embodiment provides an information search method, such as Figure 1 As shown, the specific steps include:

[0075] S101 : Acquire the current context information of the user, the current search text input by the user, the current character image of the user, and a sequence of historical search results.

[0076] First of all, it should be noted that obtaining the user's current context information, the current search text entered by the user, the user's current character image, and the historical search result sequence all require prompts to the user to clear and can only be obtained after the user's authorization.

[0077] Among them, the current context information includes current environment information and historical behavior information within the current preset time period. Since the relevant information of the device used and the actual environment can feedback the user's search intention, in an embodiment of the present application, the current environment information includes device environment information and actual environment information. For example, when a user is in a company and uses office equipment to search, he usually searches for work-related content. When a user is in a scenic spot and uses a personal private device to search, he may search for content related to travel guides. Among them, the device environment information includes but is not limited to the device type, the device connection network type, etc. The actual environment information includes geographic location, time, etc.

[0078] The historical behavior information is the device operation behavior performed by the user on the device before the current moment, which can also reflect the user's search intention to a certain extent.

[0079] Since emotion is also an important factor reflecting the user's search intention, for example, for the same input search text, when the user is in a relatively calm mood, he usually needs to search out the detailed content of the relevant information to be searched, so as to have an in-depth understanding of the relevant content, such as principles, specific operating methods, other replaceable methods, etc. When he is in a more nervous mood, he usually needs to search out simple and key content, for example, he only needs to quickly know how to operate, so as to quickly make a corresponding response using the searched content. Therefore, in an embodiment of the present application, the user's current character image is also obtained. Specifically, the current character image can be directly collected by the camera of the device after the user authorizes. Since human emotions are mainly reflected by facial expressions and human body postures, the current character image of the user obtained includes the facial image and posture image of the current character.

[0080] Taking into account the changes in the user's search needs, in order to analyze the changing patterns and extract and predict their possible search intentions, a historical search result sequence is also obtained, which includes the results searched and selected by the user at multiple consecutive time points before the current moment. And they are combined in the order of the search time points. Optionally, the continuous time points can be the time points for each search and selection of results, or they can be time points at regular intervals. And in order to share the patterns of the user's recent search content, the historical search result sequence obtained can be a sequence of recent historical search results.

[0081] S102: Analyze the current context information based on the context awareness mechanism to obtain the user's current analysis search intention.

[0082] It should be noted that steps S102, S103, and S104 are processes of performing corresponding analyses using the various types of information obtained in step S101, and the analysis processes of the various types of information are relatively independent. Therefore, in the embodiment of the present application, the execution order of steps S102, S103, and S104 is only an optional method, and other execution orders may also be used.

[0083] Optionally, the association between various types of context information and search intent can be analyzed in advance, so that the corresponding user's search intent can be determined based on the context perception mechanism by analyzing the association between the current context information and the search intent, thereby obtaining the user's current analyzed search intent. For example, when a user searches for information using office equipment in a company during working hours, the context perception mechanism analyzes that the user's search intent is to search for work-related content, and the specific intent of the user's desired search can be specifically analyzed.

[0084] S103, the features of the current search text and the current character image are integrated into a pre-trained convolutional neural network, and the features of the current search text and the current character image are integrated into multimodal data through the convolutional neural network, and the current emotional information of the user is analyzed using the multimodal data.

[0085] It should be noted that the tone, words, and specific content corresponding to the current search text input by the user may also reflect the user's emotions, and in order to make the emotions relevant to the searched information, that is, the analyzed emotions need to be a list that reflects the search intent. Therefore, in an embodiment of the present application, when analyzing the user's emotions, not only the character image but also the user's search text is considered.

[0086] Specifically, in the embodiment of the present application, the convolutional neural network is trained in advance using training samples consisting of collected search texts and character images, so that the convolutional neural network can be used to extract features of the input current search text and the current character image, and the extracted features are fused, and the conversation is based on the fused multimodal data to analyze the user's emotions.

[0087] Optionally, in another embodiment of the present application, before executing step S103, it is necessary to further execute:

[0088] Natural language processing technology is used to extract text features of the current search text, and the current character image is processed according to the input format of the convolutional neural network.

[0089] Among them, text features include keyword features and semantic features.

[0090] Specifically, natural language processing technology is used to extract the keywords and semantics of the current search text, and then it is processed to convert it into a feature vector in a format that can be processed by a convolutional neural network.

[0091] The current character image can also be resized and normalized to make it a format that can be processed by the convolutional neural network.

[0092] Accordingly, in another embodiment of the present application, a specific implementation of step S103 is as follows: Figure 2 As shown, the following steps are included:

[0093] S201, inputting the text features of the current search text and the processed current character image into a convolutional neural network.

[0094] S202, sequentially processing the current character image through the multiple layers of feature processing layers in the convolutional neural network to obtain image features of the current character image.

[0095] Among them, the feature processing layer may include a convolution layer, an activation layer, a pooling layer, and a flattening layer.

[0096] Specifically, the convolution layer can first apply filters to extract features from the image, and then the activation layer can apply nonlinear changes to the extracted feature data. Then, the feature data is downsampled through the pooling layer to reduce the problem of feature data. Finally, the feature data is converted into a one-dimensional array through the flattening layer.

[0097] S203, through the feature fusion layer in the convolutional neural network, the image features of the current person image are fused with the text features of the current search text to obtain multimodal data.

[0098] S204: Map the corresponding emotion category based on the multimodal data through the fully connected layer in the convolutional neural network, and output it as the user's current emotion information.

[0099] S104: Input the historical search result sequence into a pre-trained prediction model, and use the prediction model to predict the user's search intention based on the historical search result sequence to obtain the user's current predicted search intention.

[0100] Specifically, the historical search result sequence is input into a pre-trained prediction model, and the pre-trained prediction model is used to analyze the pattern of changes in the user's search results over time based on the historical search result sequence, so as to analyze the most likely search intention of the user at the current moment according to the pattern.

[0101] Optionally, the prediction model may specifically adopt a long short-term memory network.

[0102] Optionally, in another embodiment of the present application, a specific implementation of step S104 is as follows: Figure 3 As shown, the following steps are included:

[0103] S301: Combine historical search result sequences according to search time to obtain a historical search result sequence.

[0104] S302: Using natural language processing technology, convert the historical search result sequence into a historical search result sequence vector.

[0105] S303: Input the historical search result sequence vector into the prediction model, analyze the historical search result sequence vector through the prediction model, and predict the user's current predicted search intention.

[0106] S105: Using the user's current analyzed search intent, current emotional information, and current predicted search intent, adjust and sort multiple current search results based on the current search text to obtain final search results.

[0107] Optionally, the multiple current search results searched based on the current search text can specifically be based on the keywords in the current search text, and the results searched for that have a matching degree with the keywords in the current search text reaching a certain threshold, according to the multiple results with the highest matching degree. Of course, it is also possible to search based on the current search text using other algorithms. After searching for multiple current search results, the current search results are matched with the current analysis search intent, the current emotional information, and the current predicted search intent, and the current search results are further screened, and the current search results are sorted according to the matching degree with the current analysis search intent, the current emotional information, and the current predicted search intent, so as to obtain the final search results.

[0108] Optionally, in another embodiment of the present application, a specific implementation of step S105 is as follows: Figure 4 As shown, the following steps are included:

[0109] S401, calculating the matching degree between each current search result selected and the user's current analyzed search intent, current sentiment information, current predicted search intent and search text.

[0110] S402: Filter out the current search results that match any one of the user's current analyzed search intent, current sentiment information, and current predicted search intent from among the current search results obtained based on the current search text.

[0111] In order to speed up the screening efficiency, in the embodiment of the present application, the current search results that match any one of the user's current analytical search intent, current emotional information, and current predicted search intent are first screened out, and then further comprehensive consideration is performed. Optionally, matching with a certain item can specifically mean that the matching degree with a certain item is greater than a corresponding preset threshold.

[0112] S403: Weight the matching degree of each current search result with the user's current analyzed search intent, current sentiment information, current predicted search intent, and search text to obtain a comprehensive matching degree of each current search result.

[0113] The matching degree between the current search result and the search text is determined at the time of the search result.

[0114] In order to comprehensively consider the four factors and obtain accurate search results, in the embodiment of the present application, the matching degree between the current search result and the four items of information is weighted to obtain a comprehensive matching degree. The weight of each item of information can be set according to the requirements.

[0115] S404: Sort the current search results in descending order of comprehensive matching degree.

[0116] S405: The current search results ranked in the top N positions are used as the final search results according to the sorting results.

[0117] S106. Output the final search results based on the current sentiment information.

[0118] In order to make the output effect of the search results consistent with the user's current emotions, in an embodiment of the present application, the final search results are output in a corresponding manner based on the current emotional information.

[0119] Optionally, in another embodiment of the present application, a specific implementation of step S106 includes:

[0120] The result feedback interface is rendered using the interface rendering elements corresponding to the current emotional information, and the final search results are displayed through the result feedback interface.

[0121] The embodiment of the present application provides an information search method, which obtains the current context information of the user, the current search text input by the user, the current character image of the user, and the historical search result sequence. Among them, the current context information includes the current environment information and the historical behavior information within the current preset time period. The current character image includes the facial image and the posture image of the current character. The historical search result sequence includes the results searched and selected by the user at multiple consecutive time points before the current moment. Then, the current context information is analyzed based on the context perception mechanism to obtain the current analysis search intention of the user, so that the current search intention of the user can be analyzed according to the relationship between the current context information and the search intention. In addition, the features of the current search text and the features of the current character image are fused into multimodal data through the convolutional neural network in the pre-trained convolutional neural network, and the current emotional information of the user is analyzed by the multimodal data, so as to analyze the search content that meets the normal intention of the user from the emotional aspect. The historical search result sequence is also input into a pre-trained prediction model, and the user's search intention is predicted based on the historical search result sequence by the prediction model to obtain the user's current predicted search intention, so as to predict the user's intention through the user's search rules, so as to take into account the changes in the user's search needs. Finally, the user's current analyzed search intent, current emotional information and current predicted search intent are used to adjust and sort the multiple current search results based on the current search text to obtain the final search results, and the final search results are output based on the current emotional information, thereby fully considering the current context information, user emotions and predicted user search intent, so as to effectively ensure that the search results are in line with the user's actual search intent, that is, to ensure the accuracy of the search results.

[0122] Another embodiment of the present application provides an information search system, such as Figure 5 As shown, including:

[0123] The information acquisition unit 501 is used to acquire the user's current context information, the current search text input by the user, the user's current character image, and a historical search result sequence.

[0124] The current context information includes the current environment information and the historical behavior information within the preset time period. The current person image includes the face image and posture image of the current person. The historical search result sequence includes the results searched and selected by the user at multiple consecutive time points before the current moment.

[0125] The context analysis unit 502 is used to analyze the current context information based on the context awareness mechanism to obtain the user's current analysis search intention.

[0126] The sentiment analysis unit 503 is used to integrate the features of the current search text and the current character image into a pre-trained convolutional neural network, and use the convolutional neural network to fuse the features of the current search text and the features of the current character image into multimodal data, and use the multimodal data to analyze the user's current sentiment information.

[0127] The prediction unit 504 is used to input the historical search result sequence into a pre-trained prediction model, and predict the user's search intention based on the historical search result sequence through the prediction model to obtain the user's current predicted search intention.

[0128] The result optimization unit 505 is used to adjust and sort the multiple current search results based on the current search text by utilizing the user's current analysis search intent, current emotional information and current predicted search intent to obtain the final search results.

[0129] The output unit 506 is used to output the final search results based on the current sentiment information.

[0130] Optionally, in another embodiment of the present application, the information search system further includes:

[0131] The text feature extraction unit is used to extract text features of the current search text using natural language processing technology, wherein the text features include keyword features and semantic features.

[0132] The format processing unit is used to process the current character image according to the input format of the convolutional neural network.

[0133] Optionally, in the information search system provided in another embodiment of the present application, the sentiment analysis unit includes:

[0134] The input unit is used to input the text features of the current search text and the processed current character image into the convolutional neural network.

[0135] The feature processing unit is used to process the current character image in sequence through the multiple feature processing layers in the convolutional neural network to obtain the image features of the current character image.

[0136] The fusion unit is used to fuse the image features of the current person image with the text features of the current search text through a feature fusion layer in a convolutional neural network to obtain multimodal data.

[0137] The emotion determination unit is used to map the corresponding emotion category based on the multimodal data through the fully connected layer in the convolutional neural network, and output it as the user's current emotion information.

[0138] Optionally, in an information search system provided in another embodiment of the present application, the prediction unit includes:

[0139] The sequence processing unit is used to convert the historical search result sequence into a historical search result sequence vector by using natural language processing technology.

[0140] The intention prediction unit is used to input the historical search result sequence vector into the prediction model, analyze the historical search result sequence vector through the prediction model, and predict the user's current predicted search intention.

[0141] Optionally, in the information search system provided in another embodiment of the present application, the result optimization unit includes:

[0142] The matching degree calculation unit is used to calculate the matching degree between each current search result screened out and the user's current analyzed search intent, current emotional information, current predicted search intent and search text.

[0143] The primary screening unit is used to screen out the current search results that match any one of the user's current analyzed search intent, current emotional information, and current predicted search intent from the current search results searched based on the current search text.

[0144] The comprehensive calculation unit is used to weight the matching degree of each current search result with the user's current analyzed search intent, current emotional information, current predicted search intent and search text to obtain the comprehensive matching degree of each current search result.

[0145] The sorting unit is used to sort the current search results in descending order according to the comprehensive matching degree.

[0146] The result selection unit is used to select the current search results ranked in the top N positions as the final search results according to the sorting results.

[0147] Optionally, in an information search system provided in another embodiment of the present application, the output unit includes:

[0148] The output subunit is used to render the result feedback interface using the interface rendering elements corresponding to the current emotional information, and to display the final search results through the result feedback interface.

[0149] It should be noted that the specific working process of each unit provided in the above embodiments of the present application can refer to the implementation process of the corresponding steps in the above method embodiments, and will not be repeated here.

[0150] Another embodiment of the present application provides an electronic device, such as Figure 6 As shown, including:

[0151] Memory 601 and processor 602 .

[0152] The memory 601 is used to store programs.

[0153] The processor 602 is used to execute the program stored in the memory 601. When the program is executed, it is specifically used to implement the information search method provided by any one of the above embodiments.

[0154] Another embodiment of the present application provides a computer storage medium for storing a computer program. When the computer program is executed by a processor, it is used to implement the information search method provided in any of the above embodiments.

[0155] Computer storage media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0156] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0157] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An information search method, characterized in that: include: Acquire the user's current context information, the current search text input by the user, the user's current character image and a sequence of historical search results; wherein the current context information includes current environment information and historical behavior information within a preset time period; the current character image includes a facial image and a posture image of the current character; the sequence of historical search results includes the results searched and selected by the user at multiple consecutive time points before the current moment; Analyze the current context information based on a context awareness mechanism to obtain the current analyzed search intention of the user; In a convolutional neural network pre-trained for the current search text and the current character image, the features of the current search text and the features of the current character image are fused into multimodal data through the convolutional neural network, and the current emotion information of the user is analyzed using the multimodal data; Inputting the historical search result sequence into a pre-trained prediction model, and using the prediction model to predict the user's search intention based on the historical search result sequence to obtain the user's current predicted search intention; Using the current analyzed search intent, the current sentiment information, and the current predicted search intent of the user, multiple current search results searched based on the current search text are adjusted and sorted to obtain a final search result; The final search result is output based on the current sentiment information.

2. The method according to claim 1, characterized in that Before the current search text and the current person image are put into a pre-trained convolutional neural network, the method further includes: Extracting text features of the current search text using natural language processing technology; wherein the text features include keyword features and semantic features; The current character image is processed according to the input format of the convolutional neural network.

3. The method according to claim 1, characterized in that In the convolutional neural network pre-trained with the current search text and the current character image, the features of the current search text and the features of the current character image are fused into multimodal data through the convolutional neural network, and the current emotion information of the user is analyzed using the multimodal data, including: Inputting the text features of the current search text and the processed current character image into the convolutional neural network; The current character image is processed in sequence by multiple layers of feature processing layers in the convolutional neural network to obtain image features of the current character image; By means of a feature fusion layer in the convolutional neural network, the image features of the current person image are fused with the text features of the current search text to obtain the multimodal data; Through the fully connected layer in the convolutional neural network, the corresponding emotion category is mapped based on the multimodal data and output as the current emotion information of the user.

4. The method according to claim 1, characterized in that The step of inputting the historical search result sequence into a pre-trained prediction model, and predicting the user's search intention based on the historical search result sequence by the prediction model to obtain the user's current predicted search intention includes: Using natural language processing technology, converting the historical search result sequence into a historical search result sequence vector; The historical search result sequence vector is input into the prediction model, and the historical search result sequence vector is analyzed by the prediction model to predict the current predicted search intention of the user.

5. The method according to claim 1, characterized in that The method of adjusting and sorting a plurality of current search results obtained based on the current search text by using the current analyzed search intent, the current emotional information, and the current predicted search intent of the user to obtain a final search result includes: Calculate the matching degree of each of the current search results selected and the current analyzed search intent, the current sentiment information, the current predicted search intent and the search text of the user; Filter out the current search results that match any one of the current analyzed search intent, the current sentiment information, and the current predicted search intent of the user from among the current search results obtained based on the current search text; Weighting the matching degree of each current search result with the current analyzed search intent of the user, the current sentiment information, the current predicted search intent and the search text to obtain a comprehensive matching degree of each current search result; Sorting the current search results in descending order of comprehensive matching degree; The current search results ranked in the top N positions are used as the final search results according to the sorting result.

6. The method according to claim 1, characterized in that The outputting the final search result based on the current emotion information comprises: The result feedback interface is rendered using the interface rendering element corresponding to the current emotion information, and the final search result is displayed through the result feedback interface.

7. An information search system, characterized in that: include: An information acquisition unit, used to acquire the user's current context information, the current search text input by the user, the user's current character image and a historical search result sequence; wherein the current context information includes current environment information and historical behavior information within a preset time period; the current character image includes a facial image and a posture image of the current character; the historical search result sequence includes the results searched and selected by the user at multiple consecutive time points before the current moment; A context analysis unit, configured to analyze the current context information based on a context awareness mechanism to obtain the current analysis search intention of the user; A sentiment analysis unit, configured to integrate the features of the current search text and the current character image into a pre-trained convolutional neural network, fuse the features of the current search text and the features of the current character image into multimodal data through the convolutional neural network, and analyze the current sentiment information of the user using the multimodal data; A prediction unit, configured to input the historical search result sequence into a pre-trained prediction model, and predict the user's search intention based on the historical search result sequence through the prediction model to obtain the user's current predicted search intention; A result optimization unit, configured to adjust and sort a plurality of current search results obtained based on the current search text by using the current analyzed search intent, the current emotional information and the current predicted search intent of the user, to obtain a final search result; An output unit is used to output the final search result based on the current emotion information.

8. The system according to claim 7, characterized in that Also includes: A text feature extraction unit, used to extract text features of the current search text using natural language processing technology; wherein the text features include keyword features and semantic features; A format processing unit is used to process the current character image according to the input format of the convolutional neural network.

9. An electronic device, characterized in that: include: Memory and processor; Wherein, the memory is used to store programs; The processor is used to execute the program, and when the program is executed, it is specifically used to implement the information search method according to any one of claims 1 to 6.

10. A computer storage medium, characterized in that: Used to store a computer program, which, when executed by a processor, is used to implement the information search method according to any one of claims 1 to 6.

Images