Message reply method, device, storage medium and electronic device

Abstract

The present disclosure relates to a message reply method, device, storage medium and electronic device, the method comprising: obtaining a user message in a human-computer dialogue; determining multiple candidate emoticon packages from a preset emoticon package library according to the semantic information corresponding to the user message; inputting the candidate emoticon packages, the candidate text description information corresponding to each candidate emoticon package and the user message into a pre-trained message matching model to obtain an emoticon package matching value corresponding to each candidate emoticon package; determining a target emoticon package from multiple candidate emoticon packages according to the emoticon package matching value corresponding to the candidate emoticon package; outputting the target emoticon package to reply to the user message. That is, when obtaining a target emoticon package that matches the user message, the present disclosure combines the candidate text description information corresponding to the candidate emoticon package, so that the accuracy of the matched target emoticon package is higher, thereby improving the accuracy of the message reply and the effect of human-computer interaction.

Description

Technical Field

[0001] The present disclosure relates to the field of human-computer interaction technology, and in particular to a message reply method, device, storage medium and electronic device. Background Art

[0002] With the development of the Internet, information communication and artificial intelligence technology, the inherent natural convenience of human-computer interaction makes it the new generation of interaction paradigm in the future. It can be applied to website platforms, instant messaging applications, chat rooms and social robots.

[0003] In traditional human-computer interaction, machines mostly use text content to chat with users. However, "chat emoticons" have become an indispensable chat element in the current conversation and chat process. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present disclosure provides a message reply method, device, storage medium and electronic device.

[0005] According to a first aspect of an embodiment of the present disclosure, a message reply method is provided, the method comprising:

[0006] Get user messages in human-computer dialogue;

[0007] Determining a plurality of candidate emoticon packages from a preset emoticon package library according to the semantic information corresponding to the user message;

[0008] Inputting a plurality of candidate emoticon packages, candidate text description information corresponding to each candidate emoticon package, and the user message into a pre-trained message matching model to obtain an emoticon package matching value corresponding to each candidate emoticon package;

[0009] Determining a target emoticon package from a plurality of candidate emoticon packages according to the emoticon package matching values ​​corresponding to the candidate emoticon packages;

[0010] Output the target emoticon package to reply to the user message.

[0011] In some embodiments, before outputting the target emoticon package, the method further includes:

[0012] Determining a plurality of candidate text messages from a preset corpus according to the user message;

[0013] Inputting a plurality of the candidate text messages and the user message into the message matching model to obtain a text matching value corresponding to each of the candidate text messages;

[0014] Determine a target reply message from a plurality of candidate emoticons and a plurality of candidate text messages according to the plurality of emoticon package matching values ​​and the plurality of text matching values, wherein the target reply message includes a target emoticon package or a target text message;

[0015] The outputting the target expression package comprises:

[0016] When the target reply message is the target emoticon package, the target emoticon package is output.

[0017] In some embodiments, the method further comprises:

[0018] In a case where the target reply message is the target text message, the target text message is output.

[0019] In some embodiments, determining a target reply message from a plurality of candidate emoticons and a plurality of candidate text messages according to a plurality of emoticon matching values ​​and a plurality of text matching values ​​comprises:

[0020] Determine a maximum matching value among a plurality of the emoticon matching values ​​and a plurality of the text matching values;

[0021] The target candidate emoticon package or target candidate text message corresponding to the maximum matching value is used as the target reply message.

[0022] In some embodiments, the preset expression package library is pre-created in the following manner:

[0023] Get the preset emoticon package;

[0024] Determining, according to the type of each preset emoticon package, preset text description information corresponding to the preset emoticon package;

[0025] A correspondence between each of the preset emoticon packages and the preset text description information is created to obtain the preset emoticon package library.

[0026] In some embodiments, the type of the preset emoticon package includes a static picture emoticon package; and determining the preset text description information corresponding to the preset emoticon package includes:

[0027] Determining whether the preset emoticon package contains a preset emoticon package code;

[0028] In the case where the preset emoticon package includes the preset emoticon package code, using the preset emoticon package code as the preset text description information;

[0029] In the case that the preset emoticon package does not include the preset emoticon package code, label information corresponding to the preset emoticon package is obtained through a pre-trained label recognition model, and the label information is used as the preset text description information.

[0030] In some embodiments, the type of the preset emoticon package includes a picture-text combined emoticon package; and determining the preset text description information corresponding to the preset emoticon package includes:

[0031] Obtaining text information in the preset emoticon package;

[0032] The text information is used as the preset text description information.

[0033] In some embodiments, the type of the preset emoticon package includes a dynamic picture emoticon package; and determining the preset text description information corresponding to the preset emoticon package includes:

[0034] Obtaining frame text description information corresponding to each frame of the image in the preset emoticon package;

[0035] The preset text description information is determined according to the plurality of frame text description information.

[0036] In some embodiments, the message matching model is trained in the following manner:

[0037] Acquire an emoticon sample set and a text sample set, wherein the emoticon sample set includes a plurality of emoticon samples, sample text description information corresponding to each of the emoticon samples, and an emoticon sample matching value corresponding to each of the emoticon samples, and the text sample set includes a plurality of groups of text dialogue samples and a text sample matching value corresponding to each group of the text dialogue samples;

[0038] The target neural network model is trained by using the expression package sample set and the text sample set to obtain the message matching model.

[0039] In some embodiments, the training of the target neural network model by using the expression package sample set and the text sample set to obtain the message matching model includes:

[0040] According to the multiple emoticon samples, the multiple sample text description information, the multiple emoticon sample matching values, the multiple groups of text conversation samples and the multiple text sample matching values, the target neural network model is iteratively trained through the preset loss function of the target neural network model to obtain the message matching model.

[0041] In some embodiments, determining a plurality of candidate emoticon packages from a preset emoticon package library according to the user message includes:

[0042] Determine whether the user message is an emoticon package;

[0043] In the case where the user message is an emoticon package, determining emoticon text description information corresponding to the user message;

[0044] According to the semantic information corresponding to the emoticon text description information, determining a plurality of candidate text description information matching the semantic information from the preset emoticon package library;

[0045] The emoticon package corresponding to each candidate text description information is used as the candidate emoticon package.

[0046] According to a second aspect of an embodiment of the present disclosure, a message reply device is provided, the device comprising:

[0047] A user message acquisition module, configured to acquire user messages in a human-computer dialogue;

[0048] A candidate emoticon package determination module is configured to determine a plurality of candidate emoticon packages from a preset emoticon package library according to semantic information corresponding to the user message;

[0049] An emoticon package matching value acquisition module is configured to input a plurality of candidate emoticons, candidate text description information corresponding to each candidate emoticon package, and the user message into a pre-trained message matching model to obtain an emoticon package matching value corresponding to each candidate emoticon package;

[0050] An expression package determination module is configured to determine a target expression package from a plurality of candidate expression packages according to expression package matching values ​​corresponding to the candidate expression packages;

[0051] The emoticon package output module is configured to output the target emoticon package to reply to the user message.

[0052] In some embodiments, the apparatus further comprises:

[0053] a candidate text message determination module, configured to determine a plurality of candidate text messages from a preset corpus according to the user message;

[0054] A text matching value acquisition module is configured to input a plurality of the candidate text messages and the user message into the message matching model to obtain a text matching value corresponding to each of the candidate text messages;

[0055] A target reply message determination module is configured to determine a target reply message from a plurality of candidate emoticons and a plurality of candidate text messages according to a plurality of emoticon package matching values ​​and a plurality of text matching values, wherein the target reply message includes a target emoticon package or a target text message;

[0056] The expression package output module is further configured as follows:

[0057] When the target reply message is the target emoticon package, the target emoticon package is output.

[0058] In some embodiments, the apparatus further comprises:

[0059] The text message output module is configured to output the target text message when the target reply message is the target text message.

[0060] In some embodiments, the target reply message determination module is further configured to:

[0061] Determine a maximum matching value among a plurality of the emoticon matching values ​​and a plurality of the text matching values;

[0062] The target candidate emoticon package or target candidate text message corresponding to the maximum matching value is used as the target reply message.

[0063] In some embodiments, the apparatus further comprises:

[0064] The expression library creation module is configured as follows:

[0065] Get the preset emoticon package;

[0066] Determining, according to the type of each preset emoticon package, preset text description information corresponding to the preset emoticon package;

[0067] A correspondence between each of the preset emoticon packages and the preset text description information is created to obtain the preset emoticon package library.

[0068] In some embodiments, the type of the preset emoticon package includes a static picture emoticon package; the emoticon package library creation module is further configured to:

[0069] Determining whether the preset emoticon package contains a preset emoticon package code;

[0070] In the case where the preset emoticon package includes the preset emoticon package code, using the preset emoticon package code as the preset text description information;

[0071] In the case that the preset emoticon package does not include the preset emoticon package code, label information corresponding to the preset emoticon package is obtained through a pre-trained label recognition model, and the label information is used as the preset text description information.

[0072] In some embodiments, the type of the preset emoticon package includes an image-text combined emoticon package; the emoticon package library creation module is further configured to:

[0073] Obtaining text information in the preset emoticon package;

[0074] The text information is used as the preset text description information.

[0075] In some embodiments, the type of the preset emoticon package includes a dynamic picture emoticon package; the emoticon package library creation module is further configured to:

[0076] Obtaining frame text description information corresponding to each frame of the image in the preset emoticon package;

[0077] The preset text description information is determined according to the plurality of frame text description information.

[0078] In some embodiments, the apparatus further comprises:

[0079] The model training module is configured as follows:

[0080] Acquire an emoticon sample set and a text sample set, wherein the emoticon sample set includes a plurality of emoticon samples, sample text description information corresponding to each of the emoticon samples, and an emoticon sample matching value corresponding to each of the emoticon samples, and the text sample set includes a plurality of groups of text dialogue samples and a text sample matching value corresponding to each group of the text dialogue samples;

[0081] The target neural network model is trained by using the expression package sample set and the text sample set to obtain the message matching model.

[0082] In some embodiments, the model training module is further configured to:

[0083] According to the multiple emoticon samples, the multiple sample text description information, the multiple emoticon sample matching values, the multiple groups of text conversation samples and the multiple text sample matching values, the target neural network model is iteratively trained through the preset loss function of the target neural network model to obtain the message matching model.

[0084] In some embodiments, the candidate expression package determination module is further configured to:

[0085] Determine whether the user message is an emoticon package;

[0086] In the case where the user message is an emoticon package, determining emoticon text description information corresponding to the user message;

[0087] According to the semantic information corresponding to the emoticon text description information, determining a plurality of candidate text description information matching the semantic information from the preset emoticon package library;

[0088] The emoticon package corresponding to each candidate text description information is used as the candidate emoticon package.

[0089] According to a third aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the message reply method provided in the first aspect of the present disclosure are implemented.

[0090] According to a fourth aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0091] a memory having a computer program stored thereon;

[0092] A processor is used to execute the computer program in the memory to implement the steps of the message reply method provided in the first aspect of the present disclosure.

[0093] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: by obtaining user messages in human-computer dialogue; according to the semantic information of the user message, multiple candidate emoticon packages are determined from a preset emoticon package library; multiple candidate emoticon packages, candidate text description information corresponding to each candidate emoticon package, and the user message are input into a pre-trained message matching model to obtain an emoticon package matching value corresponding to each candidate emoticon package; according to the emoticon package matching value corresponding to the candidate emoticon package, a target emoticon package is determined from multiple candidate emoticon packages; the target emoticon package is output to reply to the user message. That is to say, when obtaining the target emoticon package that matches the user message, the present disclosure combines the candidate text description information corresponding to the candidate emoticon package, so that the accuracy of the matched target emoticon package is higher, thereby improving the accuracy of the message reply and improving the effect of human-computer interaction.

[0094] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0095] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0096] Figure 1 is a flow chart of a message reply method according to an exemplary embodiment of the present disclosure;

[0097] Figure 2 is a flow chart of a method for creating a preset emoticon package library according to an exemplary embodiment of the present disclosure;

[0098] Figure 3 is a flowchart of a method for training a message matching model according to an exemplary embodiment of the present disclosure;

[0099] Figure 4 is a flowchart of another message reply method according to an exemplary embodiment of the present disclosure;

[0100] Figure 5 is a block diagram of a message reply device according to an exemplary embodiment of the present disclosure;

[0101] Figure 6 is a block diagram of another message reply device according to an exemplary embodiment of the present disclosure;

[0102] Figure 7 It is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0103] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0104] First, the application scenario of the present disclosure is described. The human-computer dialogue in the embodiment of the present disclosure can be a voice dialogue or a text dialogue. In the voice dialogue scenario, the user can send a voice message to the electronic device, and the electronic device can use the message reply method of the embodiment of the present disclosure to process the voice message and generate a corresponding reply.

[0105] In a text conversation scenario, a user may input text conversation information into the human-computer interaction interface of a client of an electronic device, and the electronic device may process the text information using the message reply method of an embodiment of the present disclosure and generate a corresponding reply.

[0106] The present disclosure is described below in conjunction with specific embodiments.

[0107] Figure 1 is a flowchart of a message reply method according to an exemplary embodiment of the present disclosure. Figure 1 As shown, the method can be applied to electronic devices, such as mobile terminals or servers. Mobile terminals include mobile phones, notebooks, tablet computers, desktop computers, smart speakers, smart TVs, wearable devices, smart robots, vehicle terminals, and terminals equipped with smart voice assistants. Servers include local servers and cloud servers.

[0108] The electronic device may also be an electronic device in a server. That is, a local smart speaker or smart terminal may complete the function of collecting audio data or text data, thereby sending the audio data or text data to the server for processing, and after receiving the reply information, the server sends the reply information to the local smart speaker or smart terminal for output.

[0109] The method includes:

[0110] S101, obtaining a user message in a human-computer dialogue.

[0111] The user message may be a text message, a voice message, or an emoticon package, which is not limited in the present disclosure.

[0112] In this step, the user may input the user message through the human-computer interaction interface of the client of the electronic device, or may send the user message to the electronic device through other electronic devices, which is not limited in the present disclosure.

[0113] S102: Determine a plurality of candidate emoticon packages from a preset emoticon package library according to semantic information corresponding to the user message.

[0114] in, Figure 2 is a flowchart of a method for creating a preset expression package library according to an exemplary embodiment of the present disclosure. Figure 2 As shown, the method may include:

[0115] S1021. Obtain a preset emoticon package.

[0116] The preset emoticon package may be a collection of multiple commonly used emoticon packages. The preset emoticon package may include only pictures, pictures and text, or dynamic pictures, such as gif pictures, which is not limited in the present disclosure.

[0117] S1022. Determine, according to the type of each preset emoticon package, preset text description information corresponding to the preset emoticon package.

[0118] After the preset emoticon package is obtained, for each preset emoticon package, the preset text description information corresponding to the preset emoticon package can be determined in different ways according to the type of the preset emoticon package.

[0119] In the case where the type of the preset emoticon package includes a static picture emoticon package, it can be determined whether the preset emoticon package contains a preset emoticon package code. The preset emoticon package code can be added when the preset emoticon package is created. In the case where the preset emoticon package contains the preset emoticon package code, the preset emoticon package code is used as the preset text description information; in the case where the preset emoticon package does not contain the preset emoticon package code, the label information corresponding to the preset emoticon package is obtained through a pre-trained label recognition model, and the label information is used as the preset text description information. The label information can be the type, emotion, etc. corresponding to the preset emoticon package. For example, the label information can be an animal, such as a cat, a dog, etc. The label information can also be a person, such as a little boy, a little girl, an old man, etc. The label information can also be happy, sad, etc. The specific content of the label information is not limited in the present disclosure.

[0120] In the case where the tag information includes multiple items, the multiple items of tag information can be combined to obtain the preset text description information.

[0121] It should be noted that the label recognition model can be obtained by training using a model training method in the prior art, which will not be described in detail here.

[0122] When the type of the preset emoticon package includes a graphic and text combined emoticon package, the text information in the preset emoticon package can be obtained and used as the preset text description information. For example, the text information in the preset emoticon package can be obtained by OCR (Optical Character Recognition) technology.

[0123] In the case that the type of the preset emoticon package includes a dynamic picture emoticon package, the frame text description information corresponding to each frame of the picture in the preset emoticon package can be obtained, and the preset text description information can be determined based on multiple frame text description information.

[0124] For example, each frame of the picture in the preset emoticon package can be obtained first, and then for each frame of the picture, the type of the frame of the picture can be determined. When the frame of the picture only includes the picture, it can be determined whether the frame of the picture contains the preset emoticon package code. The preset emoticon package code can be added when the frame of the picture is created. When the frame of the picture contains the preset emoticon package code, the preset emoticon package code is used as the frame text description information corresponding to the frame of the picture; when the frame of the picture does not contain the preset emoticon package code, the label information corresponding to the frame of the picture is obtained through a pre-trained label recognition model, and the label information is used as the frame text description information.

[0125] In the case that the frame of picture includes a picture and text, the text information in the frame of picture may be obtained and used as the text description information of the frame.

[0126] Furthermore, after obtaining the frame text description information corresponding to each frame of the picture, multiple frame text description information may be combined to obtain combined frame text description information, and after filtering out repeated content in the combined frame text description information, the preset text description information may be obtained.

[0127] S1023: Create a corresponding relationship between each preset emoticon package and the preset text description information to obtain the preset emoticon package library.

[0128] After determining the preset text description information corresponding to the preset emoticon package, a corresponding relationship between each preset emoticon package and the preset text description information corresponding to the preset emoticon package may be created to obtain the preset emoticon package library.

[0129] It should be noted that during the human-computer dialogue process, if the received user message is an unknown emoticon package, that is, the emoticon package is not included in the preset emoticon package library, the preset emoticon package library can be updated and the unknown emoticon package can be added to the preset emoticon package library, so that the expressions of the preset emoticon package library are richer, further improving the user experience of human-computer interaction.

[0130] In addition, the preset expression package library can be pre-created by the electronic device or by a cloud server, which is not limited in the present disclosure. In the case where the preset expression package library is pre-created by a cloud server, the electronic device can periodically obtain the preset expression package library from the cloud server.

[0131] In this step, after receiving the user message, multiple candidate emoticon packages matching the user message can be determined from the preset emoticon package library according to the user message. For example, when the user message is a text message, multiple candidate emoticon packages matching the user message can be determined according to the keywords of the user message; when the user message is a voice message, the voice message can be converted into a text message by the method of the prior art, and then multiple candidate emoticon packages matching the user message can be determined; when the user message is an emoticon package, the emoticon text description information corresponding to the user message can be obtained by referring to the above method for obtaining preset text description information, and then multiple candidate emoticon packages matching the user message can be determined according to the keywords of the emoticon text description information.

[0132] S103, inputting a plurality of candidate emoticon packages, candidate text description information corresponding to each candidate emoticon package, and the user message into a pre-trained message matching model to obtain an emoticon package matching value corresponding to each candidate emoticon package.

[0133] in, Figure 3 is a flowchart of a method for training a message matching model according to an exemplary embodiment of the present disclosure, such as Figure 3 As shown, the method may include:

[0134] S1. Obtain an emoticon sample set and a text sample set.

[0135] The emoticon package sample set includes a plurality of emoticon package samples, sample text description information corresponding to each emoticon package sample, and an emoticon package sample matching value corresponding to each emoticon package sample, and the text sample set includes a plurality of groups of text dialogue samples and a text sample matching value corresponding to each group of text dialogue samples. The emoticon package sample can be an emoticon package in the preset emoticon package library or other emoticon packages, which is not limited in the present disclosure. Each group of dialogue samples can include a user message and a reply message of the electronic device to the user message.

[0136] The emoticon sample matching value corresponding to the emoticon sample can be predetermined based on the matching relationship between the emoticon sample and the sample text description information corresponding to the emoticon sample. For example, for an emoticon sample and sample text description information with a high matching degree, the emoticon sample matching value can be 1, and for an emoticon sample and sample text description information with a low matching degree, the emoticon sample matching value can be 0. Similarly, the text sample matching value can also be determined based on the matching degree of the text conversation sample. For example, for a text conversation sample with a high matching degree, for example, the electronic device's reply message to the user's message is relatively accurate, the text sample matching value can be 1; for a text conversation sample with a low matching degree, for example, the electronic device's reply message to the user's message is inaccurate, the text sample matching value can be 0.

[0137] It should be noted that the above-mentioned emoticon sample matching value and text sample matching value are examples, and the emoticon sample matching value and the text sample matching value may also be other values, such as 0.9, 0.1, etc., and the present disclosure does not limit this.

[0138] In a possible implementation, the emoticon package sample set may include an emoticon package positive sample set and an emoticon package negative sample set, wherein the emoticon package samples in the emoticon package positive sample set have a higher matching degree with the sample text description information, wherein the emoticon package sample matching value is 1, and the emoticon package samples in the emoticon package negative sample set have a lower matching degree with the sample text description information, wherein the emoticon package sample matching value is 0. The text sample set may include a text positive sample set and a text negative sample set, wherein the text conversation samples in the text positive sample set have a higher matching degree, wherein the text sample matching value is 1, and the text conversation samples in the text negative sample set have a lower matching degree, wherein the text sample matching value is 0.

[0139] S2. The target neural network model is trained by using the emoticon sample set and the text sample set to obtain the message matching model.

[0140] Among them, the target neural network model can be a multimodal neural network model. By way of example, the target neural network model can be a VisualBERT multimodal model.

[0141] After obtaining the emoticon sample set and the text sample set, multiple emoticon samples, multiple sample text description information, multiple emoticon sample matching values ​​in the emoticon sample set, and multiple groups of text dialogue samples and multiple text sample matching values ​​in the text sample set can be input into the target neural network model, and the target neural network model is iteratively trained through the preset loss function of the target neural network model to obtain the message matching model. The preset loss function can be a commonly used loss function in the prior art, and the present disclosure does not limit this.

[0142] It should be noted that the method of iteratively training the target neural network model through the preset loss function of the target neural network model can refer to the model training method of the prior art, which will not be repeated here.

[0143] S104: Determine a target emoticon package from multiple candidate emoticon packages according to the emoticon package matching value corresponding to the candidate emoticon package.

[0144] In this step, after obtaining the emoticon package matching value corresponding to each candidate emoticon package, the maximum target emoticon package matching value can be determined from the emoticon package matching values, and the candidate emoticon package corresponding to the target emoticon package matching value is used as the target emoticon package.

[0145] S105: Output the target emoticon package to reply to the user message.

[0146] In this step, after determining the target emoticon package, the target emoticon package can be output according to the way of receiving the user message. For example, if the electronic device receives the user message through the target application, the target emoticon package can be output below the user message in the target application.

[0147] By adopting the above method, when obtaining the target emoticon package that matches the user message, the candidate text description information corresponding to the candidate emoticon package is combined, so that the accuracy of the matched target emoticon package is higher, thereby improving the accuracy of the message reply and enhancing the effect of human-computer interaction.

[0148] Figure 4 is a flowchart of another message reply method according to an exemplary embodiment of the present disclosure, such as Figure 4 As shown, the method includes:

[0149] S401, obtaining a user message in a human-computer dialogue.

[0150] The user message may be a text message, a voice message, or an emoticon package, which is not limited in the present disclosure.

[0151] S402: Determine whether the user message is an emoticon package.

[0152] S403: When the user message is an emoticon package, determine the emoticon text description information corresponding to the user message.

[0153] In this step, when the user message is an emoticon package, the emoticon text description information corresponding to the user message can be determined by referring to the method of determining the preset text description information corresponding to the preset emoticon package in step S102, which will not be repeated here.

[0154] S404: According to the semantic information corresponding to the emoticon text description information, determine a plurality of candidate text description information matching the semantic information from the preset emoticon package library.

[0155] In this step, after obtaining the emoticon text description information corresponding to the user message, the emoticon text description information can be semantically parsed to obtain at least one keyword corresponding to the emoticon text description information, and the at least one keyword is used as the feature information corresponding to the user message. For example, the present disclosure can determine at least one keyword corresponding to the emoticon text description information by a semantic parsing method of the prior art, which will not be repeated here.

[0156] It should be noted that, when the user message is a text message, the user message can be directly semantically parsed to obtain at least one keyword corresponding to the user message, and the at least one keyword is used as feature information corresponding to the user message.

[0157] Further, after determining the characteristic information corresponding to the user message, a plurality of candidate text description information matching the characteristic information can be determined by a semantic matching algorithm of the prior art. The number of the candidate text description information can be preset, for example, the number of the candidate text description information can be 5, which is not limited in the present disclosure.

[0158] S405: Use the emoticon package corresponding to each candidate text description information as the candidate emoticon package.

[0159] S406, inputting a plurality of candidate emoticon packages, candidate text description information corresponding to each candidate emoticon package, and the user message into a pre-trained message matching model to obtain an emoticon package matching value corresponding to each candidate emoticon package.

[0160] S407: Determine a plurality of candidate text messages from a preset corpus according to the user message.

[0161] The preset corpus may be a corpus of the prior art, and the preset corpus may include multiple groups of dialogue text dialogues.

[0162] In this step, a plurality of candidate text messages matching the user message can be determined from the preset corpus by using a semantic matching algorithm in the prior art. The number of the candidate text messages can be preset, for example, the number of the candidate text messages can be 4, which is not limited in the present disclosure.

[0163] S408: Input multiple candidate text messages and the user message into the message matching model to obtain a text matching value corresponding to each candidate text message.

[0164] It should be noted that the above steps S407 to S408 may also be performed before steps S402 to S406, and the present disclosure does not limit this.

[0165] S409: Determine a target reply message from a plurality of candidate emoticon packages and a plurality of candidate text messages according to a plurality of emoticon package matching values ​​and a plurality of text matching values.

[0166] The target reply message may include a target emoticon package or a target text message.

[0167] In this step, after obtaining multiple emoticon package matching values ​​and multiple text matching values, the maximum matching value among the multiple emoticon package matching values ​​and the multiple text matching values ​​can be determined, and the target candidate emoticon package or target candidate text message corresponding to the maximum matching value is used as the target reply message. For example, if the maximum matching value is the target emoticon package matching value, the target emoticon package corresponding to the target emoticon package matching value is used as the target reply message; if the maximum matching value is the target text matching value, the target text message corresponding to the target text matching value is used as the target reply message.

[0168] S410, determining whether the target reply message is the target emoticon package, and if the target reply message is the target emoticon package, executing step S411, and if the target reply message is not the target emoticon package, executing step S412.

[0169] S411, output the target emoticon package.

[0170] In this step, after the target emoticon package is output, the target text description information corresponding to the target emoticon package can be obtained, and the text description information can be played in the form of voice, which can increase the fun of interaction between people.

[0171] S412: When the target reply message is the target text message, output the target text message.

[0172] In this step, after the target text message is output, the target text message may also be played in the form of voice.

[0173] By adopting the above method, when obtaining the target emoticon package that matches the user message, the candidate text description information corresponding to the candidate emoticon package is combined, so that the accuracy of the matched target emoticon package is higher, thereby improving the effect of human-computer interaction. In addition, the target emoticon package disclosed in the present invention can be a picture, or a picture and text, or a dynamic picture, so that the emoticon package replied by the electronic device is richer; further, the present invention can reply to both emoticon packages and text messages, making the human-computer interaction method more flexible and further improving the user experience.

[0174] Figure 5 is a block diagram of a message reply device according to an exemplary embodiment of the present disclosure. Figure 5 As shown, the device comprises:

[0175] The user message acquisition module 501 is configured to acquire user messages in the human-computer dialogue;

[0176] The candidate emoticon package determination module 502 is configured to determine a plurality of candidate emoticon packages from a preset emoticon package library according to the semantic information corresponding to the user message;

[0177] The emoticon package matching value acquisition module 503 is configured to input the plurality of candidate emoticons, the candidate text description information corresponding to each candidate emoticon package, and the user message into a pre-trained message matching model to obtain an emoticon package matching value corresponding to each candidate emoticon package;

[0178] The expression package determination module 504 is configured to determine a target expression package from a plurality of candidate expression packages according to the expression package matching value corresponding to the candidate expression package;

[0179] The emoticon package output module 505 is configured to output the target emoticon package to reply the user message.

[0180] In some embodiments, Figure 6 is a block diagram of another message reply device according to an exemplary embodiment of the present disclosure, such as Figure 6 As shown, the device also includes:

[0181] A candidate text message determination module 506 is configured to determine a plurality of candidate text messages from a preset corpus according to the user message;

[0182] The text matching value acquisition module 507 is configured to input the plurality of candidate text messages and the user message into the message matching model to obtain a text matching value corresponding to each candidate text message;

[0183] A target reply message determination module 508 is configured to determine a target reply message from a plurality of candidate emoticons and a plurality of candidate text messages according to a plurality of emoticon package matching values ​​and a plurality of text matching values, wherein the target reply message includes a target emoticon package or a target text message;

[0184] The expression package output module 505 is further configured to:

[0185] When the target reply message is the target emoticon package, the target emoticon package is output.

[0186] In some embodiments, the apparatus further comprises:

[0187] The text message output module 509 is configured to output the target text message when the target reply message is the target text message.

[0188] In some embodiments, the target reply message determination module 508 is further configured to:

[0189] Determine a maximum matching value among a plurality of matching values ​​of the emoticon package and a plurality of matching values ​​of the text;

[0190] The target candidate emoticon package or target candidate text message corresponding to the maximum matching value is used as the target reply message.

[0191] In some embodiments, the apparatus further comprises:

[0192] The expression package library creation module 510 is configured to:

[0193] Get the preset emoticon package;

[0194] Determine, according to the type of each preset emoticon package, preset text description information corresponding to the preset emoticon package;

[0195] A correspondence between the preset emoticon package and the preset text description information is created to obtain the preset emoticon package library.

[0196] In some embodiments, the type of the preset emoticon package includes a static picture emoticon package; the emoticon package library creation module 510 is further configured to:

[0197] Determine whether the preset emoticon package contains a preset emoticon package code;

[0198] In the case where the preset emoticon package includes the preset emoticon package code, the preset emoticon package code is used as the preset text description information;

[0199] In the case that the preset emoticon package does not include the preset emoticon package code, the label information corresponding to the preset emoticon package is obtained through a pre-trained label recognition model, and the label information is used as the preset text description information.

[0200] In some embodiments, the type of the preset emoticon package includes an image-text combined emoticon package; the emoticon package library creation module 510 is further configured to:

[0201] Get the text information in the preset emoticon package;

[0202] The text information is used as the preset text description information.

[0203] In some embodiments, the type of the preset emoticon package includes a dynamic picture emoticon package; the emoticon package library creation module 510 is further configured to:

[0204] Get the frame text description information corresponding to each frame of the picture in the preset emoticon package;

[0205] The preset text description information is determined according to the plurality of frames of text description information.

[0206] In some embodiments, the apparatus further comprises:

[0207] The model training module 511 is configured to:

[0208] Acquire an emoticon sample set and a text sample set, wherein the emoticon sample set includes a plurality of emoticon samples, sample text description information corresponding to each of the emoticon samples, and an emoticon sample matching value corresponding to each of the emoticon samples, and the text sample set includes a plurality of groups of text dialogue samples and a text sample matching value corresponding to each group of the text dialogue samples;

[0209] The target neural network model is trained by using the emoticon sample set and the text sample set to obtain the message matching model.

[0210] In some embodiments, the model training module 511 is further configured to:

[0211] According to multiple emoticon samples, multiple sample text description information, multiple emoticon sample matching values, multiple groups of text conversation samples and multiple text sample matching values, the target neural network model is iteratively trained through the preset loss function of the target neural network model to obtain the message matching model.

[0212] In some embodiments, the candidate expression package determination module 502 is further configured to:

[0213] Determine whether the user message is an emoticon package;

[0214] In the case where the user message is an emoticon package, determining emoticon text description information corresponding to the user message;

[0215] According to the semantic information corresponding to the emoticon text description information, determining a plurality of candidate text description information matching the semantic information from the preset emoticon package library;

[0216] The emoticon package corresponding to each candidate text description information is used as the candidate emoticon package.

[0217] Through the above device, when obtaining the target emoticon package matching the user message, the candidate text description information corresponding to the candidate emoticon package is combined. In this way, the accuracy of the matched target emoticon package is higher, thereby improving the accuracy of the message reply and enhancing the effect of human-computer interaction.

[0218] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0219] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, which implement the steps of the message reply method provided by the present disclosure when the program instructions are executed by a processor.

[0220] Figure 7 is a block diagram of an electronic device 700 according to an exemplary embodiment of the present disclosure. For example, the electronic device 700 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0221] Reference Figure 7 , the electronic device 700 may include one or more of the following components: a processing component 702 , a memory 704 , a power component 706 , a multimedia component 708 , an audio component 710 , an input / output (I / O) interface 712 , a sensor component 714 , and a communication component 716 .

[0222] The processing component 702 generally controls the overall operation of the electronic device 700, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 702 may include one or more processors 720 to execute instructions to complete all or part of the steps of the above-mentioned message reply method. In addition, the processing component 702 may include one or more modules to facilitate the interaction between the processing component 702 and other components. For example, the processing component 702 may include a multimedia module to facilitate the interaction between the multimedia component 708 and the processing component 702.

[0223] The memory 704 is configured to store various types of data to support operations on the electronic device 700. Examples of such data include instructions for any application or method operating on the electronic device 700, contact data, phone book data, messages, pictures, videos, etc. The memory 704 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0224] The power component 706 provides power to the various components of the electronic device 700. The power component 706 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 700.

[0225] The multimedia component 708 includes a screen that provides an output interface between the electronic device 700 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 708 includes a front camera and / or a rear camera. When the electronic device 700 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0226] The audio component 710 is configured to output and / or input audio signals. For example, the audio component 710 includes a microphone (MIC), and when the electronic device 700 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 704 or sent via the communication component 716. In some embodiments, the audio component 710 also includes a speaker for outputting audio signals.

[0227] I / O interface 712 provides an interface between processing component 702 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0228] The sensor assembly 714 includes one or more sensors for providing various aspects of status assessment for the electronic device 700. For example, the sensor assembly 714 can detect the open / closed state of the electronic device 700, the relative positioning of components, such as the display and keypad of the electronic device 700, and the sensor assembly 714 can also detect the position change of the electronic device 700 or a component of the electronic device 700, the presence or absence of user contact with the electronic device 700, the orientation or acceleration / deceleration of the electronic device 700, and the temperature change of the electronic device 700. The sensor assembly 714 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 714 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 714 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0229] The communication component 716 is configured to facilitate wired or wireless communication between the electronic device 700 and other devices. The electronic device 700 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 716 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 716 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0230] In an exemplary embodiment, the electronic device 700 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned message reply method.

[0231] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 704 including instructions, and the instructions can be executed by a processor 720 of an electronic device 700 to complete the message reply method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0232] In another exemplary embodiment, a computer program product is further provided. The computer program product includes a computer program that can be executed by a programmable device. The computer program has a code portion for executing the above-mentioned message reply method when executed by the programmable device.

[0233] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the present disclosure. This application is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0234] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A message reply method, characterized in that: The method comprises: Get user messages in human-computer dialogue; Determining a plurality of candidate emoticon packages from a preset emoticon package library according to the semantic information corresponding to the user message; Inputting a plurality of candidate emoticon packages, candidate text description information corresponding to each candidate emoticon package, and the user message into a pre-trained message matching model to obtain an emoticon package matching value corresponding to each candidate emoticon package; Determining a target emoticon package from a plurality of candidate emoticon packages according to the emoticon package matching values ​​corresponding to the candidate emoticon packages; Output the target emoticon package to reply to the user message; The message matching model is trained in the following way: Acquire an emoticon sample set and a text sample set, wherein the emoticon sample set includes a plurality of emoticon samples, sample text description information corresponding to each of the emoticon samples, and an emoticon sample matching value corresponding to each of the emoticon samples, and the text sample set includes a plurality of groups of text dialogue samples and a text sample matching value corresponding to each group of the text dialogue samples; The target neural network model is trained by using the expression package sample set and the text sample set to obtain the message matching model; The target neural network model is trained by the expression package sample set and the text sample set to obtain the message matching model, which includes: According to the multiple emoticon samples, the multiple sample text description information, the multiple emoticon sample matching values, the multiple groups of text conversation samples and the multiple text sample matching values, the target neural network model is iteratively trained through the preset loss function of the target neural network model to obtain the message matching model.

2. The method according to claim 1, characterized in that: Before outputting the target emoticon package, the method further includes: Determining a plurality of candidate text messages from a preset corpus according to the user message; Inputting a plurality of the candidate text messages and the user message into the message matching model to obtain a text matching value corresponding to each of the candidate text messages; Determine a target reply message from a plurality of candidate emoticons and a plurality of candidate text messages according to the plurality of emoticon package matching values ​​and the plurality of text matching values, wherein the target reply message includes a target emoticon package or a target text message; The outputting the target expression package comprises: When the target reply message is the target emoticon package, the target emoticon package is output.

3. The method according to claim 2, characterized in that The method further comprises: In a case where the target reply message is the target text message, the target text message is output.

4. The method according to claim 2, characterized in that: Determining a target reply message from a plurality of candidate emoticons and a plurality of candidate text messages according to the plurality of emoticon package matching values ​​and the plurality of text matching values ​​comprises: Determine a maximum matching value among a plurality of the emoticon matching values ​​and a plurality of the text matching values; The target candidate emoticon package or target candidate text message corresponding to the maximum matching value is used as the target reply message.

5. The method according to claim 1, characterized in that The preset expression package library is pre-created in the following way: Get the preset emoticon package; Determining, according to the type of each preset emoticon package, preset text description information corresponding to the preset emoticon package; A correspondence between each of the preset emoticon packages and the preset text description information is created to obtain the preset emoticon package library.

6. The method according to claim 5, characterized in that The types of the preset emoticon packages include static picture emoticon packages; The step of determining the preset text description information corresponding to the preset emoticon package includes: Determining whether the preset emoticon package contains a preset emoticon package code; In the case where the preset emoticon package includes the preset emoticon package code, using the preset emoticon package code as the preset text description information; In the case that the preset emoticon package does not include the preset emoticon package code, label information corresponding to the preset emoticon package is obtained through a pre-trained label recognition model, and the label information is used as the preset text description information.

7. The method according to claim 5, characterized in that The types of the preset emoticon packages include emoticon packages combining pictures and texts; The step of determining the preset text description information corresponding to the preset emoticon package includes: Obtaining text information in the preset emoticon package; The text information is used as the preset text description information.

8. The method according to claim 5, characterized in that The types of the preset emoticon packages include dynamic picture emoticon packages; The step of determining the preset text description information corresponding to the preset emoticon package includes: Obtaining frame text description information corresponding to each frame of the image in the preset emoticon package; The preset text description information is determined according to the plurality of frame text description information.

9. The method according to any one of claims 1 to 8, characterized in that: The step of determining a plurality of candidate emoticon packages from a preset emoticon package library according to the semantic information corresponding to the user message comprises: Determine whether the user message is an emoticon package; In the case where the user message is an emoticon package, determining emoticon text description information corresponding to the user message; According to the semantic information corresponding to the emoticon text description information, determining a plurality of candidate text description information matching the semantic information from the preset emoticon package library; The emoticon package corresponding to each candidate text description information is used as the candidate emoticon package.

10. A message reply device, characterized in that: The device comprises: A user message acquisition module, configured to acquire user messages in a human-computer dialogue; A candidate emoticon package determination module is configured to determine a plurality of candidate emoticon packages from a preset emoticon package library according to semantic information corresponding to the user message; An emoticon package matching value acquisition module is configured to input a plurality of candidate emoticons, candidate text description information corresponding to each candidate emoticon package, and the user message into a pre-trained message matching model to obtain an emoticon package matching value corresponding to each candidate emoticon package; An expression package determination module is configured to determine a target expression package from a plurality of candidate expression packages according to expression package matching values ​​corresponding to the candidate expression packages; An emoticon package output module, configured to output the target emoticon package to reply to the user message; The device also includes: The model training module is configured to obtain an emoticon sample set and a text sample set, wherein the emoticon sample set includes a plurality of emoticon samples, sample text description information corresponding to each of the emoticon samples, and an emoticon sample matching value corresponding to each of the emoticon samples, and the text sample set includes a plurality of groups of text dialogue samples and a text sample matching value corresponding to each group of the text dialogue samples; The target neural network model is trained by using the expression package sample set and the text sample set to obtain the message matching model; The model training module is also configured to iteratively train the target neural network model through a preset loss function of the target neural network model according to multiple emoticon samples, multiple sample text description information, multiple emoticon sample matching values, multiple groups of text conversation samples and multiple text sample matching values ​​to obtain the message matching model.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 9 are implemented.

12. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 9.

Images